Here is an academic essay I did for university recently. It explores how the social elements of the web, via Github and others, have contributed to Open Innovation and specifically open source software. Enjoy.
Open Innovation meets Social Media:
The Rise of Social Coding
by Mark Gibaud
Introduction
This essay explores the new phenomenon of ‘social coding’ within the context of open innovation. Social coding is a radical evolution in open source software development caused largely by the convergence of traditional open source software development and the new social networking paradigm. In our analysis we begin by exploring the transition from closed innovation to open innovation that has taken place largely over the last decade. We then look the open source software movement, and specifically review the literature explaining the motivations of software developers for contributing to open source projects. We briefly review the rise of social networking on the internet before examining the current leading ‘social coding’ website, Github.com, and analyse how this website allows software developers to experience traditional OSS contribution motivations more acutely than before and thus sustain and even increase their open innovation activity. Finally, we consider the opportunities for non high-tech sectors and discuss the challenges that might be faced in transplanting this specific flavour of open innovation embodied by social coding into arenas outside of software engineering.
From Closed to Open
For much of the 20th century there was only one way to innovate. Firms would invest as many resources as they could afford into their own private R&D labs while simultaneously seeking to fill those labs with the best and brightest engineers and scientists in an effort to invent, develop and commercialize innovations that would yield profit and growth (Chesbrough 2003). They also aggressively managed intellectual property related to these products which would protect their revenues and in turn allow them to further invest in R&D and thus fund the next wave products, reinforcing a cycle of “closed innovation” (Chesbrough 2003). The prevailing school of thought was to have “innovation through total control” (Chesbrough 2003).
 |
| Figure 1: Closed Innovation (Chesborough 2003) |
But several changes in the economic landscape of the late 20th and early 21st centuries have induced an eroding effect of this model of innovation. Firstly, the steady and persistent emergence of the knowledge-powered economy (Drucker 1992) has dramatically reduced the chance of any single company owning even a majority of a particular industry’s leading thinkers, much less all. Secondly, Chesbrough (2006) cites the increase in quantity and employment mobility of knowledge workers as contributory to the eroding of closed innovation, as firms naturally find it more difficult to own their employees’ ideas when they no longer work for the same company for decades at a time, as was prevalent in the 20th century. Along with this, the rise of globalisation has meant that even within a specific technology or science, “knowledge is distributed” (Chesbrough 2003) and experts in a particular field may often find themselves on opposite sides of the planet employed by different companies or even in different sectors. Another major catalyst for open innovation was the growth of the venture capital market, which better enabled innovation spillovers from both corporate research labs and universities to become commodotized within startups and therefore serve the marketplace, whereby previously, an innovation that was not considered related to the core offering of a firm would’ve died out. For these reasons, it is increasingly recognized (Chesbrough 2003, von Hippel 2001) that supplementing internal R&D with knowledge from outside the boundary of the firm can give rise to commercialization prospects previously unavailable.
By the same token, organisations are more vigorously exploring efforts for internal innovations that may not relate intimately to their core offerings to follow novel paths to market and realise benefits other than profit for these products and services. For example, a firm could license out intellectual property (IP) for financial gain, contibute to a patent pool to access new technologies to use internally, or making technology freely available so as to encourage the market adoption of a standard (von Krogh and von Hippel, 2006). Organisations are finding out that the usual idea-commoditization route is only one way to capitalize from internal R&D, and that through exploring other ways to the market an organisation can benefit in novel ways.
 * |
| Figure 2: Open Innovation (Chesborough 2006) |
Chesbrough’s defined open innovation in his 2003 book “Open Innovation: The new imperative for creating and profiting from technology”:
“Open innovation is a paradigm that assumes that firms can and should use external ideas as well as internal ideas, and internal and external paths to market, as the firms look to advance their technology.”
- Henry Chesbrough (2006)
A definition that extends Chesbrough’s original by usefully including the notions of the learning organisation, knowledge management and organisational boundaries is provided by Ulrich Lichtenthaler:
“Open innovation is defined as systematically performing knowledge exploration, retention, and exploitation inside and outside an organization’s boundaries throughout the innovation process.”
- Ulrich Lichtenthaler (2010)
When considering open source software (OSS) development, as we do next, this definition better frames open innovation in this industry. This is because programmers routinely explore the world of OSS (which is external to the firm) for software that will satisfy their own product needs, retain the knowledge needed to integrate that project into their software ecosystem, and seek to exploit some value from that integration.
Open Source Software: Radical Open Innovation
No industry has embodied the principles of open innovation so passionately as the software product development industry. For decades, informal networks of professional and hobbyist computer programmers have sought to contribute to and participate in the communal creation of a great variety of software products in order to fulfil their own technological needs (von Hippel 2001). Two of the most successful open source projects are Apache Web Server and the Linux computer operating system (Hertel et al 2003). According to the Netcraft January 2011 Web Server Survey (2011), Apache has been the most popular server software in use since April 1996, and currently serves more than half of all websites on the internet and almost two thirds of websites with the highest internet traffic. The Linux open source project also demonstrates the viability of open source software (Hars and Ou 2001). The Linux operating system is one of the most popular server operating systems in the world (most Apache web servers installations run on Linux servers) and is selected to run on the fastest supercomputers in the world (Cray 2011). Open source software products are also dominating emergent industry areas in high-tech including cloud computing, for which most products (Cassandra, MongoDB, Hadoop) are licensed similarly to Apache.
There can be no doubt that in the wake of the success of these and other products, open source software development has become the flagship industry for the new open innovation paradigm.
But why do programmers give up their spare time to contribute to open source projects? From Maslow Hierarchy to Hetzberg’s Two Factor theory, much research has been done on motivation. Motivation has been broadly divided into internal and external motivation (Deci and Ryan 1995). Internal motivation is more likely to be present when individuals believe that they have the capabilities and skills to contribute (rather than being out of their depth), that their efforts effect the goal (rather than being determined by luck), and when they are interested in mastering a particular skill (rather than learning by rote to affect an entirely different outcome, like passing a test) (Deci and Ryan 1985). Examples of positive external motivations are various forms of financial rewards and increased power and influence, while the threat of punishment can serve as an external demotivation.
It is not surprising that open source programmers realise some forms of internal motivation. Firstly, there is a motivation to write code so as to become better at software programming, ie. pursuing mastery. Hars and Ou found that 70.9% of respondents to their OSS survey included “improving my programming skills” in reasons for participation (Hars and Ou 2001). Raymond (2001) found that the process whereby code is peer-reviewed before making it into the product provided programmers with learning opportunities further enhancing both their motivation to participate and the quality of the product. Hars and Ou also suggest that open source programmers are altruistic, as they derive motivation from “increasing the welfare of others”, specifically the users other than themselves of the product that they are contributing to. Another motivation outlined was the sense of belonging to a community.
Open source projects also benefit from the external motivations of contributors. Programmers who submit code are quite usually improving a product that they themselves need and use (von Hippel 2001), so they would naturally benefit from using a better product. Recognition from a community would also serve as motivation, and programmers experience rewards as they increase their marketability should they publicize their contributions (Hars and Ou 2001). Lakhani and Wolf (2003) further confirm the presence of these motivations as they also found that programmers pursue the enjoyment of being creative when programming software, with 44.9% of respondents to their survey indicating that “Project code is intellectually stimulating to write”. These programmers also have a strong sense of community identification with the OSS movement, and are motivated to increase their marketability as they become more attractive to firms that are themselves heavy users of open source products (Lakhani and Wolf 2003).
But a new evolution of open source software development is emerging. A form driven by the same social networking constructs as we have seen at the core of the success enjoyed by such websites as Facebook and Twitter. The era of ‘social coding’ has begun and is energising open innovation in software engineering at a scale heretofore unseen by increasing and improving the realisation of the above motivations through new social web mechanisms.
Social Media
The previous decade has been described as the decade of social media (Wide Angle 2011). As of January 2011, Facebook had over 600 million registered users (MSNBC 2011). As of February 2010, Twitter processes over 50 million “tweets” per day (TechCrunch 2010).YouTube is the third most-visited site on the internet (according to Alexa.com) and users upload 35 hours of video every minute (YouTube 2010). There can be no doubt that people connect and share using the internet now more than ever before, whether in personal or professional capacities, or even as ambassadors for the brands of their organisations (Kaplan and Haenlein 2010). However, innovation in social media is by no means destined to plateau any time soon. The current flagship websites of the social media phenomenon all cater to mass-market needs, eg. friends (Facebook), information-sharing (Twitter) and video-sharing (YouTube). The next phase in the evolution of social media will be the emergence of websites that cater to more specialized needs. For example, sharing specific types of information which are more intricate in their content, like architectural blueprints or perhaps 3D wireframes of model aeroplanes. Let us explore how one website has achieved this for software programs.
Github
Github (http://www.github.com) (see Addendum A) launched in February 2008 as a web-based source code hosting service for computer programmers using the increasingly popular Git source control system. As of 2009 it was the most popular online hosting service for the Git source control system (which was originally created by Linus Torvalds, the creator of the Linux computer operating system and related open source project). In July 2010 Github hit the milestone of 1 million hosted repositories (Techcrunch 2010) clearly indicating that open source has come a long way since von Krogh and von Hippel reported in 2006 with admiration that SourceForge.net, also a source code hosting site, hosted “more than 110 000 projects” (von Krogh and von Hippel, 2006). Github has quickly become a key part of the open source software development ecosystem, hosting the codebases for the new hot software development platform Ruby on Rails, the world’s most popular javascript framework jQuery, and even the Linux kernel itself. Even engineering teams from the likes of Facebook and Twitter have chosen Github to host technologies that these companies have decided to open source.
The typical modus operandi of Github is thus: A developer signs up for an account and can then fork an existing project (see Addendum B), copying a complete repository of that project to his own account space online (or create a new repository and upload his own project to that repository in his account space. After copying the repository content to his own workstation from his Github account space, a developer could then make a few changes to the source code on his own computer and then “push” these changes to his project repository in his account space. Finally, he can issue a “pull request” (see Addendum C) to the original author (or current manager) of the project and that user will get notified immediately of a potentially valuable incoming changeset. An online code review and discussion can then take place (see Addenum D) before the original owner incorporates these new changes into the “master copy” of the project’s source code. Github enables all of these steps through an easy-to-use, socially-powered web interface in true Web 2.0 style.
 *
Figure 3: A typical Github developer’s workflow |
The team behind the design of Github have taken a number of decisions to greatly foster collaboration. First and foremost, the site charges users for wanting to host their source code privately, ie. in a private repository where only certain users may access the code. This in turn means that the vast majority of projects hosted on Github are open source by default, as users get “public” code hosting at no cost. Therefore, any user can fork these projects and begin contributing.
Github also redefined the concept of forking an open source project. Most research on open source software has concurred that forking a project is a harmful last-resort (Kogut and Metiu 2001), but Github has turned this phenomenon on it’s head by greatly encouraging developers to fork projects, innovate on the product and then issue pull requests.
But most critically, Github has a mature and developed social layer that is designed to make key OSS operations easier, quicker and more satisfying to do than ever before, and leverages key motivations of open source programmers as outlined earlier. Users have profiles with job descriptions and pictures, can “watch” projects they are interested in and “follow” other programmers activity. They can also mention usernames in discussions (for eg. @markgibaud) immediately notifying those users. Mechanisms like these greatly encourage community collaboration and increase participation behaviours, foster belonging, and enable the pursuing of reputational gains among developers. Users can easily and clearly engage in pull request discussions, providing and receiving feedback on code changes and thereby providing all participants with greater and more satisfying learning opportunities (Raymond 2001).
Github also continues to add new functionality that caters to the most common motivations of OSS developers. For example, it recently made Github user data available to other websites. Stack Overflow Careers is a website where thousands of programmers have created online curriculum vitaes and users can now incorporate a view of their Github activity (see Addendum E) directly to their online CV. This is a direct facilitation of the motivation of OSS developers to increase their marketability.
Github has been very successful in creating an active, motivated community of open source developers to collaborate and communicate through a social web application to be able to produce superior quality innovations in the high tech and software industries. It remains the goal of other industries to find ways to replicate this successful open innovation in their own sectors.
Challenges
When referencing the innovative practices that open source software displays, it has been propounded that “There is no reason to believe that those practices cannot spread to other areas of economic and social activity.” (Von Krogh and von Hippe,l 2006). This author disagrees. It will take very careful and considerate analysis of sectors that do not share a few key traits with OSS to allow them to enjoy open innovation as successfully as OSS.
OSS is a natural leader in open innovation because of a few factors. Firstly, the actual output of innovative activity, computer code, is highly distributable at negligible cost. Along with this, the components of contributions in open source software (textual differentials) are sufficiently defined and discrete so as to easily allow focused discussion on each individually, before that contribution is accepted into the product (or rejected). The fact that incremental innovations can be so readily appraised, discussed and accepted using a low-friction process through an easy-to-use social website is a major factor enabling high levels of open innovation in software development. It is not easy to intuit how more sophisticated forms of data representation, for example architectural blueprints or 3D models, can take on forms that allow the same kind of distributed online collaboration that OSS enjoys.
Software application development itself is also at an advantage when exploiting open innovation because of it’s inherently component-based nature. Solutions to software “problems” like database access, integration with Twitter.com and authentication mechanisms are discrete enough that they can be implemented without impacting other modules, thus making decisions regarding one problem independent from another. What this translates into is massively increased cross-pollination and collaboration, rather than developer communities forming ‘silos’ around particular technologies and this in turn facilitates innovation across the sector.
Lastly, one critical distinction between software development and most other sectors is that open innovation in software has traditionally occurred in non-competitive areas. For example, for the most part, banks don’t compete with other banks when it comes to database access, and airlines don’t compete with other airlines on their website login systems. This enables developers across organisations to freely help each other with solving these non-competitive problems, such that they can then spend more time on improving their respective organisations’ core offerings. It remains a challenge for organisations in other industries to more clearly separate their core and differentiated offerings from problems encountered by all players in a particular market. This author argues that the lessons learnt from open innovation literature could, for example, advise the airline industry to form a coalition to create a global flight-searching platform that all players would benefit from. Individual airlines should then focus on increasing the value of their differentiated offerings such as customer service, in-flight comfort and entertainment, etc.
Conclusion
In this essay we have reviewed how organisations are increasingly undergoing a change in their R&D regime from a closed model to a open model of innovation. At the forefront of this movement are the participants of the open source software industry who have been innovating across firm boundaries for decades. We also reviewed specific motivations as to why this behaviour is sustained. We discussed how, in parallel to the increase of open innovation, social media has enabled people from all over the world to communicate and collaborate in unprecedented ways and at unprecedented scale. We then looked at how the most popular source code hosting website leverages many popular social media mechanisms in order to facilitate OSS developers experiencing their motivations for contribution more acutely than ever before. Finally, we elucidated the challenges that other sectors face if they wish to replicate the open innovation success enjoyed by the OSS fraternity, including desegregating their core offerings from their non-competitive pursuits so as to better enable openly innovating on the latter.
Chesbrough, H. W, W. Vanhaverbeke, and J. West. 2006. Open innovation: Researching a new paradigm. Oxford University Press, USA.
Cray Inc., The Supercomputer Company – Products – XT – XT5. http://www.cray.com/Products/XT/Systems/XT5.aspx.
Deci, Edward L., and Richard M. Ryan. 1985. Intrinsic motivation and self-determination in human behavior. Springer, August 31.
Drucker, P.F., 1992. The Age of Discontinuity 2nd ed., Transaction Publishers.
Eclipse Community Survey 2010 http://www.eclipse.org/org/community_survey/Eclipse_Survey_2010_Report.pdf.
TechCrunch: GitHub Hits One Million Hosted Projects. http://techcrunch.com/2010/07/24/github-one-million/.
Git User’s Survey 2009 http://www.survs.com/WO/WebObjects/Survs.woa/wa/shareResults?survey=2PIMZGU0&rndm=678J66QRA2.
MSNBC.com: Facebook has 600 million users – Technology & science – Tech and gadgets – Business Insider – msnbc.com. http://www.msnbc.msn.com/id/40929239/ns/technology_and_science-tech_and_gadgets/.
Hars, A., and S. Ou. 2001. Working for Free? – Motivations of Participating in Open Source Projects. In Hawaii International Conference on System Sciences, 7:7014. Los Alamitos, CA, USA: IEEE Computer Society. doi:http://doi.ieeecomputersociety.org/10.1109/HICSS.2001.927045.
Hertel, G., Niedner S. and Herrmann S. 2003. “Motivation of software developers in Open Source projects: an Internet-based survey of contributors to the Linux kernel.” Research Policy 32 (7) (July): 1159-1177. doi:10.1016/S0048-7333(03)00047-7.
January 2011 Web Server Survey | Netcraft. http://news.netcraft.com/archives/2011/01/12/january-2011-web-server-survey-4.html.
Kaplan, A. M., and Haenlein, M. 2010. “Users of the world, unite! The challenges and opportunities of Social Media.” Business Horizons 53 (1): 59-68. doi:10.1016/j.bushor.2009.09.003.
Kogut, B., and A. Metiu. 2001. “Open-Source Software Development and Distributed Innovation.” Oxford Review of Economic Policy 17 (2): 248.
Lakhani, K., and Wolf, R.G. 2003. “Why Hackers Do What They Do: Understanding Motivation and Effort in Free/Open Source Software Projects.” SSRN eLibrary (September). http://papers.ssrn.com/sol3/papers.cfm?abstract_id=443040.
Raymond, E. S. 2001. The cathedral and the bazaar: musings on Linux and open source by an accidental revolutionary. O’Reilly & Associates, Inc.
TechCrunch: Twitter Hits 50 Million Tweets Per Day. http://techcrunch.com/2010/02/22/twitter-50-million-tweets-day/.
Von Hippel, E. 2001. “Learning from open-source software.” MIT Sloan management review 42 (4): 82–86.
Von Krogh, G., and E. von Hippel. 2006. “The promise of research on open source software.” Management Science 52 (7): 975.
Wide Angle » This Decade’s Social Media: The Game Layer. http://mlcwideangle.exbdblogs.com/2011/03/14/this-decades-social-media-the-game-layer/.
YouTube Blog: Great Scott! Over 35 Hours of Video Uploaded Every Minute to YouTube. http://youtube-global.blogspot.com/2010/11/great-scott-over-35-hours-of-video.html.
). All in all, only around 7 or 8 days of actual work across Spec/Dev/QA were needed, but this feature still took 23 real days (or 17 working days) to turn around. That means that over 50% of this task’s total duration, nothing was getting done on it! That seems like a problem worth solving.
I believe most of this waste is simply due to a lack of focus on the individual features as they go through the system, or what Lean describes as too much work-in-progress. Most of the time, an individual engaging in multi-tasking takes longer to do each task than if they’d done each task one at a time. In the same way, a development team working on too many features at the same time means that each task takes longer to complete than if there were fewer features in progress, principally because there is more waiting time between functional roles while complete work on all the other items being worked on. Research also shows that increased work-in-progress in a system also negatively affects quality.
voice commands to, like “Flights to Paris”, and it would use various travel APIs to conduct the relevant search. The panel discussion was entitled “What THack did to the industry” and Kevin elaborated on a series of THacks conducted around the world and what the outcomes were: a variety of sites and apps that integrated travel data to solve a (travel) business need. A THack is a developer day or weekend where developers come together to ‘build something cool’ using APIs made exclusively available to them without any commercial barriers to entry. Apparently THack 2011 was where Expedia announced that they were making their APIs publicly accessible, and today any developer can sign up and start building applications interacting with Expedia services. The overall feeling of the panel participants was that APIs are the future and that travel companies, especially tour operators with data available for their own products, need to make this data accessible via an API or risk being left behind by those operators that do. Hopefully iTropics can scratch this itch for the Travel Corporation with our new API version 3! Expedia also recommended creating a developer portal so that developers could ‘self-serve’ accessing the APIs, and it was interesting to see that they have partnered with 
I also found it useful to use a software tool called 
Why MoneyDashboard probably sucks
A few months ago, in my continuing quest to find a personal finance manager (UK equivalent of Mint.com) I registered on www.moneydashboard.com. After getting past the initial shock/horror of Silverlight, I tried to perform some basic tasks but found the site cryptic to figure out and difficult to use throughout subsequent attempts.
After forgetting about the site, I received an email a couple of weeks ago informing me of all their whizz-bang new features. Unfortunately for them I took it as a reminder to unsubscribe, and requested that they delete all my info. Today I received a confirmation email apologising for the delay and asking for any feedback. Turns out I was in a feedback kind of mood…
Hi [name],
Was I too harsh? Probably. I am sick of half-ass tech out there? Definitely.