The Future of Open Source

William Gibson has a concept of the future in relation to science-fiction, which he’s mentioned on many occasions in slightly different forms. My favorite source is the earliest one I can find, which isn’t quite as quotable as the more polished form on Gibson’s Wikipedia page, but it has a more authentic ring of roughness to it, as if he articulated the idea on-the-fly during the interview:

The future is already here and it is very unevenly distributed and it arrives in bits and pieces constantly.
– William Gibson, “There’s no future in Sci-Fi”, London Sunday Express (April 2, 2000)

The future isn’t a binary switch, where suddenly, someday we’ll find ourselves “bang!”, in the future. The future is a progression over time, and while it’s impossible to predict the future with 100% accuracy, we see the “bits and pieces” of our future all around us. Sometimes it’s hard to figure out which bits will be significant, and which will simply fall by the wayside. But, significance develops over time too, so by looking back far enough we can trace which bits increase in significance and which decline, and the apparently random distribution begins to develop a pattern.

Lately I’ve been applying this idea to the industry that’s grown up around software, and especially free software/open source, sort of looking though the past to see the future. In the process, I’ve hit on an analogy from physics that provides a decent framework for exploring the history of software and the distribution of innovation over time. It’s based on the concept of an object traveling faster than the speed of sound. Similar principles apply for a particle traveling faster than the speed of light, but the speed of sound is generally easier for people to relate to.

First Age of Software

The early days of software, from about the 1940s to the 1970s, were like traveling at subsonic speeds. Sound naturally travels in waves, outward from the source of noise in all directions. To visualize this, you can picture a pebble dropped into a still pond, and how the water ripples outward. By analogy, innovation in software was initially slow and happened in scattered pockets. An innovation strike is the bit where “the future is here”, while the ripples in waves out from the center are the uneven distribution of that “future” into the wider world.

In the First Age of software, the pace of innovation was slow and regular. The tech industry didn’t value software terribly highly, regarding it as a mere extension of hardware.

“There was little or no interest in protecting software technology separately, because patent protection for computer hardware adequately rewarded innovation. […] When computer programmers first approached the U.S. Copyright Office about protecting programs by copyright law in the mid-1960s, the Office expressed some doubts about whether machine-executable forms of programs could be protected by copyright law on account of their utilitarian nature.”
– Software & Internet Law, 4th Edition, page 31-32

All software was “free and open” in this age, because there was no legal restriction on copying, modifying, and redistributing it.

Back to our analogy: as an object accelerates, the pattern of sound waves changes. The ripples are no longer concentric circles, because each wave travels outward from the object at a specific point in time and space, which the object leaves behind as it travels onward. The fundamental dynamics of flight are the same across a range of subsonic speeds, but the “bunching” of sound waves in the direction of travel hints at what might come next. Similarly, toward the end of the First Age of software, several bits increased in significance, accelerating toward what was to come in the next age.

The U.S. Copyright Office started to issue copyright registrations under the “rule of doubt” in the 1960s, but uptake was quite slow.

“Between 1966 and 1978, only about 1,200 copyright registration certificates were issued [for software].”
– Software & Internet Law, 4th Edition, page 32

Public sentiment started to shift, and while sharing software freely was still perfectly legal, some people thought it shouldn’t be.

“As the majority of hobbyists must be aware, most of you steal your software. Hardware must be paid for, but software is something to share. Who cares if the people who worked on it get paid?”
– Bill Gates, An Open Letter to Hobbyists (February 3, 1976)

In 1974, U.S. Congress established the Commission on New Technological Uses of Copyrighted Works (CONTU) as part of the review for drafting the U.S. Copyright Act of 1976. CONTU’s final report in 1978 concluded that software should be copyrightable, but too late to be included in the 1976 Act, which was finalized October 19, 1976 and went into effect January 1, 1978.

“A majority of the CONTU Commissioners concluded that computer programs already were copyrighted under the Copyright Act of 1976, but it recommended some changes to this Act to make appropriate rules for programs.”
– Software & Internet Law, 4th Edition, page 32

This recommendation can be regarded as the tipping point for software copyright, but apparently the tech industry took a “wait and see” approach to it, since no copyright infringement cases were filed for software until after U.S. copyright law was officially amended.

Middle Age of Software

A funny thing happens when an object accelerates to the speed of sound (also called Mach One). Because the sound waves are traveling at the exact same speed as the object, they keep pace with it, and the longer the object flies at Mach One, the more overlapping sound waves build up around it, creating a massively turbulent zone. For a long time, people thought it was utterly impossible to fly faster than the speed of sound. They called that leading edge of overlapping waves the “sound barrier” and pilots died trying to break it. Traveling at Mach One is analogous to the rush to capitalize on software copyrights in the 1980’s and 1990’s, and the sound barrier corresponds to an area of disruption where business models based on licensing proprietary software can succeed. The calmer area behind the turbulent zone is the massive body of free and open software that persisted and grew, despite all predictions to the contrary.

The defining moment of the second age of software was in one sense quite a tiny thing, no more than a few small changes (pages 14-15) to the lengthy title 17 of the United States Code, enacted on December 12, 1980.

“In 1980, Congress expressly incorporated protection for computer programs into the Copyright Act.”
– Intellectual Property in the New Technological Age, 6th Edition, page 433

But, those few words had a radical impact on the software industry. What followed was a period when companies eagerly and actively sought to enforce their new rights, through litigation or direct action.

“The first questions about copyright for computer programs involved the simplest form of copyright infringement – direct copying of the program code. […] the disputed issue in these early cases was whether computer programs could be protected by copyright at all. The first major case confronting this issue involved Apple Computer.”
– Software & Internet Law, 4th Edition, page 33

Apple Computer, Inc. v. Franklin Computer Corp., filed on May 12, 1982, was initially ruled in favor of Franklin, but Apple appealed successfully, establishing case law precedent for operating systems as software subject to copyright law. In the original ruling in favor of Franklin, the district court concluded that Apple Computer failed to show irreparable harm, stating “It is also clear that Apple is better suited to withstand whatever injury it might sustain during litigation than is Franklin to withstand the effects of a preliminary injunction.” In hindsight, the district court had a reasonable point, and in light of Apple’s statements that they already “had annual sales of $335,000,000 for fiscal year 1981”, it seems fairly unlikely that the case had any significant impact on Apple’s success and future growth as a business, or that Franklin’s continued business would have detracted from their profits in any meaningful way.

Over the same time period, another story arc was unfolding, which was barely noticed at the time, but had a far greater impact on the course of software history. Richard Stallman, working at MIT’s AI Lab, was surprised by an encounter with changing attitudes toward software ownership, inconveniently preventing him from porting a fix to send out paper jam notifications from an old printer at the lab to a newer printer.

“In the course of looking up the Xerox laser-printer software, however, Stallman made a troubling discovery. The printer didn’t have any software, at least nothing Stallman or a fellow programmer could read. Until then, most companies had made it a form of courtesy to publish source-code files–readable text files that documented the individual software commands that told a machine what to do. Xerox, in this instance, had provided software files only in compiled, or binary, form.”
– Free As in Freedom (2.0), page 4

Richard tried to gain access to the source code, reaching out to academic colleagues as he would have done in the past. He was unsuccessful, as those who had the source code declined his request for a copy, citing a contractual agreement with Xerox as the reason. As surprise gave way to thoughtful reflection, Richard began to shape the seed of an idea that would grow to become his life-long passion.

“I already had an idea that software should be shared, but I wasn’t sure how to think about that. My thoughts weren’t clear and organized to the point where I could express them in a concise fashion to the rest of the world. After this experience, I started to recognize what the issue was, and how big it was.”
– Richard Stallman, Free As in Freedom (2.0), page 9

Beginning quite informally with comments on “communal sharing” in a paper about Emacs, he fleshed out his ideas over the course of several years, leading to the announcement of the GNU Project in 1983; the publication of the GNU Manifesto, formation of the Free Software Foundation, and first release of the GNU Emacs License in 1985; and a generalized version of the Emacs license (and other variants for the GNU C compiler and debugger) dubbed the GNU General Public License in 1989.

In another parallel story arc, in 1977 UC Berkeley released a mixture of their own and code from AT&T Unix, under the name Berkeley Software Distribution (BSD). They released before software copyrights existed, so imagine their surprise 15 years later, when a subsidiary of AT&T filed a lawsuit against against them for copyright infringement. The judge denied the injunction, and the case was settled out of court.

Notable milestones in proprietary software in the Middle Age include the first release of the Oracle database in 1979 (rewritten in C in 1983); the first release of MS-DOS in 1981; the first release of Norton Utilities in 1982; the first releases of IBM DB2, Lotus 1-2-3 , and Microsoft Word in 1983; the first releases of Apple’s Macintosh operating system, MacPaint, MacWrite, MacDraft, AppleWorks, and Quicken in 1984; the first releases of Microsoft Windows, Aldus (later Adobe) PageMaker, and StarOffice in 1985; the first releases of Adobe Illustrator and OS/2 in 1987; the first release of NeXTSTEP in 1989; the first release of Adobe Photoshop in 1990; the first release of NCSA Mosaic web browser in 1993; the first release of Netscape Navigator in 1994; the first releases of Microsoft Internet Explorer, Sun Java , and Netscape JavaScript, and launch of Amazon.com in 1995; the first release of Macromedia (later Adobe) Flash in 1996; the launch of Google search in 1997, the first release of Netscape Communicator in 1997; and the first release of Mac OS X Server in 1999.

Notable milestones in free software in the Middle Age include the first release of LaTeX in 1984; the first releases of GCC and Perl, and the Artistic License in 1987; the BSD and MIT licenses, the first release of POSTGRES (later PostgreSQL), and commercial support for free software by Cygnus in 1989; the first release of CERN httpd in 1990; the first releases of the Linux Kernel and Python in 1991; the first releases of Debian, FreeBSD, NetBSD, and Lua, and commercial sales of machines pre-installed with a free software operating system by VA Linux in 1993; the first release of Red Hat in 1994; the first releases of MySQL, the Apache HTTP Server, PHP, and Ruby in 1995; the first releases of GIMP and KDE  in 1996; the first release of the Mozilla Application Suite (formerly Netscape Navigator & Communicator) in 1998; and the first releases of GNOME, Asterisk , and CUPS in 1999.

The Middle Age of software was characterized by proprietary software at the leading edge of software innovation, and free software playing catch up, combined with a belief that the world would always be this way. No one considered the possibility that the introduction of software copyrights might have induced a unique set of conditions, sustained for a time but fading and not reproducible. In the latter half of the Middle Age, more and more new software innovations were built on free software or as free software, and yet conventional wisdom still held that business profiting from free software was an oddity. Ultimately, the golden age of proprietary software lasted a grand total of about 15 years, a mere blip on the scale of software history.

Modern Age of Software

The most important thing to understand about the sound barrier is that it doesn’t actually exist. The turbulence experienced at Mach One is really no more than a convergence of transition effects between subsonic speeds and supersonic speeds. With advances in technology, “breaking the sound barrier” quickly shifted from deadly to merely dangerous, and eventually became so easy it wasn’t even a noticeable disruption in flight. Once an object accelerates past Mach One, it travels faster than the sound waves it creates, and so the waves fall behind the object, creating an outward spreading cone. The compressed sound waves at the outer edge of the cone form a zone called the “shock wave”, which is experienced by observers as a sonic boom (comparable to the noise level of firecrackers) as the edge passes by them. You can visualize this like a wake stretching out behind a fast boat. In software, the shock wave is the same disruptive zone we saw in software’s Middle Age, where business models based on proprietary software can succeed. But at a supersonic pace of innovation, this zone is much thinner, and presents no barrier to open solutions overtaking and surpassing proprietary solutions.

There was no single defining watershed moment at the start of the Modern Age of software, like there was for the start of the Middle Age, but a convergence of factors places the beginning sometime around the year 2000. That doesn’t mean the final outcome was obvious to everyone in the year 2000, but in hindsight we can approximately place the point at which the future was “here” in 2000, and characterize what followed as a process of evening out the distribution and accelerating the pace.

• While it’s tempting to identify the open source movement as a catalyst for the Modern Age of software, it’s more precisely accurate to say that events and conditions surrounding the start of the Modern Age were the catalysts for the open source movement, in much the same way that events and conditions surrounding the start of the Middle Age were the catalysts for the free software movement. What was a significant factor in the start of the Modern Age was that attitudes started to shift: corporations were more open to using free software, and many free software projects were more open to corporate involvement. A small group of people in favor of this shift chose the name “open source” and it gained popularity surprisingly quickly. But under any other name (“collaborative software”, “shared software”, “modern software”, or simply continuing under “free software”), the outcome would have been nearly identical. The market forces driving the start of the Modern Age were broad, deep, and strong.
• It was difficult to predict the outcome of Netscape’s underdog maneuver of releasing their Navigator web browser as open source through the Mozilla Organization in 1998. Mozilla’s announcement 7 months later that they’d be scrapping Netscape’s codebase further clouded the issue. But looking back, the beginning of the end was clear in Internet Explorer’s slowed growth from 1999-2001 and market share peak at 96% in 2002, followed by a subsequent decline to 10% today.
• The burst of the dot-com bubble in 2000 and subsequent evaporation of lush VC funding, meant that paying for proprietary software licenses to programming languages, databases, web servers, etc. simply wasn’t a viable option for the vast majority of startups, especially in the face of stable, reliable free software/open source alternatives already used extensively in production environments.
• Apple transitioned their flagship operating system to a free software/open source base over 1999-2001, just one indicator out of many of the shifting attitudes.
• 20 years of experience with proprietary software was enough to see and begin articulating its practical disadvantages. Since the free software movement started at pretty much the same time as proprietary software, it could predict the failure modes, but its (impressively accurate) predictions were unfortunately often dismissed by those with high hopes for making big profits from software copyrights. The start of the Modern Age was littered with catchy aphorisms about collaborative, dynamic, faster-paced development models: “given enough eyeballs, all bugs are shallow”, “scratching a developer’s personal itch”, and “users as co-developers” for “rapid code improvement and effective debugging” (Eric Raymond, The Cathedral & the Bazaar). The subsequent 15 years have shown that the benefits of open development models aren’t automatic. It takes work to build a healthy collaborative community, and simply slapping a free software/open source license on a chunk of code and tossing it over the wall rarely succeeds. But the fact remains that proprietary development models exclude any possibility of collaborative benefits. The fundamental nature of proprietary software restricts its own potential for growth.
• The rise of software patents from 1995 onward was regarded as a potential threat to free software and open source, and lead to the addition of explicit patent license clauses in the Mozilla Public License 1.1 in 1999, the Apache License 2.0 in 2004, the Artistic License 2.0 in 2006, and the GNU General Public License 3.0 in 2007. Ironically, it turns out that one of the safest ways to drive forward rapid innovation in a space that’s heavily patented by multiple companies, is to gather as many of those companies as possible, license their patents into a free software/open source project, and run along with collaborative development while effectively ignoring patents, on the strength that all the companies together own far more patents in the space than any single company outside the pool.

The period from 2000 to 2005 was marked by growing acceptance of a new way of looking at software. Fearful predictions that open source would undermine the software industry, gave way to recognition of open source as a new model of decentralized economic production. The message of free software and open source didn’t change, but more and more people were repeating it. In 2006, Tim O’Reilly predicted that by 2010 open source would “be part of every business and every company’s strategy”. And indeed, 2010 saw the rise of the meme that “Open Source Won”.

After the celebrations died down, the open source movement suffered from anti-climax: If the true goal of open source was “enablement of a commercial industry”, then what’s left to do after you succeed? In recent years this has been followed by a form of low-grade disillusionment: If we won, why are we still repeatedly explaining how open source collaboration works? The answer to both questions is that corporate adoption was never the sole defining goal of the open source movement, it was only a significant milestone along the way. The true goal of the open source movement is encapsulated in the Open Source Definition, advocating for the freedom to use, modify, and redistribute software, and the benefits of exercising that freedom. If this sounds stunningly similar to the philosophy of the free software movement, it should come as no surprise: the Open Source Definition is a slightly modified copy of the Debian Free Software Guidelines, which were drafted as an explicit expression of the philosophy of free software. That doesn’t mean the two movements are identical, just very, very similar.

To a large extent, the free software movement was unaffected by the open source identity crisis. Free software’s focus from the beginning was on the freedom to use, modify, and redistribute software, and while it offered several ideas around profiting from free software using business models based on support or services, attracting corporations was never one of its goals. However, the free software movement hasn’t been excluded from the benefits of growing acceptance in the Modern Age of software. While it’s only one indicator out of many, an easily accessible form of public hard data for U.S. non-profits is their tax returns, which show that the Free Software Foundation saw an increase in annual revenue from $376k in 2001 to $1.25m in 2012. (Even though it’s really comparing apples to oranges, I feel obligated at this point to share the same public information for the Open Source Initiative, which saw an increase of annual revenue from $25k in 2001 to $117k in 2014.)

Some feared that open source “winning” would undermine the work of the free software movement, by tempting people to settle for the lesser goal of corporate adoption,and lose sight of the ultimate goal of software freedom. I can’t find any evidence that this has turned into a significant trend. Anecdotally, over the past few years, I’ve seen quite a few open source developers do a bit of soul-searching and find their true roots in the free software movement, but I haven’t seen a single free software developer decide to give up on software freedom because free software and open source achieved corporate adoption.

In the period from 2010-2015 free software and open source continued to grow rapidly, without any concrete vision from the open source movement on what should happen next, and with the free software movement still firmly focused on a more distant future where all software is free software. This success is driven by a self-reinforcing cycle of economic necessity and acceleration of software innovation. The greater the body of stable, reliable free software and open source, the greater the pressure to use it and build on it. If your competitors are using and building on free software and open source, then avoiding it puts you at a competitive disadvantage, because they can innovate faster and cheaper through free reuse rather than wasting resources on reinventing wheels. Once the use of free software and open source is ubiquitous among a group of competitors, the game shifts and “use” becomes mere table stakes for entry. The new competitive edge is participation, because the companies who dedicate resources to fix bugs and add features in the projects they use, end up reducing their own cost of providing support and services for those projects, and by contributing changes back upstream they reduce their own cost of maintenance and integration of new releases. Greater contributions over time lead to more stable, reliable, and successful free software and open source solutions, which in turn increase the pressure for even more use, and even more participation.

That sounds like we’re done, right? Well, not yet, but we’re definitely headed in the right direction.

What Lies Ahead

Looking back over the past 70 years or so, some patterns emerge. The legal system is moving toward increasing restrictions on the intellectual property of software, with no sign of rescinding any significant portion of earlier restrictions or the accumulation of case law built on those restrictions, and every indication that new restrictions will continue to be added in the foreseeable future. The good news is that free software and open source have always found ways to overcome each increasing restriction, so the successive introduction of new intellectual property laws for software ownership hasn’t ultimately presented a serious obstacle to software freedom. In a surprising twist, the increasing restrictions have served to boost the growth of free software and open source, by making it more difficult to successfully innovate using a proprietary business model.

Over the next 20-50 years, we can expect to see an increasing number of technical innovations released initially as free software and open source. Of the technical innovations that are initially released as proprietary, we can expect an increasing number to be either undercut and sidelined by rapidly innovating open alternatives, or else released later as free software and open source by their creators to avoid being undercut and sidelined. We can’t say proprietary software is dead, and it’s likely to linger in one form or another for decades into the future. But the patterns of significant bits through history brand proprietary software as a less-than-healthy offshoot in the evolution of software business models, and the trend for proprietary software from the 1980’s to today is one of slow decline and increasing dependence on free software and open source to survive. Some companies attribute their success to proprietary software, but a deeper analysis tends to reveal their true success lies in some combination of other business models (support, services, integration, content, or hardware) that are compatible with free software and open source licensing, so the perceived effect of proprietary software licensing is a mirage.

We can expect to see an increasing number of companies who go beyond use, to participate in creating free software and open source. More and more of these companies understand the principles of software freedom, from the most junior developers to the most senior executives. Others are still only driven to adopt free software and open source out of economic necessity, and herein lies one of the great challenges of the next few decades. Inexperienced companies can cause a great deal of harm as they blunder around blindly in a collaborative project, throwing resources in ways that ultimately benefit no one, not even themselves. It is in our best interest as a community to actively engage with companies and teach them how to participate effectively, how to succeed at free software and open source. Their success feeds the success of free software and open source, which feeds the self-reinforcing cycle of accelerating software innovation.

We can expect to see an increasing diversity in the individuals who participate in free software and open source, not just social diversity (culture, geography, ethnicity, gender, etc) but also diversity of skills beyond development to things like graphic design, user experience, product design, and user support. This change is already happening, and we’re making good progress, but we can expect more growing pains in the coming years. Diversity isn’t a matter of us teaching them to fit in, it’s a matter of them teaching us what they need, and us giving them room to explore and find their own way to work with us. Lately, one of the places I’ve frequently seen the need for embracing diversity of skills is when multiple companies hire user experience and product people to work on the same free software/open source project, but the project doesn’t provide channels for them to collaborate across company boundaries, because they aren’t developers. The OpenStack project has started experimenting in collaboration for user experience and product skills, and I’m pleased with the results so far.

It’s looking promising that over the coming decades we’ll see increasing unification across the free software movement and the open source movement, under the banner of software freedom, or perhaps some other name. (Personally, I’ll gladly accept whatever name the two communities can agree on, and for the past decade that seems to be “software freedom”.) As free software and open source succeed and proprietary software continues to decline, the motivation for debating about tactics dissipates, leaving a growing motivation to collaborate around accelerating, solidifying, and celebrating our success.

(The illustrations of subsonic, Mach One, and supersonic pace of innovation are based on “Transonico” by Ignacio Icke, licensed under Creative Commons Attribution-ShareAlike 3.0 Unported.)