The evolution of open source computer and the battle between advocates of open and closed source software are both fascinating. The outcome will shape the way we do computing.. Adapted from a talk given to the British Computer Society .

A story

In my spare time I've enjoyed messing around with music sequencing software. I've also occasionally had to produce music scores on computer. The software I've used most often for this is from a German company called Emagic. It was once called Notator, and then it was renamed Logic Audio. Logic Audio was developed for the PC and for the Mac. Of these, around 60% were Mac users and 40% PC users.

I say “were” because in July 2003 Emagic was purchased by Apple. Apple immediately announced that the PC version was no longer going to be developed. The company generously offered a discount for Logic Audio users who wanted to trade in their PCs for a Mac.

I must say I was surprised how angry this made me feel. After working with Logic Audio for 10 years or so, I'd become familiar and productive with it. Now the software was being scrapped, the likely reason being that Apple thought it should promote its own hardware rather than supporting a broader range of users. I fired off some futile emails, and purchased some alternative PC software called Cubase. It wasn't a critical business issue for me, just annoying. But I imagine for some in the music industry it was more than annoying, a costly problem whether they allowed themselves to be coerced into buying new computers, or scrapped existing skills and data in order to use different software. It is a horrible position to be in. And it nicely illustrates the dangers of software that is proprietary and closed source. The owner of the software had a de facto power over its users. This is why, for Richard Stallman and some others, it is an issue of freedom.

If Logic Audio had been both free and open source, then nobody could have stopped its continued development for PC users. Rather than being driven by one company's decisions on how to allocate its resources, its development would be organic. If a lot of users want a particular feature, they agitate for it or provide the resources for it to be developed. If there are few users, the project withers away naturally. Either way, the users have their fate in their own hands.

According to Glyn Moody's history of Linux, a book called Rebel Code, something a little like this happened to Richard Stallman in his early days. He'd been working on a system called ITS, and one day Digital ceased production of the only machine on which it ran. Years of work were seemingly rendered immediately obsolete. “I didn't want that kind of thing to happen again,” said Stallman. “The only way to prevent it from happening again was to write a portable system.”

So what is open source?

There's a fair amount of confusion over terms like open source, free, non-proprietary, standards-based and so on. The root of the problem is that software is stored electronically. The media on which it is stored is cheap, and trying to stop it being copied is like trying to catch the wind. So software is in essence intellectual property, which means its ownership is abstract, a matter for legislators and lawyers. It's actually easier to define closed source software than its opposite. Software is closed source if the source code is unavailable. All you are allowed to see is compiled binaries. If you want to make a change, well, maybe you could reverse-engineer it and patch it that way, probably breaking your license agreement in the process, but in essence you have to go back to its owner and negotiate or plead or agitate for the change. If you have a concern about some issue like security or predictability, you can't address it by poking around in the code. You have to take the owner's word for it.

By contrast, if the software is open source, the source code is available. How available, and what you are allowed to do with it, is another matter. Most software companies will, under the right circumstances, allow their users to see the source code. Often that is hedged around with non-disclosure agreements, and often the access is read-only. Sometimes you are only allowed to view the code on the owner's systems, and not to take your own copy away. At the other extreme is software where the source is posted on the Internet for all to see, and the owner or owners have waived their rights. You can read it, change it, redistribute it, or whatever you like. Of course it gets a bit complex if you change it slightly and then sell it. That sounds a bit like stealing. So most open source software has some kind of license that tries to spell out what you can and can't do, and to protect the interests of people who gave away their work. But if there's any sort of deal that lets you see the source, then you might get away with calling it open source.

The advocates of what we think of as the open source movement add further conditions before they regard software as open source. Some essentials are:

• The source is publicly available
• The software can be distributed freely
• You have the right to derive new works through modification

It's this last point that is the most awkward. Richard Stallman's original General Public License states that you are free to modify software and combine it with other libraries, but that all such extensions must also be distributed under the GPL. Commentators have described this kind of license as contagious, or if you particularly dislike it “viral”, in that it infects other software to which it is attached. Other open source licenses have appeared with less stringent conditions, not least the GNU Library or Lesser GPL, which allows open-source libraries to be used in proprietary applications. Richard Stallman doesn't much like it, because it's not so good for promoting free software; but developers like it, because it gives them freedom over how they distribute their software. The Apache license, to take another example, enforces hardly anything on its users. You can do what you like with the code; all it notes is that you can't call another product Apache without consent. There is an organization, called the Open Source Initiative, which lists licenses that it has approved as conforming to the spirit of Open Source software. There's also the somewhat arcane business of deciding which licenses are compatible with one another, in the sense that you could combine code under these different licenses and make a new product which breaching any terms. The nutshell summary is that:

• Open source is a broadly-defined term
• Open source does not mean free, but is closely associated with free software

OK, so what if I have a non-free library that I licensed from a software vendor, and I now link this library with software licensed under the GPL. I've now breached the terms both of the GPL and of the third-party library. However, others who acquire my software believe it to be covered by the GPL. It's a principle of law that you cannot pass on better title than you have, so we must assume that the third-party library does not pass magically into the public domain. Should such a thing ever happen, many honest developers are entrapped into breaching the terms of a license they know nothing of. What about a more subtle case, where a developer works on commercially licensed software by day, and on GPL software in his or her spare time? It's a common scenario. What if the developer consciously or unconsciously used algorithms developed and sold commercially, in the GPL software? It is a legal minefield, and I'm certainly no legal expert able to answer these questions. But let's put it another way. Is it likely that a system as complex as Linux, with contributions from hundreds of developers, contains no code anywhere that breaches someone's license?

A moment's thought shows that the same reasoning applies to commercial projects. Lawsuits are common. There is also I believe a degree of give and take. In software, developers continually build on the insights of others, whether making open or closed source software. But irregardless of the merits of its case, it is not really surprising to find a company like SCO, losing out in the Unix wars, jumping up and saying that some of its code has found its way into Linux. It is a cloud that hangs over the open source movement. I feel that in the end common sense will prevail, but not without pain.

The ethics of Open Source

There are a few people who believe that open source software is somehow morally superior to closed source software. Richard Stallman is one. To him, the issue is one of freedom versus slavery. Note that Richard Stallman's organization is called the Free Software Foundation, not the Open Source Software Foundation. On the home page of the web site, there's a reference to “ FSF's mission to preserve, protect and promote the freedom to use, study, copy, modify, and redistribute computer software.” This is missionary zeal. However, it's not the reason for the success of open source software. It is critically important that we think clearly about this. The real reason for the success of open source is not the missionary zeal of Stallman and others, but a more humdrum factor called commoditization. In industry, something is a commodity if it is widely available from many suppliers and relatively homogenous. Bread is a commodity, and so is a teabag and a kettle. The kettle is an interesting case, because it comes in different shapes, colours and sizes, but you're going to be reluctant to spend too much on a kettle because you know that ultimately they all do the same thing, which is to boil water. And this last point is critical. You can make money selling commodities, but not very much, because if you try to charge a premium someone else will quickly undercut you.

There is a process of commoditization happening in software as well. We know what constitutes the bread-and-butter of the computing world. Operating systems, web servers, email clients, database managers, spreadsheets, word processors, and so on. They may not be as easy to make as kettles, but the problems they pose are now well-known and they are available from multiple suppliers. What then should be their cost? Well, probably not very much. Consider a common tool like a word processor. How many people need a word processor? Almost everyone. So, if everyone who needs a word processor pools their resources, you can easily make a very good word processor which then belongs to everyone. It makes enormous sense, not in terms of politics or religion, but in sheer commercial terms. Put another way, it makes little sense for everyone individually to pay hundreds of pounds each for a word processor license, when the technology required to provide this piece of software is commonplace.

Of course, the problem with this sort of reasoning is that it requires organization and communication. The key enabler is the internet. This extraordinary network provides both communication and distribution. It enables the pooling of resources, management of source code, distribution of both code and binaries, and peer-to-peer support. With hindsight, something along the lines of Linux and Open Office has been inevitable ever since the Internet took off.

What we are seeing with the open source movement is commoditization of software to match what has already taken place with hardware. Ironically, Microsoft – my first mention of the M word – built its success in part on the commoditization of the PC. It forced down hardware prices, and make Microsoft's hardware platform by far the cheapest. What Microsoft conspicuously failed to do was to reduce the operating system and application cost in line with falling hardware prices. The result is an increasing discrepancy that strengthens the incentive to look for alternatives.

It is easy to see how well this logic applies to what you might call bread-and-butter software. Web servers, email clients, database managers, office productivity applications, even programming languages. However, it breaks down as software becomes more specialist. What about software for dental practitioners or estate agents, can they find or resource free software to meet their needs? Possibly not. And when it comes to custom software to manage specific business needs, again commoditization does not apply. Following that logic through, we should expect software essentials to be free or very cheap, but will still have to pay commercial rates for the skills to develop custom solutions. There's also the matter of knowing how to configure and support software. We should expect then that specialist software will always be costly, and that computer services are a good way to make money, but that making all your money by selling standard operating system or application software is going to be difficult.

Of course those companies who do make their money in such a way will do their best to resist the trend. They can do so in various ways. First, they can add so many premium features that you don't mind paying extra. In other words, the proprietary item offers more than the free equivalent. Vendors are doing this with varying degrees of success. I'd rather store my data on Oracle, DB2 or SQL Server than on MySQL, PostgresSQL or Firebird, the open source version of Borland's Interbase. However, if you just want a simple database server to run behind a small-scale web site, the benefits of the premium solution are zero. Equally, Microsoft works strenuously to keep Office ahead of its open-source competition. Last year I attended a press conference about the new release of Office, called Office 2003. I stuck my hand up and asked how the company intended to persuade users who simply typed letters and the minutes of meetings that the new features offered anything of value to them. The Office product manager was very offended, responding along the lines that “Office 2003 is not about writing letters and the minutes of meetings. It is about enhanced collaboration and improved productivity” – that's not an exact quote, but it is close. Unfortunately for Microsoft, a lot of people do just type simple documents. Put another way, if only those people who really use the advanced features purchased Office, Microsoft's business model would disappear.

The problem for all these vendors, whether Oracle, IBM or Microsoft, is that with these core products the open source community can quickly catch up. If enough people press for a particular feature, it will undoubtedly appear. My conclusion is that the downward pressure on prices for commodity software products will continue. Services on the other hand will remain a strong profit centre. If like IBM you make most of your money from services that's fine; it also happens to be the business model of open-source companies like Red Hat.

Software vendors have other strategies. The favourite is called vendor lock-in. What this means is that the customer might like to move to a different product, perhaps with a cheaper or free license, but they cannot because it is too costly. Database administrators and programmers suffer badly from this. Typically, moving complex database applications to another DBMS is difficult, because vendor-specific features and optimizations mean that queries and stored procedures have to be rewritten. In this case, you might say that the vendor lock-in is the features of the database manager that go outside standard ANSI SQL. Another aspect of vendor lock-in is in the guise of integration. Microsoft is a master of this, making sure that the Windows operating system, the Exchange messaging server and the Office productivity applications all work better together than any of its competitors can manage. It becomes hard to shift away to something different. There are also things like document compatibility. Many people use Microsoft Office because they know that everyone has to produce documents that Office can read, while the reverse may not be true. However, vendor lock-in never works over the long term. If there are better and cheaper solutions out there, eventually either you will find them or your competitors will.

The Microsoft Factor

Microsoft has an enormous influence on the open-source movement. Nothing has united the open-source world so much as a sense that they had a common enemy or rival, and rightly or wrongly Microsoft has fulfilled that role.

What then is wrong with Microsoft? Mainly I think the factors I've already mentioned, that it is the company that most obviously makes more money than it should from products that should be cheap or free commodities. I say “should” not in an ethical sense, but in an economic sense. However, it goes beyond this. Because Microsoft dominates the desktop, it is the visible face of computing, even if the back end is an IBM mainframe or an Oracle database running on Solaris. People have the impression that this one company is controlling everything. It's never been true, although in certain areas like Office applications it has come close. There's also a view that Microsoft's software is too buggy and insecure, which is very costly for the industry. The view has some merit, although equally Microsoft is not guilty of everything of which it is accused. Certainly, the company has made some curious decisions in the past, concerning things like scripts running in unsolicited emails, or hiding application extensions to make it easy for malicious attachments to pass themselves off as something else. Such decisions have come back to haunt them. On the other hand, I have no patience with the idea, fondly portrayed by open source advocates, that Microsoft somehow represents the dark side of the industry. The dark side, as I found to my cost with Logic Audio, is equally resident in Apple, while those who have to compete with IBM don't find the company as warm and cuddly as its generosity in giving the Apache Foundation all its web server code might suggest. The dark side is the part of the industry that wants to unfairly squeeze out competition, to snuff out innovation to protect its commercial interests, and to ratchet prices up when its customers seem to be hooked in beyond escape.

It is true though that the open source movement has been hugely beneficial. If people have a choice, then the competitive pressure drives up quality and drives down price. Undoubtedly, Windows 2003, for example, is a better and more secure product than it would have been without Linux eating away at its real and potential market share.

Practical points

Let me quickly address a few practical points concerning open source software. One common question is whether publishing the source code makes software less secure. The rationale for this is that by inspecting the source code, hackers can see exactly how to break it. I think this idea has been completely debunked. In fact, there is a problem with closed-source code, in that weaknesses are more likely to remain hidden because of the lack of peer review. The open source community has proved extremely responsive when security problems are discovered in widely used products such as, for example, Sendmail. Patches appear quickly and administrators are as quick or quicker to apply them, then their closed-source equivalents.

On the other hand, the internet is awash with half-completed open source projects some of low quality. By itself, open source status is no guarantee of quality.

Let me address another point. Is it possible to run your business entirely on open-source software? I can't speak for every business. What I can say is that you can easily assemble a full-featured business network, from database to file server to groupware to office applications, entirely with open source software and without compromising on quality.

So why don't more people do that? It's a good question, and I suppose the right answer, along with factors like vendor lock-in already mentioned, is that the closed source option is pretty good too. Some things, Exchange Server for example, have features that are hard to match. And while IT costs are important, it's also true that expenditure on software licenses is unlikely to be figure hugely as a proportion of most company's expenditure. You can argue that that, say, a Linux based system is easier to administer; but you can equally argue the reverse. The main factor is available skills, rather than the inherent quality of the software.

Copyright Tim Anderson January 2004. All rights reserved.
You are welcome to post links to this article. If you wish to print, distribute, or copy all or part of it, please contact me for permission, which may be subject to a fee.