Commercial open source software has arrived. There have been commercial sponsors of open source projects for a long time, but the recent spate of high-price (for open source, at least) acquisitions of companies with open source products at the core of their business has made a splash in the technology industry. In addition, Sun’s convulsions into the open source world have left the Java landscape decisively in open source territory.
Is this recent trend toward commercial open source good for open source software? Is it good for the Java developer? Is the utopian spirit of the open source movement about to get crushed by greed, profit seeking, and evil corporate maneuvering? Well…let’s see.
Open source software has had a clearly beneficial impact on the tools we use everyday. It provides a rich and varied ecosystem where technologies can compete purely on their merits in the hands of the people who actually use them. This ecosystem allows the virtues of natural selection to shape the tools, libraries, and frameworks we use to get software built.
As a result of this healthy selection pressure, the open source projects that gain widespread adoption and thrive are those that solve real problems in the best way. Technologies that fail to solve problems fail to gain acceptance and die. The open source landscape is littered with the carnage of this competition. Sourceforge, for example, is strewn with the corpses of thousands of projects that, for one reason or another, couldn’t compete and therefore died.
In stark contrast, the monolithic piles of technology offered as complete solutions from proprietary software vendors look increasingly like sleepy isolated atolls whose pace of innovation is far outstripped by the technologies on the open source mainland. Of course, competition in the marketplace exerts selective pressure even on proprietary software. However, because open source software is, in general, more freely available than proprietary software, the adaptive process happens much faster. If you’re a developer living on one of those insular vendor islands, you’re often presented with a “solution” to every problem. Unfortunately, the solution is seldom best of breed and may not actually work.
In a previous life, before working on Terracotta, I was a software architect on one at the largest retail Web sites in the world. We built a software stack out of best-of-breed components, most of which were open source. We used Apache as our Web server, Tomcat as our application server, Linux (though, later, we switched to Solaris x86) as the server operating system, and many smaller open source libraries to solve specific problems like database connection pooling. We also had some very expensive proprietary load-balancing hardware and an even more expensive proprietary database server. But, everything we assembled was the best implementation of its function that we could find.
There were a number of services, however, that we couldn’t find solutions for, either proprietary or open source, that we had to build ourselves. Specifically, we were forced to build a sophisticated caching infrastructure (which, incidentally, informed much of the thinking that led to Terracotta). But, the openness of our platform allowed us to integrate our solutions into the software stack. There were a number of times when we cracked open the Tomcat source code to add our own improvements and to provide hooks for our bespoke solutions.
However, we were in a constant battle with our corporate parent to take advantage of a proprietary turnkey Web application stack for which we had an enterprise-wide license. More than once, we put the vendor stack to the test against our open stack and every time it failed to meet the scalability requirements of our rapidly growing site traffic. The vendor stack offered all of the features of our open stack, but in nearly every case, its implementation fell short.
Of course, there are significant attractions to using vendor-supplied software, from both a business and technology perspective. At the very least, the vendor provides a throat to choke when things go wrong. And, the bleeding edge of open source is not a comfortable place to live. Open source is a meritocracy, both for contributors and users. When open source software doesn’t work right or doesn’t scale properly, it’s pretty much up to you to figure out how to solve the problem. As a developer using open source components, if things don’t go well, you’d better be ready to pick up a shovel and dig through some code.
In the vendor world, on the other hand, you can often pay your way out of sticky situations. If a component doesn’t work as advertised, you can often use your status as paid customer to squeeze out a fix or a suitable workaround. And, if you’re facing a business-crushing capacity or scalability problem that you can’t figure out yourself, you can likely pay the vendor a lot of money to keep your application running.
While we were focused on building a best-of-breed software stack, from the perspective of our corporate parents, it made a lot more sense to have a uniform software stack throughout its enterprise and a stable relationship with a vendor to help solve problems. In the long term, a retailer has no business investing in and maintaining the kinds of cutting-edge technology we built. The engineering team of a Web store should spend time adding features to it Web site, not building a caching server. There was no alternative at the time, but, had the vendor solution worked, it would have made much more business sense to use it – even if it wouldn’t have been as much fun for us developers.
This is where commercial open source can step in and provide a great deal of benefit by offering the safety and customer focus of a vendor-supplied product combined with the accelerated innovation that the open source ecosystem brings to bear on technology.
Now that I work on Terracotta, I see the vendor’s perspective on building a software stack. In addition, since we’ve been open source for more than a year, I have the privilege of witnessing first hand the benefits to our product of exposure to the open source ecosystem.
The only way to create a high-quality software product is to have it run through compute cycles. Lots of them. At Terracotta, we have a huge arsenal of automated tests that run our software through its paces 24 hours a day. But, there’s simply no substitute for running software in the real world deployed against real problems in real situations. That’s where your code really develops quality and hardness.
Going open source has exposed the Terracotta code to a massively larger user base in a much shorter amount of time than we had when we were a proprietary enterprise software company. Our code has run on thousands of computers whose users constantly engage us in discussions on our forums and our mailing lists about features, improvements, and corner cases we could never have found just in our labs. And, we find out much earlier if there are any bugs or usability problems with new versions since people are downloading and experimenting with our nightly builds.
As an open source company, it is much easier to integrate with other open source technologies. For example, our integration with Jetty, an open source, standards-based, full-featured Web server implemented entirely in Java, came about as friendly cooperation between core developers on both sides that took place over a matter of weeks, unencumbered by a drawn-out business development process with the usual hand-wringing about non-disclosure agreements and the like that could have stretched it out over months. The same can be said for many of the other integrations that were made possible once we went open source such as Wicket, RIFE, Geronimo, and many others.
The open source software development and delivery model enables much wider adoption than is usually possible with proprietary software. However, for a company that needs to make money, it’s a tremendous leap of faith to open up your intellectual property investment to the world and hope that enough people see value in it to pay for it. In fact, we get asked a lot, “If you give it away for free, how do you make money?”
Again, the answer here is good news for the Java developer. Proprietary vendors make money by promising that you’ll see value. By contrast, open source vendors make money only after you actually see value. There’s a big difference here for the software consumer. An open source company must deliver a product so good, you can’t live without it. As an application developer, you use open source because it’s the right technology, not because your manager or some vice-president has made a deal and spent a bunch of money on something that you’re now stuck using.
What you pay for, then, is not the promise that the technology will provide its purported benefits. Open source means that you have the chance to see for yourself if the technology is right at no risk to you except the time it takes to evaluate. Rather, what you pay for is the “customer experience” that you don’t get from traditional, non-commercial open source, for example:
In general, commercial open source gives you someone to call when there’s a problem and a crucial buffer between you and the bleeding edge.
So, is open source good for the Java developer? I think so. Speaking as a Java developer, I couldn’t get my job done without a vast array of tools and libraries that have evolved out of the open source ecosystem. In any given day, I use Java (of course), Eclipse, Ant, Maven, Jetty, Apache httpd, Tomcat, Firefox, Zimbra, XWiki, Terracotta…this list goes on.
Does commercial open source threaten to subvert the open source ethos and destroy it? I suppose it remains to be seen, but I really doubt it. In fact, I think it’s quite the opposite. It’s not open source that’s being invaded by corporate pirates; rather, it’s the monolithic technology vendors that are being forced to adopt the open source development and distribution model to survive.
The open source model is emerging not just as a way to make cheaper software, but a way to produce and deliver the next wave of innovation in the software industry. As such, open source is stripping commercial software of its vestigial tails. Open source vendors, on the other hand, are helping open source technologies thrive in places that can’t afford to live on the bleeding edge by making them easier to consume.
Who doesn’t want a lot more access to highly innovative best-of-breed software that’s fun and easy to use?
© 2008 SYS-CON Media Inc.