It gets worse...
The MySQL source code base had gotten increasingly unmaintainable over the years, and key engineers were extremely resistant to changing the internals of MySQL and modernising it. There were some good reasons for being resistant, and some poor reasons (such as “this is the way we’ve always done it”). Overall, it’s tough to question the strategy that Zack, Marten Mickos, and others had regarding the short term gains. After all, they managed to manoeuvre MySQL into a winning position that Sun Microsystems thought was worth one billion dollars. Because of this, it’s tough to argue with them.
MySQL was, in some sense, a victim of its own success. Once it became valuable and produced plenty of money, there was a natural reluctance to fix what wasn't broken. That's one issue that every fast-growing open source company will have to face, along with the more general one of corporate ossification. But it's not the only one.
Even before the Drizzle team decided they couldn't take it any more, there were other signs MySQL was perhaps moving in the wrong direction. One was co-founder Widenius leaving Sun and setting up his own company to produce an alternative version of his original database:
I have now departed from Sun and joined my own company,
The main reason for leaving was that I am not satisfied with the way the MySQL server has been developed, as can be seen on my previous blog post. In particular I would have like to see the server development to be moved to a true open development environment that would encourage outside participation and without any need of differentiation on the source code. Sun has been considering opening up the server development, but the pace has been too slow. …
One shouldn't regard the MySQL-Maria tree as a fork but as a branch as we intend to pull in all changes from the official tree to the MySQL-Maria tree; Some of the changes will probably be reworked but we will do our best to ensure that for the end user they should look and feel the same.
As if this “branch-not-a-fork” of MySQL weren't enough, a completely different approach was also gaining support at this time:
The relatively young but rapidly growing "nosql" community met last Thursday in San Francisco. The idea was to give attendees a solid introduction to how distributed, non relational databases work as well as an overview of the various projects out there.
But the NoSQL approach – it's very name a kind of reproach to MySQL – is no mere theoretical exercise. Recently, both Digg and Twitter have decided to dump MySQL for Cassandra, a database that Facebook felt it needed to write:
Facebook, for instance, created its Cassandra data store to power a new search feature on its Web site rather than use its existing database, MySQL. According to a presentation by Facebook engineer Avinash Lakshman , Cassandra can write to a data store taking up 50GB on disk in just 0.12 milliseconds, more than 2,500 times faster than MySQL.
As Oscar Wilde might have said, to lose one iconic Web site is unfortunate, to lose three looks like a trend. The issue here is the changing nature of online activity, where vast quantities of data not in conventional formats need to be manipulated incredibly rapidly. MySQL, by contrast, was designed to serve traditional enterprise needs – something it did, and still does, extremely well.
Alongside dealing with the deadening hand of success, the issue of re-inventing projects for changing circumstances is also one that open source coders and companies need to bear in mind. The question here is whether to try to evolve with the market, or to create an entirely new project to serve that area. The nimbler response of others has already made MySQL's answer largely irrelevant. That might be unfortunate for MySQL, which seems to be entering its twilight years, but for the rest of us, it's just part of the ruthless Darwinian selection that lies at the heart of open source's success.
Follow me @glynmoody on Twitter or identi.ca. For other feature articles by Glyn Moody, please see the archive.