This was the title of an article back in 2002 by Shelley Doll and other articles with more or less the same concerns including the secretary of the ANSI database committee. In my recent blog post I made the point that I do hope that MySQL AB tries to follow the standards as much as possible (while retaining the freedom to add things as deemed necessary .. LIMIT and friends). In a chat conversation with Jan, he asked me what my thoughts are in regards to the SQL standard in particular. Most people will for example agree that standards compliant SQL routines are no fun to write. Unfortunately I had to agree with him that SQL today isn't what it should be.

Since I am not sure if everybody is aware of this so just let me briefly explain the scope of the SQL standard even though this is not directly necessary for the discussion I am trying to raise with this blog post. The SQL standard is not concerned with how data is read or stored. As such the access algorithms and storage concepts are up to the vendors to decide upon. As a matter of fact the standard just covers DML statements, so there is no standard for the syntax for creating indexes or even the concept of an index itself! Also a lot of details were also not covered. For example the ability to control how NULLs should be sorted was only added in the SQL2003 version of the standard.

Another topic I want to briefly cover is who gets to define the standard. Basically the requirement is to become a member of ANSI. The fee for an individual is 500 USD per year. I am not sure what the fee's are for an organization. At this point OSS is not very well represented. Well since both Oracle and IBM are involved directly (or semi directly in the case of SUN) in OSS databases there is some representation, but nobody from MySQL or PostgreSQL is in the "H2 databases" working group. As part of my thesis paper I talked about this point with representatives from many OSS projects and it seemed interest was low to work on becoming able to influence future versions of the standard.

Anyways to get back to the topic at hand, the key breaking point is identified as the point in time when NIST stopped providing SQL compliance tests. Back then it was a requirement for vendors to pass NIST compliance testing in order to be considered for government projects, which was the bulk of the money that could be made in the RDBMS market. This had a number of benefits. For one people had an interest in keeping the standard to a size that was implementable and clear. Furthermore they had an interest to stick with the standard and finally there was a reference testing suite to get rid of too much ambiguity that is unfortunately hard to avoid in standards.

But without testing a key piece of the chain was broken and with it the flood gates of new features, non compliant syntax and general disregard for the standard followed. Of course standards can cause innovation to stall, but vendors could still add their own extra features on top of the standard, though I am not sure if back then government departments were allowed to make use of them. In the above linked article by Michael Goremann he notes that:

"What Makes a Standard a Standard? Simple. Not implementation, but conformance. And, conformance is “known” only after conformance testing."

The last SQL standard version that was covered by NIST was released in 1992, called SQL 92 as a result and that is more or less what most RDBMS vendor's have implemented today. The updates to the SQL standard in 1999 and 2003 have only been rudimentary even by large vendors like Oracle and IBM. We can see the necessity for compliance testing in other fields as well. I am sure that few people will dispute that without ACID compliance testing of browsers life for web designers would be even worse. That being said its also clear that there must be a "force" that pushes vendors not to ignore standards. In the case of SQL it was a requirement for government contracts. In the case of web browsers I think it is the adoption rate of new features by developers which pushed the likes of Microsoft to at least care a bit more and maybe its us that have to push RDBMS vendors today to care about the SQL standard and to diverge only with sound technical reasoning.

I guess a related topic is if SQL is a standard worth having, but I will leave that to user comments or another blog post.