Do we need Cloud API Standards ?

Of late there has been lot of buzz around cloud API standards. For a while there was only one cloud and one set of APIs and that is of Amazon. But now we have several public and private clouds. With so many solutions/products coming up, it is common (and expected of) for  vendors, ISVs and developers to start thinking about a standard set of APIs to access these cloud services.

This seemed to attract lot of criticism from a section of blogosphere. Here are few that I came across:

One of the common criticism seems to be around that standardization is bad for innovation from vendors and that “lowest common denominator” rules. I don’t think this is something we should worry about.  DOM and SAX APIs were the only standards to process XML for several years. That didn’t stop one to innovate and come up with StaX API.  That didn’t stop several other Domain specific languages (DSL) still innovating on different mechanims to access and process XML.

Look at  standards to access  SQL.  Both JDBC and ODBC are around for several year now. Have you seen anyone writing a SQL application that doesn’t use these standards (or some of the new ones that are coming up for each of PHP/Python/Ruby etc) ?  If you are developing a new Java database application today, would you worry that JDBC exposes only “lowest common denominator” functionality and that you go with some native API exposed by the RDBMS vendor. Nah. I don’t think so.

And neither of these standards prevented innovation from vendors. Infact, it made life easy for lot of vendors.

Clay Lovelace‘s post mentions that standard APIs tend to leak abstractions. Reading more carefully, the post is pointing out issues with some tools that generate SQL and not standard API like JDBC.  Leaky abstractions are more prominent when the underlying models are fundamentally different. That is not the case with JDBC/ODBC like standards. Underlying model for these standards is common – SQL and RDBMS.  Same with  XML API standards – underlying model is same.

 If you talk about standard API for  Amazon’s EC2 and Rackspace’s cloud – then there is a good chance that abstractions would leak as the underlying models are different.  Having such a common API is probably ok for vendors like Rightscale(and may be required for their survival), but no good for a developer.

But if you talk about standard API for cloud storage, esp for CRUD like operations – this is very much needed. The underlying models are similar.

I say – This is the right time to start the standardization process. Even if the API addresses only 20% of the functionality, doesn’t matter. Time and adoption will drive the rest. Let us get going in defining standard models (I didn’t say API) for specific cloud services such as cloud storage. Standard APIs just follow.

Then I came across Tom Nolle‘s post on “Multiple standards could spoil cloud computing“.  Interesing. I didn’t know that there are so many self proclaimed standard bodies around for Cloud computing. Clearly, you cannot have a single standard when there is more than one group working on standardization. As he says, it is worse than not having a single standard.  We have seen this in the post where it didn’t succeed. One classic example that comes to the mind is the spec on “WebServices Reliability” where two self-proclaimed group of vendors went on defining separate standards and ended up finally merging into one.

The unexpected and “Byzantine faults”

I was discussing with a colleague of mine the other day about handling of node failures in a distributed system, esp the case where a node A thinks that node C is down but node B thinks otherwise. The ramifications of this could be fatal to the system unless handled properly.

Later I came across the term “Byzantine faults” in another unrelated search and started following lot of other literature around this. Here are couple of good reads.