The battle of APIs rages for IaaS domination

AWS is becoming the Windows of cloud era

Thanks to its disruptive and fast-paced innovations, Amazon has built a comprehensive and dominant IaaS stack that inspires all of us. It started first with virtualized machines (EC2) and large scale file storage (S3) and has completed its offering both horizontally (DynamoDB, Elastic MapReduce, ElasticSearch, IAM, etc.).

Many applications have tied themselves to their web APIs which are necessary to use their web services. AWS APIs have therefore become the “de facto” standard APIs for cloud computing, despite the fact that they are proprietary, lack in RESTful design and use custom security schemes.

Using AWS APIs to compete with AWS

Other infrastructure providers have been working hard to build alternatives. The first approach is simple in theory: let’s accept the AWS APIs as the current standard and try instead to provide and alternative implementation, as much compatible as possible.

Eucalyptus has been a leader on this front with its open source project, even if other providers such as Apache CloudStack and OpenStack are offering some level of compatibility as well. Eucalyptus currently provides alternative implementations for most popular AWS APIs : S3, EC2, EBS, AMI and IAM.

Even though this strategy illustrates the power of web APIs, there are several issues to keep in mind.

The API compatibility issue

The first one is that it is very hard to support all AWS APIs and then to support 100% of their features in a way that is fully portable. This difficulty is strongly pointed out by Lew Moorman, President of Rackspace, in a talk he gave at GigaOM Structure conference and further explained by Thorsten von Eicken from RightScale in this post.

The API copyright issue

Another major concern is the fact that it isn’t clear at this stage whether or not implementing those APIs respects their copyright. This question is not to be taken lightly as we can see Oracle and Google currently fighting in Court regarding the fact that Google copied the core Java APIs in order to provide an alternative implementation in Android. See this blog post from Kin Lane for details.

It is interesting to note that Eucalyptus and AWS have recently signed an agreement, that encourages the implementation of all AWS APIs which is likely a way for AWS to counter the lock-in argument that they probably face frequently. Still, this is an unstable legal position and until AWS releases those APIs in open source, that legal uncertainty will remain.

Update: a recent ruling indicates that in Android’s case, Google used Java APIs didn’t infringe on Oracle’s copyright. Thanks to J.P. Figer for the remark.

Creating alternative open source stacks

Instead of focusing on AWS APIs, OpenStack, Apache CloudStack or vmWare CloudFoundry are building open source alternatives to AWS proprietary stack, not just clones.

Below is an illustration for OpenStack, which is promoted by RackSpace and a large number of partner companies. Note that at this stage it doesn’t provide an alternative to managed databases such as AWS RDS or DynamoDB.

There is no winner yet and we’ll have to watch those projects evolve and new one emerge. While OpenStack appears to have a lot of traction, it recently saw NASA and Citrix changing strategy. NASA now promotes usage of AWS (thanks to Jean-Paul Figer for the pointer) and Citrix has bought Cloud.com and gave the code to Apache Foundation to initiate the CloudStack project.

Looking ahead

In the long term, some truly standard and open APIs will definitely emerge for commodity IaaS features, following the example of Atom/AtomPub experience in the blog space or the POSIX experience for file systems.

However, it is unlikely that a one size fits all solution will soon become the Linux of the cloud. It will probably take more time for things to settle down.

For now, we will live in a world of diversity, innovation and complexity, where APIs and open source infrastructure pieces coexist and fight for developers mind share and usage. Here is an example of alternative stack to AWS.

Many alternative object storages systems are competing such as Ceph, providing APIs for compatibility with S3 API, POSIX API and others such as CIFS.

The same is true for computing and image management, and especially for cloud-scale databases such as Cassandra, MongoDB, Redis and so on.

The main drawback for this diversity is the need to operate those pieces, either directly or via partners when AWS offers an all-in-one solution.

Conclusion

Watching this battle going on will be fun. The GigaOM Structure event that was held this week in San Francisco provided a great stage for debate and collective thinking as summarized by this series of blog posts:

If you missed it, you have a chance to catch up with the first edition of this event that will be held in Europe next October.

Leave a Reply