Separating IaaS into Two Layers

For some time now, I've been watching cloud architects consider their strategy for deploying wide-scale Infrastructure-as-a-Service (IaaS). Many of my friends are quick to draw the standard Gartner cloud stack (SaaS, PaaS followed by IaaS). And although I think this is a simple way to look at the layers, it can be dangerous if that's where the conversation ends.

I'd like to suggest that we consider at least two distinct IaaS layers:

Some people call the first layer, "Hardware-as-a-Service". It primarily focuses on the 'virtualization' of hardware enabling better manipulation by the upper layers. This was the core proposition of the original EC2. There are some great vendors in this space like Eucalyptus, Cloud.com and VMware. Cool projects are also emerging out of OpenStack which many of the aforementioned companies hope to adopt and extend.

The second layer is the 'automation layer'. It focuses on providing convenience mechanisms around layer 1 services. This includes everything from making multi-step human tasks more easily accomplished through orchestrations, to closed-loop systems akin to the problem defined in autonomic computing. The core elements delivered in layer 2 includes self-inspection, self-healing, self-protection and resource optimization. These are some pretty powerful concepts. So powerful in fact, that it often makes sense for consuming technologies to bind to layer 2, rather than directly to layer 1.

We're starting to see this layered approach unveil itself at Amazon. Services like Elastic Beanstalk focus on integrating many of the lower layer building blocks into an easy to consume bundle, while also delivering several of the autonomic properties. It's pretty cool stuff. But, it's only cool if you actually use it. I loved that Amazon started off real low in the stack (EC2 servers) and worked their way up. It was fundamentally the right way to rethink the problem. The downside is that many engineers are now overly comfortable using the original atomic elements when they need to be looking harder at the new convenience layers (e.g., CloudFormation, Elastic Beanstalk, etc.)

The announcement we made yesterday regarding custom implementations of Amazon CloudWatch, Elastic Load Balancer and Auto Scaling for private cloud demonstrate our commitment to this approach. We're also big believers in industry standards. In my younger (and more naive) days, I would have preached about 'open standards' over 'industry standards' but I've sat in on too many industry conference calls listening to vendors with agendas bicker over standards only to wait years to get a solution which was designed by a committee. When it comes to cloud standards, I'll gladly let those younger (or more patient) than I fight those fights. Until then, we're backing the de-facto standard, AWS. And to those who say "standardized API isn't important", I'll have to kindly disagree ;-)