Providing Cloud Service Tiers

In the early days of cloud computing emphasis was placed on 'one size fits all'. However, as our delivery capabilities have increased, we're now able to deliver more product variations where some products provide the same function (e.g., storage) but deliver better performance, availability, recovery, etc. and are priced higher. I.T. must assume that some applications are business critical while others are not. Forcing users to pay for the same class of service across the spectrum is not a viable option. We've spent a good deal of time analyzing various cloud configurations, and can now deliver tiered classes of services in our private clouds.

Reviewing trials, tribulations and successes in implementing cloud solutions, one can separate tiers of cloud services into two categories: 1) higher throughput elastic networking; or 2) higher throughput storage. We leave the third (more CPU) out of this discussion because it generally boils down to 'more machines,' whereas storage and networking span all machines.

Higher network throughput raises complex issues regarding how one structures networks – VLAN or L2 isolation, shared segments and others. Those complexities, and related costs, increase dramatically when adding multiple speed NICS and switches, for instance 10GBase-T, NIC teaming and other such facilities. We will delve into all of that in a future post.

Tiered Storage on Private Cloud

Where tiered storage classes are at issue, cost and complexity is not such a dramatic barrier, unless we include a mix of network and storage (i.e., iSCSI storage tiers). For the sake of simplicity in discussion, let's ignore that and break the areas of tiered interest into: 1) elastic block storage (“EBS”); 2) cached virtual machine images; 3) running virtual machine (“VM”) images. In the MomentumSI private cloud, we've implemented multiple tiers of storage services by adding solid state drives (SSD) drives to each of these areas, but doing so requires matching the nature of the storage usage with the location of the physical drives.

Consider implementing EBS via higher speed SSD drives. Because EBS volumes avail themselves over network channels to remain attachable to various VMs, unless very high speed networks carry the drive signaling and data, a lower speed network would likely not allow dramatic speed improvements normally associated with SSD. Whether one uses ATA over Ethernet (AoE), iSCSI, NFS, or other models to project storage across the VM fabric, even standard SATA II drives, under load could overload a one-gigabit Ethernet segment. However, by exposing EBS volumes on their own 10Gbe network segments, EBS traffic stands a much better chance of not overloading the network. For instance, at MSI we create a second tier of EBS service by mounting SSD on the mount points under which volumes will exist – e.g., /var/lib/eucalyptus/volumes, by default, on a Eucalyptus storage controller. Doing so gives users of EBS volumes the option of paying more for 'faster drives.'

While EBS gives users of cloud storage a higher tier of user storage, the cloud operations also represent a point of optimization, thus tiered service. The goal is to optimize the creation of images, and to spin them up faster. Two particular operations extract significant disk activity in cloud implementation. First, caching VM images on hypervisor mount points. Consider Eucalyptus, which stores copies of kernels, ramdisks (initrd), and Eucalyptus Machine Images (“EMI”) files on a (usually) local drive at the Node Controllers (“NC”). One could also store EMIs on an iSCSI, AoE or NFS, but the same discussion as that regarding EBS applies (apply fast networking with fast drives). The key to the EMI cache is not so much about fast storage (writes), rather rapid reads. For each running instance of an EMI (i.e., a VM), the NC creates a copy of the cached EMI, and uses that copy for spinning up the VM. Therefore, what we desire is very fast reads from the EMI cache, with very fast writes to the running EMI store. Clearly that does not happen if the same drive spindle and head carry both operations.

In our labs, we use two drives to support the higher speed cloud tier operations: one for the cache and one for the running VM store. However, to get a Eucalyptus NC, for instance, to use those drives in the most optimal fashion, we must direct the reads and writes to different disks,– one drive (disk1) dedicated to cache, and one drive (disk2) dedicated to writing/running VM images. Continuing with Eucalyptus as the example setup (though other cloud controllers show similar traits), the NC will, by default, store the EMI cache and VM images on the same drive -- precisely what we don't want for higher tiers of services.

By default, Eucalyptus NCs store running VMs on the mount point /usr/local/eucalyptus/???, where ??? represents a cloud user name. The NC also stores cached EMI files on /usr/local/eucalyptus/eucalyptus/cache -– clearly within the same directory tree. Therefore, unless one mounts another drive (partition, AoE or iSCSI drive, etc.) on /usr/local/eucalyptus/eucalyptus/cache, the NC will create all running images by copying from the EMI cache to the run-space area (/usr/local/eucalyptus/???) on the same drive. That causes significant delays in creating and spinning up VMs. The simple solution: mount one SSD drive on /usr/local/eucalyptus, and then mount a second SSD drive on /usr/local/eucalyptus/eucalyptus/cache. A cluster of Eucalyptus NCs could share the entire SSD 'cache' drive by exposing it as an NFS mount that all NCs mount at /usr/local/eucalyptus/eucalyptus/cache. Consider that the cloud may write an EMI to the cache, due to a request to start a new VM on one node controller, yet another NC might attempt to read that EMI before the cached write completes, due to a second request to spin up that EMI (not an uncommon scenario). There exist a number of ways to solve that problem.

The gist here: by placing SSD drives at strategic points in a cloud, we can create two forms of higher tiered storage services: 1) higher speed EBS volumes; and 2) faster spin-up time. Both create valid billing points, and both can exist together, or separately in different hypervisor clusters. This capability is now available via our Eucalyptus Consulting Services and will soon be available for vCloud Director.

Next up – VLAN, L2, and others for tiered network services.

Auto Scaling as a Service

The Tough Auto Scaling Service is our offering to enable the automated scaling of an application tier at runtime. System data collected by a monitoring service provides the intelligence to provision or deprovision resources according to SLA's. Out of the box, our service uses our own Tough Monitoring Service, however, since we're using the de facto standard (Amazon Web Services), you can plug in any implementation that is AWS compatible.

The auto scaling service works by defining an 'auto scaling group'. This identifies the kind of service which will shrink or expand based on system load. The most common use case for auto scaling is for the Web tier where additional Web servers are added on the fly to respond to heavy loads. Auto scaling can also be used on stateful tiers but extra attention must be spent on managing the state replication mechanisms (clustering, etc.)

As new servers are provisioned to respond to the load request, they can be added to a dynamically programmable load balancer. This enables in-bound application traffic to be evenly divided across the array of virtual servers identified in an auto scaling group. Conversely, when the load returns to normal levels, the virtual servers are taken out of the load balanced pool allowing a graceful shutdown. To enable this scenario, we're using our Tough Load Balancing Service, but once again, customers can use any AWS compatible load balancer to perform this operation.

One of the key concepts of cloud computing is the concept of 'elasticity'; another is 'automation'. The Auto Scaling Service brings these two concepts together and applies them toward the compute side of the world to provide three key benefits:

1. Increased success rates on Service Level Agreements - The system auto scales to meet SLA's

2. Higher utilization rates - Unused virtual servers are released back to the pool

3. Reduced operating costs - Predefined policies automate activities that previously would have been human intensive tasks.

Combined, these three benefits make auto scaling a critical component of any private / hybrid cloud environment. It's also worth pointing out that the auto scaling service is a fundamental building block to enable other scalable services such as Platform Services (PaaS).

Non-Invasive Cloud Monitoring as a Service

The Tough Cloud Monitoring solution is our next generation offering targeting virtualized workloads, as well as PaaS services, housed in either traditional data centers or private cloud environments.

By monitoring, we mean 'health and performance' monitoring of infrastructure and platforms. Our service provides the traditional statistical information: CPU utilization, disk I/O, network traffic, etc. This begs the question, "why does the world need yet-another monitoring solution?" Quite frankly, we were surprised that there weren't better options available on the market. So, once again, we started from scratch with a new design center:

1. Make it massively scalable and highly available

Some of our customers currently have 1,000's of virtualized work loads operating and it is clear that the next generation service providers will have 10's of thousands running. Our design needed to easily scale itself from both a data collection perspective and burst storage. We bit the bullet and designed a solution from scratch to use Apache Cassandra at the core. This enabled us to leverage it's built-in cross-data center peer replication schemes and dynamic partitioning. In addition, Cassandra was a good fit for us because it was designed to accept very fast (stream oriented) writes of data.

2. The monitors should be non-invasive and agent free

Being non-invasive is always a good goal; it makes it easier to collect data on targets without having to install additional software on the machine (which can be a real problem when you already have lots of machines running in production). Knock-on-wood, but so far, we've been able to deliver all of our monitors completely out-of-band. No need to install Ganglia, collectD, etc. on hundreds/thousands of boxes...

3. The monitors should support a standard, service oriented API

In building our early private clouds, we were surprised to see that most of the system monitoring tools were "closed" systems. They collected the data but didn't make it easily available to other systems; they were designed to deliver the information to humans in HTML. This was a non-starter for us since the new world is about achieving higher levels of system automation (not human tasking). Naturally, we went with the de facto standard Amazon Web Services and the CloudWatch API. Our solution delivers full compatibility with CloudWatch from a WSDL, AWS Query and command line perspective. This makes it real easy for the monitoring data to be consumed by other services like Auto Scale.

4. Use a consistent model for IaaS and PaaS

By supporting the AWS service interface model, we inherited this feature. Just as CloudWatch monitors services like their Elastic Load Balancer and Relational Data Services, we'll be providing similar support for internal PaaS platforms.

We believe that we have achieved all of our design goals. The Tough Cloud Monitor is available today for traditional data centers, private clouds or service providers.

Tough Load Balancing as a Service

Last week, MomentumSI announced the availability of our Tough Load Balancing Service along with a Cloud Monitoring and Auto Scaling solution.

The concept of load balancing has been around for decades - so nothing too new there. However, applying the 'as a Service' model to load balancing remains a fairly new concept, especially in the traditional data center. Public cloud providers like Amazon have offered a similar function for the last couple of years and have seen significant interest in their offering. We believe LB-aaS offers an equivalent productivity boost to traditional data centers, private cloud customers or service providers who want to extend their current offerings.

Our design goals for the solution were fairly simple - and we believe we met each of them:

1. Don't interfere with the capability of the underlying load balancer

The LB-aaS solution wraps traditional load balancers (currently, software based only) to enable rapid provisioning, life-cycle management, configuration and high availability pairing. All of these functions run outside of the ingress / egress path of the data. This means you do not incur additional latency in the actual balancing. Also, our design enables us to snap in various load balancer implementations. Our current solution binds to HAProxy and Pound for SSL termination. Based on customer demand, we anticipate adding additional providers (e.g, F5, Zeus, etc.) Our goal is to nail the "as-a-Service" aspect of the problem and to be able to easily swap in the right load balancer implementation for our customers specific needs.

2. Make life easier for the user

I was recently as one of my enterprise customers speaking with an I.T. program manager. She commented that her team was in a holding pattern while they ordered a new load balancer for their application. Her best guess was that it was going to take about 5 weeks to get through their internal procurement cycle and then another 2-3 weeks to get it queued up for the I.T. operations people to get around to installing, configuring and testing it out. When I told her about our LB-aaS solution (2-3 minutes to provision and another whopping 5-10 minutes to configure), she just started laughing... and made a comment about necessity being the mother of all invention.

3. Deliver an open API

Delivering an open API was an easy decision for us. We went with the Amazon Web Service Elastic Load Balancer API. We maintained compatibility with their WSDL as well as providing command line capabilities and the use of their AWS Query protocol. As the ecosystem around AWS continues to grow, we want companies to be able to immediately plug into our software without code-level changes.

4. Don't cause pain down the road

We've seen some companies put software based load balancers into their VM image templates. We see this as last-years stop-gap solution. The lack of device-specific life-cycle management leads to configuration drift and no service-oriented interface means you can't use the load balancer as part of an integrated solution pattern (like auto-scale). Let's face it, the world is moving to an 'as a service' model for some good reasons.

Again, the Tough Load Balancing Service is available today and can easily work in current data centers, private clouds or service providers.

Separating IaaS into Two Layers

For some time now, I've been watching cloud architects consider their strategy for deploying wide-scale Infrastructure-as-a-Service (IaaS). Many of my friends are quick to draw the standard Gartner cloud stack (SaaS, PaaS followed by IaaS). And although I think this is a simple way to look at the layers, it can be dangerous if that's where the conversation ends.

I'd like to suggest that we consider at least two distinct IaaS layers:

Some people call the first layer, "Hardware-as-a-Service". It primarily focuses on the 'virtualization' of hardware enabling better manipulation by the upper layers. This was the core proposition of the original EC2. There are some great vendors in this space like Eucalyptus, Cloud.com and VMware. Cool projects are also emerging out of OpenStack which many of the aforementioned companies hope to adopt and extend.

The second layer is the 'automation layer'. It focuses on providing convenience mechanisms around layer 1 services. This includes everything from making multi-step human tasks more easily accomplished through orchestrations, to closed-loop systems akin to the problem defined in autonomic computing. The core elements delivered in layer 2 includes self-inspection, self-healing, self-protection and resource optimization. These are some pretty powerful concepts. So powerful in fact, that it often makes sense for consuming technologies to bind to layer 2, rather than directly to layer 1.

We're starting to see this layered approach unveil itself at Amazon. Services like Elastic Beanstalk focus on integrating many of the lower layer building blocks into an easy to consume bundle, while also delivering several of the autonomic properties. It's pretty cool stuff. But, it's only cool if you actually use it. I loved that Amazon started off real low in the stack (EC2 servers) and worked their way up. It was fundamentally the right way to rethink the problem. The downside is that many engineers are now overly comfortable using the original atomic elements when they need to be looking harder at the new convenience layers (e.g., CloudFormation, Elastic Beanstalk, etc.)

The announcement we made yesterday regarding custom implementations of Amazon CloudWatch, Elastic Load Balancer and Auto Scaling for private cloud demonstrate our commitment to this approach. We're also big believers in industry standards. In my younger (and more naive) days, I would have preached about 'open standards' over 'industry standards' but I've sat in on too many industry conference calls listening to vendors with agendas bicker over standards only to wait years to get a solution which was designed by a committee. When it comes to cloud standards, I'll gladly let those younger (or more patient) than I fight those fights. Until then, we're backing the de-facto standard, AWS. And to those who say "standardized API isn't important", I'll have to kindly disagree ;-)