Rackspace open sources its cloud computing platform

by

With a surprising move, Rackspace today announced the release under open source license of the technology that powers its cloud computing platform.
Customers will be able to download it and use it to build a Rackspace-like on-premises cloud.

Named OpenStack, the platform initially includes only the part of the The Rackspace Cloud (formerly Mosso) called Cloud Files (formerly Cloud FS).

Cloud Files is a storage-as-a-service cloud that Rackspace launched in May 2008 and that competes with Amazon Simple Storage Service (S3), part of the Amazon Web Services (AWS).

This component will be called OpenStack Storage:

OpenStack Object Storage aggregates commodity servers to work together in clusters for reliable, redundant, and large-scale storage of static objects. Objects are written to multiple hardware devices in the datacenter, with the OpenStack software responsible for ensuring data replication and integrity across the cluster. Storage clusters can scale horizontally by adding new nodes, which are automatically configured. Should a node fail, OpenStack works to replicate its content from other active nodes. Because OpenStack uses software logic to ensure data replication and distribution across different devices, inexpensive commodity hard drives and servers can be used in lieu of more expensive equipment.

In late 2010 the company will also include in OpenStack the portion of its current platform called Cloud Servers.

Cloud Servers is an Infrastructure-as-a-Service (IaaS) cloud computing platform that competes with Amazon Elastic Compute Cloud (EC2), part of the Amazon AWS division as well.
Cloud Servers is built on top of Xen, but in May Rackspace announced a plan to migrate everything to XenServer, so OpenStack will be based on Citrix XenServer too at the beginning and the Xen Cloud Platform (XCP) going forward (which means that the Open vSwitch will be the building block of the distributed networking infrastructure).

Rackspace owns a second IaaS cloud computing platform: the Slicehost one, which is based on Xen too.
The hosting provider acquired Slicehost in October 2008 but kept the company as an independent subsidiary.
There’s no mention of any plan to include the Slicehost technology in the OpenStack infrastructure, probably because Cloud Servers are a superset of Slicehost offering.

In late 2010 OpenStack will also include the technology that powers the NASA IaaS cloud called Nebula (not to be confused with OpenNebula, an IaaS cloud computing management platform).
Nebula is available to NASA’s internal project groups. Nebula is not available for use by private industry or by the public. Nebula is also currently acting as a test-bed for Federal Cloud Computing technology for The Office of Management and Budget.

The Cloud Servers component and the NASA Nebula component will be called OpenStack Compute:

OpenStack Compute is software for provisioning and managing large-scale deployments of compute instances. It is written in Python, using the Tornado and Twisted frameworks, and relies on the standard AMQP messaging protocol as well as the Redis distributed KVS.

The use of AMQP and Redis technologies is extremely interesting and may create the opportunity to interoperate with the VMware’s clouds of tomorrow.
VMware in fact hired the creator and maintainer of Redis in March, and acquired Rabbit Technologies in April.
Rabbit Technologies offers an open source enterprise messaging system called RabbitMQ which is based on Advanced Message Queuing Protocol (AMQP). Along with the Redis database, both technologies are probably being integrated in a future cloud computing platform that VMware is building with SpringSource.

Currently, the OpenStack Compute technical board includes six members:

  • Rick Clark – Senior Manager of Software Product Development at Rackspace 
    Clark, who is the Chief Architect and Project Lead for OpenStack, has been the Engineering Manager of Ubuntu Server at Canonical.
  • Paul Voccio – Network Security Admin IV at Rackspace
  • Soren Hansen – Systems Architect at Rackspace
  • Joshua Mckenty – Web Apps Architect and Technical Lead at NASA
  • Ray O’Brien – Nebula Program Manager at NASA
  • Jesse Andrews – Nebula Tech Lead at NASA

The underlying hardware infrastructure will be provided by Dell, which is a remarkable achievement considering the lack of vision demonstrated by the company’s founder at the Citrix Synergy 2010 conference this May.

A lot of players confirmed attended a special event about OpenStack that took place last week in Austin: AMD, Autonomic Resources, Citrix, Cloud.com, Cloudkick, Cloudscaling, CloudSwitch, Dell, enStratus, FathomDB, Intel, iomart Group, Limelight, Nicira, NTT DATA, Opscode, PEER 1, Puppet Labs, RightScale, Riptano, Scalr, SoftLayer, Sonian, Spiceworks, Zenoss and Zuora.
The group is incredibly interesting. It includes a metering & billing player, Zuora, and a semi-stealth network virtualization startup, Nicira, where the founder and former CEO of VMware, Diane Greene, invested.

Because the initial API for OpenStack essentially is the current Rackspace API, some of these vendors are immediately supporting the new platform. Like enStratus.
In the future OpenStack will support both standard APIs and vendors’ specific ones, like the VMware vCloud APIs.

Rackspace published a 3-minutes supporting video that describes how things will work and what’s the commitment in OpenStack.
YouTube also offers another couple of interesting videos, both recorded at the inaugural OpenStack Design Summit last week:

OpenStack, released under the Apache 2.0 license and completely unrestricted (no Enterprise Editions planned) can be downloaded here.

The initiative is pretty significant and the support demonstrated by the industry players is remarkable. But while many people will highlight that VMware is not included in the list above, the real vendor that can be potentially impacted by this announcement is Amazon.

Since the launch of EC2, customers have wondered if and when Amazon would release an on-premises version of its cloud computing platform. But Amazon is paranoid about its infrastructure, which is built on top of the Red Hat implementation of Xen. Probably for the company giving away the technology to customers equals to losing revenue and exposing the internals of the custom management stack to attackers.
But if OpenStack will have a lot of traction, Amazon may be obliged to surrender and give away an on-premises EC2 sooner than planned.