Hortonworks, Red Hat and Mirantis to bring easy Hadoop to OpenStack

Hortonworks, Red Hat and Mirantis have announced that they will be cooperating on Project Savanna which aims to make provisioning Hadoop clusters on OpenStack systems fast and easy. Savanna is being designed as an OpenStack component with a REST API and UI accessible through OpenStack's Horizon Dashboard.

Savanna is an existing project within OpenStack and the three companies say they hope to accelerate its development so that it can be demonstrated at June's Hadoop Summit in San Jose. To that end they will all contribute engineering resources to the project which had been initiated by Mirantis. Hortonworks is one of the leading commercial vendors of Apache Hadoop, while Red Hat has become a major contributor to OpenStack.

With Savanna, the developers say that will be easy to tell an OpenStack managed IaaS cloud that a Hadoop cluster, with a particular cluster topology, specific Hadoop version and hardware nodes of a specific type is needed and have Savanna deploy the cluster in minutes. The idea is to allow easy provisioning of OpenStack based Hadoop, especially in dev and QA environments, and to quickly make use of unused OpenStack compute nodes in a system easily or to just make the deployment of ad-hoc Hadoop clusters for analytics simple.

Savanna will plug into OpenStack's Keystone for authentication, provision VMs through Nova, retrieve Hadoop VM images from the Glance service and Swift, the storage service, can act as storage for data to be processed by the Hadoop Map/Reduce nodes. A video shows the proof-of-concept designs for Savanna:

Proof-of-concept demonstration of the Savanna Project.

Savanna is also expected to act as an integration point for other Hadoop management tools such as Apache Ambari (currently in the Apache Incubator) or Cloudera's Management Console. The current plan for Savanna envisages a Hadoop "FileSystem" implementation for Swift, currently in development, which would allow Hadoop jobs to work as naturally with Swift as they do with HDFS. The project was presented to the OpenStack community as part of the "Unconference Track" at the OpenStack Summit, currently running in Portland.

(djwm)