The vanilla plugin is a reference implementation which allows users to operate a cluster with Apache Hadoop.
Since the Newton release Spark is integrated into the Vanilla plugin so you can launch Spark jobs on a Vanilla cluster.
For cluster provisioning, prepared images should be used.
Version (image tag) | Distribution | Build method | Version (build parameter) | Notes |
---|---|---|---|---|
2.8.2 | Ubuntu 16.04, CentOS 7 | sahara-image-create | 2.8.2 | Hive 2.3.2, Oozie 4.3.0 |
2.7.5 | Ubuntu 16.04, CentOS 7 | sahara-image-create | 2.7.5 | Hive 2.3.2, Oozie 4.3.0 |
2.7.1 | Ubuntu 16.04, CentOS 7 | sahara-image-create | 2.7.1 | Hive 0.11.0, Oozie 4.2.0 |
For more information about building image, refer to Sahara documentation.
Vanilla plugin requires an image to be tagged in Sahara Image Registry with two tags: ‘vanilla’ and ‘<hadoop version>’ (e.g. ‘2.7.1’).
The image requires a username. For more information, refer to the registering image section of the Sahara documentation.
When sahara-image-create
is used, you can override few settings
by exporting the corresponding environment variables
before starting the build command:
DIB_HADOOP_VERSION
- version of Hadoop to installHIVE_VERSION
- version of Hive to installOOZIE_DOWNLOAD_URL
- download link for Oozie (we have built
Oozie libs here: https://tarballs.openstack.org/sahara-extra/dist/oozie/)SPARK_DOWNLOAD_URL
- download link for SparkThe image building tools described in Building guest images add the required software to the image and their usage is strongly suggested. Nevertheless, here are listed the software that should be pre-loaded on the guest image so that it can be used to create Vanilla clusters:
See Swift Integration for information on using Swift with your sahara cluster (for EDP support Swift integration is currently required).
To support EDP, the following components must also be installed on the guest:
When user creates or scales a Hadoop cluster using a Vanilla plugin, the cluster topology requested by user is verified for consistency.
Currently there are the following limitations in cluster topology for Vanilla plugin:
For Vanilla Hadoop version 2.x.x:
Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.