Fuel Development Environment¶

Getting the Source Code¶

Source code of OpenStack Fuel can be found at git.openstack.org or GitHub.

Follow these steps to clone the repositories for each of the Fuel components:

apt-get install git
git clone https://github.com/openstack/fuel-main
git clone https://github.com/openstack/fuel-web
git clone https://github.com/openstack/fuel-ui
git clone https://github.com/openstack/fuel-agent
git clone https://github.com/openstack/fuel-astute
git clone https://github.com/openstack/fuel-ostf
git clone https://github.com/openstack/fuel-library
git clone https://github.com/openstack/fuel-docs

Building the Fuel ISO¶

The “fuel-main” repository is the only one required in order to build the Fuel ISO. The make script then downloads the additional components (Fuel Library, Nailgun, Astute and OSTF). Unless otherwise specified in the makefile, the master branch of each respective repo is used to build the ISO.

The basic steps to build the Fuel ISO from trunk in an Ubuntu 14.04 environment are:

apt-get install git
git clone https://github.com/openstack/fuel-main
cd fuel-main
./prepare-build-env.sh
make iso

If you want to build an ISO using a specific commit or repository, you will need to modify the “Repos and versions” section in the config.mk file found in the fuel-main repo before executing “make iso”. For example, this would build a Fuel ISO against the v5.0 tag of Fuel:

# Repos and versions
FUELLIB_COMMIT?=tags/5.0
NAILGUN_COMMIT?=tags/5.0
FUEL_UI_COMMIT?=tags/5.0
ASTUTE_COMMIT?=tags/5.0
OSTF_COMMIT?=tags/5.0

FUELLIB_REPO?=https://github.com/openstack/fuel-library.git
NAILGUN_REPO?=https://github.com/openstack/fuel-web.git
FUEL_UI_REPO?=https://github.com/openstack/fuel-ui.git
ASTUTE_REPO?=https://github.com/openstack/fuel-astute.git
OSTF_REPO?=https://github.com/openstack/fuel-ostf.git

To build an ISO image from custom gerrit patches on review, edit the “Gerrit URLs and commits” section of config.mk, e.g. for https://review.openstack.org/#/c/63732/8 (id:63732, patch:8) set:

FUELLIB_GERRIT_COMMIT?=refs/changes/32/63732/8

If you are building Fuel from an older branch that does not contain the “prepare-build-env.sh” script, you can follow these steps to prepare your Fuel ISO build environment on Ubuntu 14.04:

1. ISO build process requires sudo permissions, allow yourself to run commands as root user without request for a password:

   echo "`whoami` ALL=(ALL) NOPASSWD: ALL" | sudo tee -a /etc/sudoers
   

2. Install software:

   sudo apt-get update
   sudo apt-get install apt-transport-https
   echo deb http://mirror.yandex.ru/mirrors/docker/ docker main | sudo tee /etc/apt/sources.list.d/docker.list
   sudo apt-key adv --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys 36A1D7869245C8950F966E92D8576A8BA88D21E9
   sudo apt-get update
   sudo apt-get install lxc-docker
   sudo apt-get update
   sudo apt-get remove nodejs nodejs-legacy npm
   sudo apt-get install software-properties-common python-software-properties
   sudo add-apt-repository -y ppa:chris-lea/node.js
   sudo apt-get update
   sudo apt-get install build-essential make git ruby ruby-dev rubygems debootstrap createrepo \
   python-setuptools yum yum-utils libmysqlclient-dev isomd5sum \
   python-nose libvirt-bin python-ipaddr python-paramiko python-yaml \
   python-pip kpartx extlinux unzip genisoimage nodejs multistrap \
   lrzip python-daemon
   sudo gem install bundler -v 1.2.1
   sudo gem install builder
   sudo pip install xmlbuilder jinja2
   

3. If you haven’t already done so, get the source code:

   git clone https://github.com/openstack/fuel-main
   

4. Now you can build the Fuel ISO image:

   cd fuel-main
   make iso
   

5. If you encounter issues and need to rebase or start over:

   make clean          #remove build/ directory
   make deep_clean     #remove build/ and local_mirror/
   

You can also use the following tools to make your work and development process with Fuel easier:

• CGenie fuel-utils - a set of tools to interact with code on a Fuel Master node created from the ISO. It provides the fuel command that gives a simple way to upload Python or UI code (with staticfiles compression) to Docker containers, SSH into machine and into the container, display the logs etc.
• Vagrant SaltStack-based -Vagrant box definition with quick and basic Fuel environment with fake tasks. This is useful for UI or Nailgun development.

You can download both tools from the fuel-dev-tools.

Nailgun (Fuel-Web)¶

Nailgun is the heart of Fuel project. It implements a REST API as well as deployment data management. It manages disk volume configuration data, network configuration data and any other environment specific data necessary for a successful deployment of OpenStack. It provides the required orchestration logic for provisioning and deployment of the OpenStack components and nodes in the right order. Nailgun uses a SQL database to store its data and an AMQP service to interact with workers.

Requirements for preparing the nailgun development environment, along with information on how to modify and test nailgun can be found in the Nailgun Development Instructions document: Nailgun Development Instructions

Astute¶

Astute is the Fuel component that represents Nailgun’s workers, and its function is to run actions according to the instructions provided from Nailgun. Astute provides a layer which encapsulates all the details about interaction with a variety of services such as Cobbler, Puppet, shell scripts, etc. and provides a universal asynchronous interface to those services.

1. Astute can be found in fuel-astute repository

2. Install Ruby dependencies:

   sudo apt-get install git curl
   curl -sSL https://get.rvm.io | bash -s stable
   source ~/.rvm/scripts/rvm
   rvm install 2.1
   rvm use 2.1
   git clone https://github.com/nulayer/raemon.git
   cd raemon
   git checkout b78eaae57c8e836b8018386dd96527b8d9971acc
   gem build raemon.gemspec
   gem install raemon-0.3.0.gem
   cd ..
   rm -Rf raemon
   

3. Install or update dependencies and run unit tests:

   cd fuel-astute
   ./run_tests.sh
   

4. (optional) Run Astute MCollective integration test (you’ll need to have MCollective server running for this to work):

   cd fuel-astute
   bundle exec rspec spec/integration/mcollective_spec.rb
   

Running Fuel Puppet Modules Unit Tests¶

If you are modifying any puppet modules used by Fuel, or including additional modules, you can use the PuppetLabs RSpec Helper to run the unit tests for any individual puppet module. Follow these steps to install the RSpec Helper:

1. Install PuppetLabs RSpec Helper:

   cd ~
   gem2deb puppetlabs_spec_helper
   sudo dpkg -i ruby-puppetlabs-spec-helper_0.4.1-1_all.deb
   gem2deb rspec-puppet
   sudo dpkg -i ruby-rspec-puppet_0.1.6-1_all.deb
   

2. Run unit tests for a Puppet module:

   cd fuel/deployment/puppet/module
   rake spec
   

Installing Cobbler¶

Install Cobbler from GitHub (it can’t be installed from PyPi, and deb package in Ubuntu is outdated):

cd ~
git clone git://github.com/cobbler/cobbler.git
cd cobbler
git checkout release24
sudo make install

Building Documentation¶

You should prepare your build environment before you can build this documentation. First you must install Java, using the appropriate procedure for your operating system.

Java is needed to use PlantUML to automatically generate UML diagrams from the source. You can also use PlantUML Server for a quick preview of your diagrams and language documentation.

Then you need to install all the packages required for creating of the Python virtual environment and dependencies installation.

sudo apt-get install make postgresql postgresql-server-dev-9.1
sudo apt-get install python-dev python-pip python-virtualenv

Now you can create the virtual environment and activate it.

virtualenv fuel-web-venv
. fuel-web-venv/bin/activate

And then install the dependencies.

pip install -r nailgun/test-requirements.txt

Now you can look at the list of available formats and generate the one you need:

cd docs
make help
make html

There is a helper script build-docs.sh. It can perform all the required steps automatically. The script can build documentation in required format.

Documentation build helper
-o - Open generated documentation after build
-c - Clear the build directory
-n - Don't install any packages
-f - Documentation format [html,signlehtml,latexpdf,pdf,epub]

For example, if you want to build HTML documentation you can just use the following script, like this:

./build-docs.sh -f html -o

It will create virtualenv, install the required dependencies and build the documentation in HTML format. It will also open the documentation with your default browser.

If you don’t want to install all the dependencies and you are not interested in building automatic API documentation there is an easy way to do it.

First remove autodoc modules from extensions section of conf.py file in the docs directory. This section should be like this:

extensions = [
    'rst2pdf.pdfbuilder',
    'sphinxcontrib.plantuml',
]

Then remove develop/api_doc.rst file and reference to it from develop.rst index.

Now you can build documentation as usual using make command. This method can be useful if you want to make some corrections to text and see the results without building the entire environment. The only Python packages you need are Sphinx packages:

Sphinx
sphinxcontrib-actdiag
sphinxcontrib-blockdiag
sphinxcontrib-nwdiag
sphinxcontrib-plantuml
sphinxcontrib-seqdiag

Just don’t forget to rollback all these changes before you commit your corrections.