Adding new Roles and Services¶

Recommended procedure to develop a role¶

1. Deploy OpenStack-Ansible (possibly using an AIO deploy) so that you have the rest of an OpenStack cluster to integrate with in your testing.
2. Deploy your service on another VM, or possibly directly on the AIO host, by hand. Configure the service to coordinate with the OpenStack cluster appropriately. When all the related systems are communicating with each other you can use the resulting configuration as a reference later.
3. Develop a role for your service. A recommended process is detailed below.

Writing a new role¶

Here are the steps to write the role:

1. You can review roles which may be currently in development by checking our specs repository and unmerged specs on review.openstack.org. If you do not find a spec for the role, propose a blueprint/spec (see also the spec template) outlining the new Role. By proposing a draft spec you can help the OpenStack-Ansible community keep track of what roles are being developed and perhaps connect you with others who may be interested and able to help you in the process.

2. Create a source repository (e.g. on Github) to start your work on the Role.

3. Generate the reference directory structure for an Ansible role which is the necessary subset of the documented Best Practice. You might use Ansible Galaxy tools to do this for you (e.g. ansible-galaxy init). You may additionally want to include directories such as docs and examples and tests for your role.

4. Generate a meta/main.yml right away. This file is important to Ansible to ensure your dependent roles are installed and available and provides others with the information they will need to understand the purpose of your role.

5. Develop task files for each of the install stages in turn, creating any handlers and templates as needed. Ensure that you notify handlers after any task which impacts the way the service would run (such as configuration file modifications). Also take care that file ownership and permissions are appropriate.

   Hint

   Fill in variable defaults, libraries, and prerequisites as you discover a need for them. You can also develop documentation for your role at the same time.

6. Add tests to the role.

7. Ensuring the role deploys with an AIO.

Role development maturity¶

In order to include a role into the integrated build implemented by the playbooks in the openstack/openstack-ansible repository, it needs to meet an appropriate level of maturity requirements. Developers are encouraged to observe the below-mentioned patterns in the existing roles.

The development of a role will usually go through the following stages:

1. Initial role development

   • Include base scaffolding. To facilitate development and the tests implemented across all OpenStack-Ansible roles, a base set of folders and files need to be implemented. A base set of configuration and test facilitation scripts must include at least the following:

     • tox.ini: The lint testing, documentation build, release notes build and functional build execution process for the role’s gate tests are all defined in this file.
     • test-requirements.txt: The python requirements which must be installed when executing the tests.
     • other-requirements.txt: The binary requirements which must be installed on the host the tests are executed on for the python requirements and the tox execution to work.
     • setup.cfg and setup.py: Information about the repository which is used during the build of any artifacts.
     • run_tests.sh: A convenient script for developers to execute all standard tests on a suitable host.
     • Vagrantfile: A convenient configuration file to allow a developer to easily create a test virtual machine using Vagrant. This must automatically execute run_tests.sh.
     • README.rst, LICENSE, CONTRIBUTING.rst: A set of standard files which have content describing their purpose.
     • .gitignore: A standard git configuration file for the repository which should be pretty uniform across all the repositories.
     • .gitreview: A standard file configured for the project to inform the git-review plugin where to find the upstream gerrit remote for the repository.

   • The role development should initially be focused on implementing a set of tasks and a test playbook which converge. The convergence must:

     • Implement developer_mode to build from a git source into a Python venv.
     • Deploy the applicable configuration files in the right places.
     • Ensure that the service starts.

     The convergence may involve consuming other OpenStack-Ansible roles (For example: galera_server, galera_client, rabbitmq_server) in order to ensure that the appropriate infrastructure is in place. Reuse of existing roles in OpenStack-Ansible or Ansible Galaxy is strongly encouraged.

   • The role must support Ubuntu 16.04 LTS. It should ideally also support CentOS7 but this is not required at this time. The patterns to achieve this include:

     • The separation of platform specific variables into role vars files.
     • The detection and handling of different init systems (init.d, SystemD).
     • The detection and handling of different package managers (apt, yum).
     • The detection and handling of different network configuration methods.

     There are several examples of these patterns implemented across many of the OpenStack-Ansible roles. Developers are advised to inspect the established patterns and either implement or improve upon them.

   • The role implementation should be done in such a way that it is agnostic with regards to whether it is implemented in a container, or on a physical host. The test infrastructure may make use of LXC containers for the separation of services, but if a role is used by a playbook that targets a host, it must work regardless of whether that host is a container, a virtual server, or a physical server. The use of LXC containers for role tests is not required but it may be useful in order to simulate a multi-node build out as part of the testing infrastructure.

   • Any secrets (For example: passwords) should not be provided with default values in the tasks, role vars, or role defaults. The tasks should be implemented in such a way that any secrets required, but not provided, should result in the task execution failure. It is important for a secure-by-default implementation to ensure that an environment is not vulnerable due to the production use of default secrets. Deployers must be forced to properly provide their own secret variable values.

   • Once the initial convergence is working and the services are running, the role development should focus on implementing some level of functional testing. Ideally, the functional tests for an OpenStack role should make use of Tempest to execute the functional tests. The ideal tests to execute are scenario tests as they test the functions that the service is expected to do in a production deployment. In the absence of any scenario tests for the service a fallback option is to implement the smoke tests instead.

   • The role must include documentation. The Documentation and Release Note Guidelines provide specific guidelines with regards to style and conventions. The documentation must include a description of the mandatory infrastructure (For example: a database and a message queue are required), variables (For example: the database name and credentials) and group names (For example: The role expects a group named foo_all to be present and it expects the host to be a member of it) for the role’s execution to succeed.

2. Integration development

   Once the role has implemented the above requirements, work can begin on integrating the role into the integrated build. This involves the preparation of the following items:

   • Host and container group configuration

     This is implemented into the dynamic inventory through the definition of content in an env.d file. A description of how these work can be found in Appendix C of the Deployment Guide.

   • Load balancer configuration

     OpenStack-Ansible deploys services in a highly available configuration by default, so all API services must be configured for implementation behind HAProxy. This is done through the modification of playbooks/vars/configs/haproxy_config.yml.

   • Install playbook

     In order to implement the role in the appropriate way, an os-<service>-install.yml playbook must be created and targeted at the appropriate group defined in the service env.d file. The playbook should also ensure that the database(s), database user(s), rabbitmq vhost and rabbitmq user are setup for the service. It is crucial that the implementation of the service is optional and that the deployer must opt-in to the deployment through the population of a host in the applicable host group. If the host group has no hosts, Ansible skips the playbook’s tasks automatically.

   • Secrets

     Any secrets required for the role to work must be noted in the etc/openstack_deploy/user_secrets.yml file.

   • Group vars

     Any variables needed by other roles to connect to the new role, or by the new role to connect to other roles, should be implemented in playbooks/inventory/group_vars. The group vars are essentially the glue which playbooks use to ensure that all roles are given the appropriate information. When group vars are implemented it should be a minimum set to achieve the goal of integrating the new role into the integrated build.

   • Documentation

     Content must be added to the Installation Guide to describe how to implement the new service in an integrated environment. This content must adhere to the Documentation and Release Note Guidelines. Until the role has integrated functional testing implemented, the documentation must make it clear that the service inclusion in OpenStack-Ansible is experimental and is not fully tested by OpenStack-Ansible in an integrated build.

   • Release note

     A feature release note must be added to announce the new service availability and to refer to the Installation Guide entry and the role documentation for further details. This content must adhere to the Documentation and Release Note Guidelines.

   • Integration test

     It must be possible to execute a functional, integrated test which executes a deployment in the same way as a production environment. The test must execute a set of functional tests using Tempest. This is the required last step before a service can remove the experimental warning from the documentation.