Control Plane Service Placement

Note

This is an advanced topic and should only be attempted when familiar with kayobe and OpenStack.

The default configuration in kayobe places all control plane services on a single set of servers described as ‘controllers’. In some cases it may be necessary to introduce more than one server role into the control plane, and control which services are placed onto the different server roles.

Configuration

Overcloud Inventory Discovery

If using a seed host to enable discovery of the control plane services, it is necessary to configure how the discovered hosts map into kayobe groups. This is done using the overcloud_group_hosts_map variable, which maps names of kayobe groups to a list of the hosts to be added to that group.

This variable will be used during the command kayobe overcloud inventory discover. An inventory file will be generated in ${KAYOBE_CONFIG_PATH}/inventory/overcloud with discovered hosts added to appropriate kayobe groups based on overcloud_group_hosts_map.

Kolla-ansible Inventory Mapping

Once hosts have been discovered and enrolled into the kayobe inventory, they must be added to the kolla-ansible inventory. This is done by mapping from top level kayobe groups to top level kolla-ansible groups using the kolla_overcloud_inventory_top_level_group_map variable. This variable maps from kolla-ansible groups to lists of kayobe groups, and variables to define for those groups in the kolla-ansible inventory.

Variables For Custom Server Roles

Certain variables must be defined for hosts in the overcloud group. For hosts in the controllers group, many variables are mapped to other variables with a controller_ prefix in files under ansible/inventory/group_vars/controllers/. This is done in order that they may be set in a global extra variables file, typically controllers.yml, with defaults set in ansible/inventory/group_vars/all/controllers. A similar scheme is used for hosts in the monitoring group.

Overcloud host variables

Variable

Purpose

ansible_user

Username with which to access the host via SSH.

bootstrap_user

Username with which to access the host before ansible_user is configured.

lvm_groups

List of LVM volume groups to configure. See mrlesmithjr.manage_lvm role for format.

mdadm_arrays

List of software RAID arrays. See mrlesmithjr.mdadm role for format.

network_interfaces

List of names of networks to which the host is connected.

sysctl_parameters

Dict of sysctl parameters to set.

users

List of users to create. See singleplatform-eng.users role

If configuring BIOS and RAID via kayobe overcloud bios raid configure, the following variables should also be defined:

Overcloud BIOS & RAID host variables

Variable

Purpose

bios_config

Dict mapping BIOS configuration options to their required values. See stackhpc.drac role for format.

raid_config

List of RAID virtual disks to configure. See stackhpc.drac role for format.

These variables can be defined in inventory host or group variables files, under ${KAYOBE_CONFIG_PATH}/inventory/host_vars/<host> or ${KAYOBE_CONFIG_PATH}/inventory/group_vars/<group> respectively.

Custom Kolla-ansible Inventories

As an advanced option, it is possible to fully customise the content of the kolla-ansible inventory, at various levels. To facilitate this, kayobe breaks the kolla-ansible inventory into three separate sections.

Top level groups define the roles of hosts, e.g. controller or compute, and it is to these groups that hosts are mapped directly.

Components define groups of services, e.g. nova or ironic, which are mapped to top level groups.

Services define single containers, e.g. nova-compute or ironic-api, which are mapped to components.

The default top level inventory is generated from kolla_overcloud_inventory_top_level_group_map. Kayobe’s component- and service-level inventory for kolla-ansible is static, and taken from the kolla-ansible example multinode inventory. The complete inventory is generated by concatenating these inventories.

Each level may be separately overridden by setting the following variables:

Custom kolla-ansible inventory variables

Variable

Purpose

kolla_overcloud_inventory_custom_top_level

Overcloud inventory containing a mapping from top level groups to hosts.

kolla_overcloud_inventory_custom_components

Overcloud inventory containing a mapping from components to top level groups.

kolla_overcloud_inventory_custom_services

Overcloud inventory containing a mapping from services to components.

kolla_overcloud_inventory_custom

Full overcloud inventory contents.

Examples

Example 1: Adding Network Hosts

This example walks through the configuration that could be applied to enable the use of separate hosts for neutron network services and load balancing. The control plane consists of three controllers, controller-[0-2], and two network hosts, network-[0-1]. All file paths are relative to ${KAYOBE_CONFIG_PATH}.

First, we must make the network group separate from controllers:

inventory/groups
 [controllers]
 # Empty group to provide declaration of controllers group.

 [network]
 # Empty group to provide declaration of network group.

Then, we must map the hosts to kayobe groups.

overcloud.yml
overcloud_group_hosts_map:
  controllers:
    - controller-0
    - controller-1
    - controller-2
  network:
    - network-0
    - network-1

Next, we must map these groups to kolla-ansible groups.

kolla.yml
kolla_overcloud_inventory_top_level_group_map:
  control:
    groups:
      - controllers
  network:
    groups:
      - network

Finally, we create a group variables file for hosts in the network group, providing the necessary variables for a control plane host.

inventory/group_vars/network
ansible_user: "{{ kayobe_ansible_user }}"
bootstrap_user: "{{ controller_bootstrap_user }}"
lvm_groups: "{{ controller_lvm_groups }}"
mdadm_arrays: "{{ controller_mdadm_arrays }}"
network_interfaces: "{{ controller_network_host_network_interfaces }}"
sysctl_parameters: "{{ controller_sysctl_parameters }}"
users: "{{ controller_users }}"

Here we are using the controller-specific values for some of these variables, but they could equally be different.

Example 2: Overriding the Kolla-ansible Inventory

This example shows how to override one or more sections of the kolla-ansible inventory. All file paths are relative to ${KAYOBE_CONFIG_PATH}.

It is typically best to start with an inventory template taken from the Kayobe source code, and then customize it. The templates can be found in ansible/roles/kolla-ansible/templates, e.g. components template is overcloud-components.j2.

First, create a file containing the customised inventory section. We’ll use the components section in this example.

kolla/inventory/overcloud-components.j2
[nova]
control

[ironic]
{% if kolla_enable_ironic | bool %}
control
{% endif %}

...

Next, we must configure kayobe to use this inventory template.

kolla.yml
kolla_overcloud_inventory_custom_components: "{{ lookup('template', kayobe_env_config_path ~ '/kolla/inventory/overcloud-components.j2') }}"

Here we use the template lookup plugin to render the Jinja2-formatted inventory template.

Fine-grained placement

Kayobe has fairly coarse-grained default groups - controller, compute, etc, which work well in the majority of cases. Kolla Ansible allows much more fine-grained placement on a per-service basis, e.g. ironic-conductor. If the operator has taken advantage of this fine-grained placement, then it is possible that some of the assumptions in Kayobe may be incorrect. This is one downside of the split between Kayobe and Kolla Ansible.

For example, Ironic conductor services may have been moved to a subset of the top level controllers group. In this case, we would not want the Ironic networks to be mapped to all hosts in the controllers group - only those running Ironic conductor services. The same argument can be made if the loadbalancer services (HAProxy & keepalived) or Neutron dataplane services (e.g. L3 & DHCP agents) have been separated from the top level network group.

In these cases, the following variables may be used to tune placement:

controller_ironic_conductor_group

Ansible inventory group in which Ironic conductor services are deployed. Default is controllers.

controller_ironic_inspector_group

Ansible inventory group in which Ironic inspector services are deployed. Default is controllers.

controller_loadbalancer_group

Ansible inventory group in which control plane load balancer services are deployed. Default is network.

controller_network_group

Ansible inventory group in which network data plane services are deployed. Default is network.