TripleO Containers Architecture¶

post.yaml.j2¶

This file is a jinja template and it’s rendered before the deployment is started. This file defines the resources that are executed before and after the container initialization.

docker-puppet.py¶

This script is responsible for generating the config files for each service. The script is called from the post.yaml file and it takes a json file as configuration. The json files passed to this script are built out of the puppet_config parameter set in every service template (explained in the Docker specific settings section).

The docker-puppet.py execution results in a oneshot container being executed (usually named puppet-$service_name) to generate the configuration options or run other service specific operations. Example: Create Keystone endpoints.

Docker specific settings¶

Each service may define an output variable which returns a puppet manifest snippet that will run at each of the following steps. Earlier manifests are re-asserted when applying latter ones.

• service_name: This setting is inherited from the puppet service templates and it shouldn’t be modified as some of the hiera tasks depend on this.

• config_settings: This setting is generally inherited from the puppet/services templates and only need to be appended to on occasion if docker specific config settings are required. For example:

  config_settings:
    map_merge:
      - get_attr: [MongodbPuppetBase, role_data, config_settings]
      - mongodb::server::fork: false
  

• step_config: This setting controls the manifest that is used to create docker config files via puppet. The puppet tags below are used along with this manifest to generate a config directory for this container.

• kolla_config: Contains YAML that represents how to map config files into the kolla container. This config file is typically mapped into the container itself at the /var/lib/kolla/config_files/config.json location and drives how kolla’s external config mechanisms work. Example:

  kolla_config:
    /var/lib/kolla/config_files/mongodb.json:
      command: /usr/bin/mongod --unixSocketPrefix=/var/run/mongodb --config /etc/mongod.conf run
      config_files:
      - dest: /etc/mongod.conf
        source: /var/lib/kolla/config_files/src/etc/mongod.conf
        owner: mongodb
        perm: '0600'
      - dest: /etc/mongos.conf
        source: /var/lib/kolla/config_files/src/etc/mongos.conf
        owner: mongodb
        perm: '0600'
  

• docker_config: Data that is passed to the heat-config-docker-cmd hook to configure a container, or step of containers at each step. See the available steps below and the related docker-cmd hook documentation in the heat-agents project.

• puppet_config:

  • step_config: Usually a reference to the one defined outside this section.

  • puppet_tags: Puppet resource tag names that are used to generate config files with puppet. Only the named config resources are used to generate a config file. Any service that specifies tags will have the default tags of ‘file,concat,file_line’ appended to the setting. For example:

    puppet_tags: keystone_config
    

    Some puppet modules do a bit more than just generating config files. Some have custom resources with providers that execute commands. It’s possible to overwrite these providers by changing the step_config property. For example:

    puppet_tags: keystone_config
    step_config:
      list_join:
        - "\n"
        - - "['Keystone_user', 'Keystone_endpoint', 'Keystone_domain', 'Keystone_tenant', 'Keystone_user_role', 'Keystone_role', 'Keystone_service'].each |String $val| { noop_resource($val) }"
          - {get_attr: [KeystoneBase, role_data, step_config]}
    

    The example above will overwrite the provider for all the Keystone_* puppet tags (except keystone_config) using the noop_resource function that comes with puppet-tripleo. This function dynamically configures the default provider for each of the puppet_tags in the array.

  • config_volume: The name of the docker volume where config files will be generated for this service. Use this as the location to bind mount into the running Kolla container for configuration.

  • config_image: The name of the docker image that will be used for generating configuration files. This is often the same container that the runtime service uses. Some services share a common set of config files which are generated in a common base container.

• docker_puppet_tasks: This section provides data to drive the docker-puppet.py tool directly. The task is executed only once within the cluster (not on each node) and is useful for several puppet snippets we require for initialization of things like keystone endpoints, database users, etc. See docker-puppet.py for formatting. Here’s an example of Keystone’s docker_puppet_tasks:

  docker_puppet_tasks:
    # Keystone endpoint creation occurs only on single node
    step_4:
      - 'keystone_init_tasks'
      - 'keystone_config,keystone_domain_config,keystone_endpoint,keystone_identity_provider,keystone_paste_ini,keystone_role,keystone_service,keystone_tenant,keystone_user,keystone_user_role,keystone_domain'
      - 'include ::tripleo::profile::base::keystone'
      - list_join:
        - '/'
        - [ {get_param: DockerNamespace}, {get_param: DockerKeystoneImage} ]
  

• host_prep_tasks: Ansible tasks to execute on the host before any containers are started. Useful e.g. for ensuring existence of directories that we want bind mounted into the containers.

• upgrade_tasks: Ansible tasks to execute during upgrade. First these tasks are run on all nodes, and then the normal puppet/docker operations happen the same way as during a fresh deployment.

Docker steps¶

Similar to baremetal, docker containers are brought up in a stepwise manner. The current architecture supports bringing up baremetal services alongside of containers. Therefore, baremetal steps may be required depending on the service and they are always executed before the corresponding container step.

The list below represents the correlation between the baremetal and the containers steps. These steps are executed sequentially:

1. Containers config files generated per hiera settings.
2. Load Balancer configuration baremetal
   1. Step 1 baremetal
   2. Step 1 containers
3. Core Services (Database/Rabbit/NTP/etc.)
   1. Step 2 baremetal
   2. Step 2 containers
4. Early Openstack Service setup (Ringbuilder, etc.)
   1. Step 3 baremetal
   2. Step 3 containers
5. General OpenStack Services
   1. Step 4 baremetal
   2. Step 4 containers
   3. Keystone containers post initialization (tenant, service, endpoint creation)
6. Service activation (Pacemaker)
   1. Step 5 baremetal
   2. Step 5 containers