TripleO CI jobs primer¶
This primer aims to demonstrate where the Triple ci jobs are defined and illustrate the difference between the check and gate queues and how jobs are executed in them. Which queue a job is executed in also affects whether the job is defined as voting or not. Generally:
new jobs are run in check and are non voting
once a job is voting in check, it needs to be added to gate too.
once a job is voting in check and gate you should add it to the promotion jobs so that tripleo promotions (i.e. from tripleo-testing to current-tripleo) will depend on successful execution of that job.
Once a job becomes voting it must be added to the gate queue too. If it isn’t then we may end up with a situation where something passes the voting check job and merges without being run in the gate queue. It could be that for some reason it would have failed in the gate and thus not have merged. A common occurrence is the check jobs run on a particular submission and pass on one day but then not actually merge (and so run in the gate) until much later perhaps even after some days.In the meantime some unrelated change merges in another project which would cause the job to fail in the gate, but since we’re not running it there the code submission merges. This then means that the job is broken in subsequent check runs.
Non tripleo-projects are not gated in tripleo. The promotion jobs represent the point at which we take the latest built tripleo packages and the latest built non-tripleo projects packages (like nova, neutron etc) and test these together. For more information about promotions refer to Promotion Stages
Where do tripleo-ci jobs live¶
Note
If you ever need to search for a particular job to see which file it is defined in or which tripleo project repos it is running for you can search by name in the openstack-codesearch (e.g. that is a search for the tripleo-ci-centos-7-scenario003-standalone job).
Note
If you ever want to see the status for a particular job with respect to how often it is failing or passing, you can check the zuul_builds status and search by job name (again the linked example is for scenario003-standalone).
The tripleo ci jobs live in the tripleo-ci repo and specifically in various files defined under the zuul.d directory. As an example we can examine one of the scenario-standalone-jobs:
- job:
name: tripleo-ci-centos-7-scenario001-standalone
voting: true
parent: tripleo-ci-base-standalone
nodeset: single-centos-7-node
branches: ^(?!stable/(newton|ocata|pike|queens|rocky)).*$
vars:
featureset: '052'
standalone_ceph: true
featureset_override:
standalone_container_cli: docker
standalone_environment_files:
- 'ci/environments/scenario001-standalone.yaml'
- 'environments/low-memory-usage.yaml'
tempest_plugins:
- python-telemetry-tests-tempest
- python-heat-tests-tempest
test_white_regex: ''
tempest_workers: 1
tempest_extra_config: {'telemetry.alarm_granularity': '60'}
tempest_whitelist:
- 'tempest.api.identity.v3'
- 'tempest.scenario.test_volume_boot_pattern.TestVolumeBootPattern.test_volume_boot_pattern'
- 'telemetry_tempest_plugin.scenario.test_telemetry_integration.TestTelemetryIntegration'
As you can see the job definition consists of the unique job name followed by the rest of the zuul variables, including whether the job is voting and which node layout (nodeset) should be used for that job. The unique job name is then used in the zuul layout (discussed in the next section) to determine if the job is run in check or gate or both. Since the job shown above is set as voting we can expect it to be defined in both gate and check.
Zuul queues - gate vs check¶
As with all OpenStack projects there are two zuul queues to which jobs are scheduled - the check jobs which are run each time a change is submitted and then the gate jobs which are run before a change is merged. There is also an experimental queue but that is invoked manually.
Which queue a given job is run in is determined by the zuul layout file for the given project - e.g. here is tripleo-heat-templates-zuul-layout. The layout file has the following general format:
- project:
templates:
.. list of templates
check:
jobs:
.. list of job names and any options for each
gate:
queue: tripleo
jobs:
.. list of job names and any options for each
The templates: section in the outline above is significant because the layout can also be defined in one of the included templates. For example the scenario-standalone-layout defines the check/gate layout for the tripleo-standalone-scenarios-full template which is then included by the projects that want the jobs defined in that template to execute in the manner it specifies.
Where do tripleo promotion jobs live¶
Note
If you even need to find the definition for a particular promotion job you can search for it by name using the rdo-codesearch.
The tripleo promotions jobs are not defined in the tripleo-ci but instead live in the rdo-jobs repository. For more information about the promotion pipeline in TripleO refer to the Promotion Stages
Similar to the tripleo-ci jobs, they are defined in various files under the rdo-jobs-zuul.d directory and the job definitions look very similar to the tripleo-ci ones - for example the periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master:
- job:
name: periodic-tripleo-ci-centos-7-multinode-1ctlr-featureset010-master
parent: tripleo-ci-base-multinode-periodic
vars:
nodes: 1ctlr
featureset: '010'
release: master
If you even need to find the definition for a particular promotion job you can search for it by name using the rdo-codesearch.
Contacting CI team¶
When in need you can contact the TripleO CI team members on one of the two
irc channels on OFTC #tripleo by mentioning @oooq
keyword in your
message as team members get notified about such messages. It is good to
remember that those nicknames with |ruck
and |rover
suffix are on duty
to look for CI status.