oslo_config_validator

About the role

An Ansible role that will loop through all the containers on selected host, find the Openstack service configuration file and leverage the [oslo-config-validator](https://docs.openstack.org/oslo.config/latest/cli/validator.html) utility to validate the current running configuration.

It’s also possible to generate a report that contains all differences between the sample or default values with current running configuration.

Finally, it will also verify that the current running configuration doesn’t contain any known invalid settings that might have been deprecated and removed in previous versions.

Exceptions

Some services like cinder can have dynamic configuration sections. In cinder’s case, this is for the storage backends. To perform validation on these dynamic sections, we need to generate a yaml formatted config sample with oslo-config-generator beforehand, append a new sample configuration for each storage backends, and validate against that newly generated configuration file by passing --opt-data to the oslo-config-validator command instead of using --namespaces. Since generating a sample config adds some delay to the validation, this is not the default way of validating, we prefer to validate directly using --namespaces.

NOTE: At the time of writing this role, oslo-config-generator has a bug [1] when generating yaml config files, most notably with cinder. Since the inclusion of oslo.config patch can’t be garanteed, the role will inject this patch [2] to the oslo.config code, inside the validation container. This code change is ephemeral for the time of the configuration file generation. The reason why we want to inject this patch is because it’s possible that we run the validation on containers that were created before it was merged. This ensures a smooth validation across the board.

[1] https://bugs.launchpad.net/oslo.config/+bug/1928582 [2] https://review.opendev.org/c/openstack/oslo.config/+/790883

Requirements

This role needs to be run on an Undercloud with a deployed Overcloud.

Role Variables

• oslo_config_validator_validation: Wether or not to run assertions on produced outputs. That also means that the role will fail if anything is output post-filtering. If this is enabled with the reporting, this will most likely trigger a failure unless executed against default configuration

• oslo_config_validator_report: Wether or not we compare the configuration files found with the default config

• oslo_config_validator_invalid_settings: When running validation, wether or not we should check for invalid settings. This adds to the time it takes to complete validation because of the way the validations_read_ini module works. This won’t work without oslo_config_validator_validation enabled.

• oslo_config_validator_report_path: The folder used when generating the reports.

• oslo_config_validator_global_ignored_messages: List of regular expressions that will filter out messages globally, across all namespaces

• oslo_config_validator_namespaces_config: Specific namespace configurations. It contains namespace-specific ignored patterns as well as invalid settings configuration.

• oslo_config_validator_service_configs: Mapping of known Openstack services with their namespace configuration.

• oslo_config_validator_checked_services: List of services being validated.

Dependencies

• podman_container

• podman_container_info

• validations_read_ini

• https://review.opendev.org/c/openstack/oslo.config/+/790883

Example Reporting Playbook

- hosts: all
  vars:
  - oslo_config_validator_report: true
  - oslo_config_validator_validation: false
  roles:
  - { role: oslo_config_validator}

Example playbook to validate only one service

- hosts: all
  vars:
  - oslo_config_validator_checked_services:
  - nova
  roles:
  - { role: oslo_config_validator}

License

Apache

Author Information

Red Hat TripleO DFG:Compute Deployment Squad

Full Description

Role Documentation

Welcome to the “oslo_config_validator” role documentation.

Role Defaults

This section highlights all of the defaults and variables set within the “oslo_config_validator” role.

# All variables intended for modification should place placed in this file.

# All variables within this role should have a prefix of "oslo_config_validator"
oslo_config_validator_debug: false

# Comparison report with current settings and default settings
oslo_config_validator_report: false

# Returns all settings with possibly invalid values or simply inexistent
# oslo.config uses a typed system for possible values of each settings.
# if a setting is not typed correctly, or is non-existent, oslo-config-validator
# will trigger an error here.
oslo_config_validator_validation: true

# This is a temporary folder used when generating reports.
# It will be created and deleted if necessary on all the nodes
oslo_config_validator_report_path: /var/tmp/config_validator_report

# Whether or not we should archive into a single file the report after creation
oslo_config_validator_report_archive: true

# This is the working folder when running validations. It will be bind mounted
# on the validation containers
# It will be created and deleted if necessary on all the nodes
oslo_config_validator_work_path: /var/lib/tripleo-config/oslo_config_validator

# When running validation, whether or not we should check for invalid settings
# This adds to the time it takes to complete validation because of the way
# the validations_read_ini module works.
oslo_config_validator_invalid_settings: true


# These messages are globally ignored and will not trigger a validation failure
oslo_config_validator_global_ignored_messages:
- INFO:keyring.backend:Loading.*
- WARNING:oslo_config.generator:normalizing group name.*
- WARNING:stevedore.named:Could not load .*
- WARNING:root:Deprecated opt .*
- INFO:oslo_utils.netutils:Could not determine default network interface, using .*
- ERROR:root:quotas/quota_[^\s]+ {{ invalid_setting_regex }}
- .*Ignoring option because it is part of the excluded patterns. This can be changed
  with the.*

# namespace configuration:
#   We can define configuration for each namespaces.
#     ignored_messages: list of regexes that, when returned from calling a validation using that
#                       specific namespace, will be ignore
#     invalid_settings: list of settings that will trigger a failure if they are set to specific
#                       values. Separator is meant to split the setting value and checking each
#                       one of the values individually like nova filters. This is useful to
#                       validate if deprecated or removed option values are still in use.
#             keys:
#               - section: configuration section (ie: DEFAULT)
#               - option: setting name (ie: debug)
#               - separator: string delimiter that will be used to convert value to a list
#               - value_list: List of strings that would be checked against.
#               - operator: Can be either these values, default being "eq":
#                   - not: current value should not match exactly any element in value_list
#                   - lt: current value should be lesser than first element of value_list
#                   - gt: current value should be greater than first element of value_list
#                   - eq: current value, if defined, should be equal to one element in value_list
#
oslo_config_validator_namespaces_config:
- namespace: glance.store
  ignored_messages:
  - "ERROR:stevedore.extension:Could not load '(glance.store.)*s3(.Store)*': No module\
    \ named 'boto3'"

- namespace: heat.common.config
  ignored_messages:
      # NOTE(dvd): openstack/heat fix: https://review.opendev.org/789680
      #            heat wasn't including its yaql and cache options by default. This is being worked
      #            on during the Xena cycle and should hopefully be backported down to train.
  - ERROR:root:(yaql|(resource_finder_)*cache)/[^\s]+ {{ invalid_setting_regex }}

- namespace: keystonemiddleware.auth_token
  ignored_messages:
  - >-
    .*Ignoring missing option "(auth_url|username|password|(user|project)_(domain_)*name)"
    from group "keystone_authtoken" because the group is known to
    have incomplete sample config data and thus cannot be validated properly.*

- namespace: neutron
  invalid_settings:
  - section: nova
    option: tenant_name
    operator: eq
    value_list:
    - service
  - section: nova
    option: project_name
    operator: eq
    value_list:
    - service
  - section: DEFAULT
    option: notify_nova_on_port_data_changes
    operator: not
    value_list:
    - 'True'
  - section: DEFAULT
    option: notify_nova_on_port_status_changes
    operator: not
    value_list:
    - 'True'

- namespace: cinder
  invalid_settings:
  - section: DEFAULT
    option: enable_v3_api
    value_list:
    - 'True'
  ignored_messages:
  - ERROR:root:DEFAULT/api_paste_config {{ invalid_setting_regex }}

- namespace: nova.conf
  invalid_settings:
  - section: filter_scheduler
    option: enabled_filters
    separator: ','
    value_list:
    - ExactCoreFilter
    - ExactRamFilter
    - ExactDiskFilter
    - CoreFilter
    - RamFilter
    - DiskFilter
    - RetryFilter
  - section: DEFAULT
    option: vif_plugging_timeout
    operator: lt
    value_list:
    - 300
  - section: DEFAULT
    option: vif_plugging_timeout
    value_list:
    - 'True'

  ignored_messages:
      # These settings are used by openstacksdk and not part of oslo_config_opts. They are not taken
      # into account by oslo-config-(generator|validator)
  - ERROR:root:(cinder|service_user)/region_name {{ invalid_setting_regex }}
      # Censoring password
  - WARNING:root:neutron/metadata_proxy_shared_secret sample value is empty but input-file
    has
      # TODO(dvd): Needs to be investigated with TLSe
  - ERROR:root:cache/tls_enabled not found

# service configuration:
#   Configuration for each openstack services is stored here
#     config_files: List of config files with their specific namespaces
#     default_namespaces: List of namespaces that should be checked against each files
#     ignored_groups: List of groups that shouldn't be checked. This is passed as --exclude-group
#                     to the oslo-config-validator command.
#     opt_data: Some sections are dynamically generated like cinder's backend sections. This will
#               generate a custom opt_data using the content of template_sections as options for
#               the list of items in index_key's values. This adds a lot of overhead to the parsing
#               because we need to spawn an oslo-config-generator to pull out the default yaml
#               config and build a new data structure from it.
oslo_config_validator_service_configs:
  # https://opendev.org/openstack/nova/src/branch/master/etc/nova/nova-config-generator.conf
  nova:
    config_files:
    - path: /etc/nova/nova.conf
      namespaces: []
    opt_data:
    - group_create:
        template: os_vif_ovs
        group_name: vif_plug_ovs
    - group_create:
        template: os_vif_linux_bridge
        group_name: vif_plug_linux_brige
    default_namespaces:
    - nova.conf
    - keystonemiddleware.auth_token
    - os_vif
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.privsep
    - oslo.service.periodic_task
    - oslo.service.service
    - oslo.db
    - oslo.db.concurrency
    - oslo.cache
    - oslo.middleware
    - oslo.concurrency
    - osprofiler
  # https://opendev.org/openstack/cinder/src/branch/master/tools/config/cinder-config-generator.conf
  cinder:
    config_files:
    - path: /etc/cinder/cinder.conf
      namespaces: []
    ignored_groups:
    - nova
    - service_user
    opt_data:
    - index_key:
        section: DEFAULT
        option: enabled_backends
        separator: ','
      template_section:
      - backend_defaults
      - backend
    default_namespaces:
    - castellan.config
    - cinder
    - keystonemiddleware.auth_token
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.privsep
    - oslo.service.periodic_task
    - oslo.service.service
    - oslo.db
    - oslo.db.concurrency
    - oslo.middleware
    - oslo.concurrency
    - osprofiler
  # Glance has multiple files
  # https://opendev.org/openstack/glance/src/branch/master/etc/oslo-config-generator
  glance:
    ignored_groups:
    - ref1
    - default_backend
    config_files:
    - path: /etc/glance/glance-api.conf
      namespaces: []
    - path: /etc/glance/glance-cache.conf
      namespaces: []
    - path: /etc/glance/glance-image-import.conf
      namespaces: []
    - path: /etc/glance/glance-registry.conf
      namespaces: []
    - path: /etc/glance/glance-scrubber.conf
      namespaces: []
    - path: /etc/glance/glance-swift.conf
      namespaces: []
    default_namespaces:
    - glance
    - glance.api
    - glance.store
    - glance.multi_store
    - keystonemiddleware.auth_token
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.privsep
    - oslo.service.periodic_task
    - oslo.service.service
    - oslo.db
    - oslo.db.concurrency
    - oslo.middleware.cors
    - oslo.middleware.http_proxy_to_wsgi
  # https://opendev.org/openstack/heat/src/branch/master/config-generator.conf
  heat:
    config_files:
    - path: /etc/heat/heat.conf
      namespaces: []
    default_namespaces:
    - heat.common.config
    - heat.common.context
    - heat.common.crypt
    - heat.engine.clients.os.keystone.heat_keystoneclient
    - heat.common.wsgi
    - heat.engine.clients
    - heat.engine.notification
    - heat.engine.resources
    - heat.api.aws.ec2token
    - keystonemiddleware.auth_token
    - oslo.messaging
    - oslo.middleware
    - oslo.db
    - oslo.log
    - oslo.policy
    - oslo.service.service
    - oslo.service.periodic_task
    - oslo.service.sslutils
  # https://opendev.org/openstack/ironic/src/branch/master/tools/config/ironic-config-generator.conf
  ironic:
    config_files:
    - path: /etc/ironic/ironic.conf
      namespaces: []
    - path: /etc/ironic-inspector/inspector.conf
      namespaces: []
    default_namespaces:
    - ironic
    - ironic_lib.disk_utils
    - ironic_lib.disk_partitioner
    - ironic_lib.exception
    - ironic_lib.json_rpc
    - ironic_lib.mdns
    - ironic_lib.metrics
    - ironic_lib.metrics_statsd
    - ironic_lib.utils
    - oslo.db
    - oslo.messaging
    - oslo.middleware.cors
    - oslo.middleware.healthcheck
    - oslo.middleware.http_proxy_to_wsgi
    - oslo.concurrency
    - oslo.policy
    - oslo.log
    - oslo.reports
    - oslo.service.service
    - oslo.service.periodic_task
    - oslo.service.sslutils
    - osprofiler
    - keystonemiddleware.auth_token
  # https://opendev.org/openstack/placement/src/branch/master/etc/placement/config-generator.conf
  placement:
    config_files:
    - path: /etc/placement/placement.conf
      namespaces: []
    default_namespaces:
    - placement.conf
    - keystonemiddleware.auth_token
    - oslo.log
    - oslo.middleware.cors
    - oslo.policy
  # https://opendev.org/openstack/neutron/src/branch/master/etc/oslo-config-generator/neutron.conf
  neutron:
    config_files:
    - path: /etc/neutron/neutron.conf
      namespaces: []
    - path: /etc/neutron/plugins/ml2/ml2_conf.ini
      namespaces: []
    default_namespaces:
    - neutron
    - neutron.agent
    - neutron.base.agent
    - neutron.db
    - neutron.extensions
    - nova.auth
    - ironic.auth
    - placement.auth
    - oslo.log
    - oslo.db
    - oslo.policy
    - oslo.privsep
    - oslo.concurrency
    - oslo.messaging
    - oslo.middleware.cors
    - oslo.middleware.http_proxy_to_wsgi
    - oslo.service.sslutils
    - oslo.service.wsgi
    - keystonemiddleware.auth_token
  # https://opendev.org/openstack/keystone/src/branch/master/config-generator/keystone.conf
  keystone:
    config_files:
    - path: /etc/keystone/keystone.conf
      namespaces: []
    default_namespaces:
    - keystone
    - oslo.cache
    - oslo.log
    - oslo.messaging
    - oslo.policy
    - oslo.db
    - oslo.middleware
    - oslo.service.sslutils
    - osprofiler

# Default value for the list of services to check. Default is we check all the services
oslo_config_validator_checked_services: '{{ oslo_config_validator_service_configs.keys()
  | list }}'

Role Variables: main.yml

# While options found within the vars/ path can be overridden using extra
# vars, items within this path are considered part of the role and not
# intended to be modified.

# All variables within this role should have a prefix of "oslo_config_validator"
metadata:
  name: Openstack services configuration validation
  description: >
    This role is intended to leverage the `oslo-config-validator` on each one
    of the configuration files found on a deployment. The goal is to quickly
    catch erroneous configurations.

    When called manually, it will also be possible to generate a report
    returning all the differences between the current configuration and the
    default configuration
  groups:
  - backup-and-restore
  - pre-upgrade
  - post-deployment
  - post-system-upgrade
  - post-update

# Placeholder variables
config_locations: {}
config_validations: []
validation_output: []
invalid_setting_regex: (is not part of the sample config|not found)

Molecule Scenarios

Molecule is being used to test the “oslo_config_validator” role. The following section highlights the drivers in service and provides an example playbook showing how the role is leveraged.

Scenario: default

Driver: podman:

Molecule Platform(s)

- name: centos
  hostname: centos
  image: centos/centos:stream8
  registry:
    url: quay.io
  dockerfile: ../../../../.config/molecule/Dockerfile
  pkg_extras: python*-setuptools python*-pyyaml
  volumes:
  - /etc/ci/mirror_info.sh:/etc/ci/mirror_info.sh:ro
  privileged: true
  environment:
    http_proxy: "{{ lookup('env', 'http_proxy') }}"
    https_proxy: "{{ lookup('env', 'https_proxy') }}"
  ulimits:
  - host

Molecule Inventory

hosts:
  all:
    hosts:
      centos:
        ansible_python_interpreter: /usr/bin/python3

Example default playbook

- name: Converge
  hosts: all
  environment:
    ANSIBLE_STDOUT_CALLBACK: yaml
    ANSIBLE_LIBRARY: ../../resources/library
  tasks:
  - block:
    - name: Include the oslo_config_validator role
      include_role:
        name: oslo_config_validator
      vars:
        oslo_config_validator_debug: true
    rescue:
    - fail:
        msg: Default test failed
      when: molecule_yml.scenario.name == "default"
    - fail:
        msg: Scenario {{ molecule_yml.scenario.name }} was suppsoed to fail
      when:
      - not validation_errors | count

Scenario: mocked_failure

Driver: podman:

Molecule Platform(s)

- name: centos
  hostname: centos
  image: centos/centos:stream8
  registry:
    url: quay.io
  dockerfile: ../../../../.config/molecule/Dockerfile
  pkg_extras: python*-setuptools python*-pyyaml
  volumes:
  - /etc/ci/mirror_info.sh:/etc/ci/mirror_info.sh:ro
  privileged: true
  environment:
    http_proxy: "{{ lookup('env', 'http_proxy') }}"
    https_proxy: "{{ lookup('env', 'https_proxy') }}"
  ulimits:
  - host

Molecule Inventory

hosts:
  all:
    hosts:
      centos:
        ansible_python_interpreter: /usr/bin/python3

Example mocked_failure playbook

- name: Converge
  hosts: all
  environment:
    ANSIBLE_STDOUT_CALLBACK: yaml
    ANSIBLE_LIBRARY: ../../resources/library
  tasks:
  - block:
    - name: Include the oslo_config_validator role
      include_role:
        name: oslo_config_validator
      vars:
        oslo_config_validator_debug: true
    rescue:
    - fail:
        msg: Default test failed
      when: molecule_yml.scenario.name == "default"
    - fail:
        msg: Scenario {{ molecule_yml.scenario.name }} was suppsoed to fail
      when:
      - not validation_errors | count