Yoga Series Release Notes

12.10.0-15

New Features

  • Adds support for custom RabbitMQ configuration.

  • CentOS Stream 8 is not supported anymore. Try migration to Rocky Linux 9.

Upgrade Notes

  • CentOS Stream 8 is not supported anymore. Try migration to Rocky Linux 9.

  • Ironic Python Agent images are now built using CentOS Stream 9 by default.

Deprecation Notes

  • Support for the devicemapper Docker storage driver is deprecated following its removal from Docker Engine 25.0. Support will be fully removed in the Caracal 16.0.0 release. Operators using devicemapper should ensure that a compatible version of Docker Engine is installed (i.e. release 24.x or below).

Bug Fixes

  • Fixes default Ubuntu Apt keyrings location to the recommended /etc/apt/keyrings.

  • Fixes issues with TLS and eventlet affecting Ironic Python Agent. IPA images should be rebuilt for affected deployments.

  • Fixes an issue with overcloud service destroy where it failed to remove the inspection store docker volume. See LP#2050092.

12.10.0

Upgrade Notes

  • If the admin network does not have a gateway defined and seed_enable_snat is false, which is the default, overcloud hosts will not have a default gateway immediately after provisioning anymore. A default gateway on another network can still be applied during the host configuration step.

  • Introduces a new variable kolla_ansible_extra_custom_passwords to avoid the need to combine kolla_ansible_default_custom_passwords and kolla_ansible_custom_passwords when adding or overriding passwords.

Bug Fixes

  • Fixes an issue where local configuration generation would be skipped when running in check mode. This would lead to Kolla Ansible checking with stale configuration. See story 2010526 for details.

  • Fixes an issue where kayobe configuration dump would fail when variables are encrypted using Ansible Vault. Encrypted variables are now sanitised in the dump output. LP#2031390

  • Fixes slow fact gathering in some environments by not configuring the seed host as the initial default gateway for overcloud hosts when seed_enable_snat is false, which is the default. LP#2039461

  • Fixes an issue where the Kolla Ansible variable kolla_admin_openrc_cacert was not set to the value of kolla_internal_fqdn_cacert.

  • Adds missing disable-selinux element when building Rocky Linux overcloud host disk images.

  • Fixes gateway assignment when seed SNAT is disabled. In this circumstance Bifrost was generating ConfigDrive data with the default gateway unset even when one is available on the admin network.

  • Fixes a bug where NetworkManager would overwrite resolv.conf when resolv_is_managed is set to True. LP#2044537

  • When determining whether or not a host needs bootstrapping, we attempt to connect to the host using ansible_user, if the login fails, we then assume that the host needs bootstrapping. In previous releases we used a manually crafted ssh command. This did not respect any customisations to the SSH arguments made through Ansible configuration. We now use the raw module so that these customisations are used when connecting to the host. One possible use case is to configure a jump host between the control host and the target hosts. If bootstrapping was needed, hosts will now show as unreachable in the summary stats at the end of the run. This can safely be ignored.

  • Fixes an issue when user forgot to combine kolla_ansible_custom_passwords, kolla_ansible_default_custom_passwords and own dictionary with custom passwords in configuration files. Now kolla_ansible_extra_custom_passwords should provide only user custom passwords to add or override in kolla/passwords.yml.

  • Deploys prometheus-elasticsearch-exporter on hosts of the opensearch group, resolving failures to connect to OpenSearch when the elasticsearch and opensearch groups are different.

12.9.0

New Features

  • The Spanning Tree Protocol (STP) can now be configured on bridge interfaces. Enable or disable STP by setting the bridge_stp attribute for a network. Note that STP is not set by default on Ubuntu, but it is disabled on Rocky Linux 9 for compatibility with network scripts, as NetworkManager enables STP on all bridges by default. For CentOS Stream 8 and Rocky Linux 8 enabling STP is not supported.

  • Attempts to log in to the kolla docker registry can be skipped by setting deploy_containers_registry_attempt_login to false.

    This is required for deployments using a non-standard registry deployed on the seed during the deploy-container step, since it takes place after the registry login attempt.

Upgrade Notes

  • For Rocky Linux 9, Kayobe now disables STP on a bridge by default. This action will cause the bridge interface to restart during the host configuration process.

  • Adds an introspection rule to update the location of the deployment kernel registered in existing Ironic nodes. Nodes discovered on a deployment running the Train release or earlier may still be using the ipa.vmlinuz kernel, which stays unchanged when deployment images get updated. If only default introspection rules are in use, existing nodes may be updated from the Bifrost container with the following command:

    OS_CLOUD=bifrost baremetal introspection reprocess $NODE_UUID_OR_NAME

    If non-default rules are used, reprocessing may revert any customisation done by the operator. In this case, a more cautious approach is to update the deployment kernel location manually:

    OS_CLOUD=bifrost baremetal node set --driver-info deploy_kernel=<http://url/to/ipa.kernel> $NODE_UUID_OR_NAME

    If the kolla_bifrost_inspector_rules list is customised, the rule inspector_rule_legacy_deploy_kernel should be added to it.

Bug Fixes

  • Fixes failure to run kayobe overcloud deprovision after Bifrost is redeployed. LP#2038889

  • Improves performance of Bifrost operations by preventing unnecessary requests to the Ironic API.

  • Fixes detection of data file path when using editable installations with a recent pip.

  • Fixes the regression in configuring additional route options on CentOS / Rocky.

  • Fixed issue of seed containers being unable to use password protected registry by adding docker login function to kayobe deploy-containers role.

  • Adds a workaround to avoid NetworkManager setting the MTU of bridge VLAN interfaces to an incorrect value. LP#2039947

  • Fixes conflicts between NetworkManager nmconnection files generated by cloud-init and those generated by Kayobe by upgrading the MichaelRigart.interfaces role to version 1.14.4. LP#2039975

12.8.0

New Features

  • Adds support for custom Multipathd configuration.

  • Adds support for Rocky Linux 9 as a host Operating System and base container image.

  • Adds support for configuring arbitrarily named VLAN interfaces using systemd-networkd. See story 2010266 for details.

  • Since Kolla containers can built with user provided repos.yaml Kayobe can override the file with their own content. The override files can be ${KAYOBE_CONFIG_PATH}/kolla/repos.yaml (default Kolla filename) or ${KAYOBE_CONFIG_PATH}/kolla/repos.yml. Multiple Environments supported.

Upgrade Notes

  • OpenSearch support has been added.

  • Modifies the default value of kolla_ansible_venv_python to /usr/bin/python3. Using operating system python to create kolla-ansible venv fixes corner cases when using older venvs created with virtualenv command.

Bug Fixes

  • Fixes download of roles from Ansible Galaxy following the renaming of the mrlesmithjr.manage_lvm role. LP#2023502

  • Fixes an issue where generation of passwords.yml for Kolla Ansible could fail if the directory containing the file does not exist. This is typical in a multiple environment setup, when creating a new environment. See story 2010293 for details.

  • Fixes an issue with systemd-networkd configuration on Ubuntu with multiple VLAN interfaces. See story 2009013 for details.

  • Fixes repositories files names in Rocky Linux 9. Distributions moved to lowercase names with RHEL 9 release.

  • Fixes various issues when applying network configuration on Rocky 9 hosts. See bugs: 2016970 and 2016971.

  • Installs ncclient dependency for Juniper switch configuration when using Ansible check mode.

12.7.0

New Features

  • Adds support for Ubuntu Jammy Jellyfish (22.04) LTS as a host Operating System for seed, seed hypervisor and overcloud hosts.

12.6.0

Bug Fixes

  • Adds missing Ansible group following the addition of support in Kolla Ansible for forwarding Prometheus alerts to Microsoft Teams.

  • Fixes an error when generating passwords.yml if an unencrypted file exists but a password has been supplied.

12.5.0

New Features

  • Updates base CentOS Stream 8 cloud image to CentOS-Stream-GenericCloud-8-20220913.0.x86_64.

12.4.0

Bug Fixes

  • Fixes an issue with nclu-switch command ordering, when description was applied first to a non-existent (virtual) interface. See story 2010279 for details.

12.3.0

Upgrade Notes

  • Updates base Rocky Linux 8 cloud image to Rocky-8-GenericCloud.latest.x86_64.qcow2

Bug Fixes

  • Fixes an issue where a custom playbook using become_user could fail when setting permissions on temporary files. The acl package is now installed on all systems by default.

12.2.0

Bug Fixes

  • Fixes an issue with undefined kolla_enable_hacluster variable.

12.1.0

Bug Fixes

  • Fixes an issue where hacluster images are not built when the service is enabled.

  • Fixes an issue where the MTU defined in Kayobe was not applied to Ironic provisioning and cleaning networks in Neutron.

12.0.0

New Features

  • Adds dependencies for EFI and LVM based overcloud images.

  • Adds support for custom Placement configuration.

  • Adds support for custom Horizon themes.

  • Adds support for setting root filesystem’s UUID via a new variable kolla_bifrost_deploy_image_rootfs. This is useful when deploying overcloud hosts with software RAID based root disk devices.

  • Adds the kolla_ansible_venv_ansible configuration option. This allows you to override the version of ansible installed in the kolla-ansible virtualenv.

  • Adds support for global configuration options for Apt in files in /etc/apt/apt.conf.d/ on Ubuntu systems. See story 2009655 for details.

  • Adds support for configuring Apt repositories on Ubuntu hosts. See story 2009655 for details.

  • Add the bonding 802.3ad aggregation selection option.

  • Adds support for building overcloud root disk images directly with DIB rather than through Bifrost. This includes support for building multiple images, each with a different configuration. See story 2002098 for details.

  • Add support for a state parameter in repositories listed in dnf_custom_repos.

  • Enables hardware clock (RTC) synchronisation by default when applying the chrony role. This setting is configurable with the new variable chrony_rtcsync_enabled.

  • Improves error handling by adding a top-level playbook for the kayobe * host configure and kayobe * host upgrade commands. This ensures that if a host fails during a host configuration command, other hosts are able to continue to completion. This is useful at scale, where host failures occur more frequently. See story 2009854 for details.

  • Adds support for inspection of L3-routed Ironic networks via DHCP-relay.

  • The new filter net_no_ip adds the attribute no_ip which can be set to true to skip IP address allocation and configuration for specific networks.

  • Adds support for running a libvirt daemon on the host, rather than in a container. This is done by setting kolla_enable_nova_libvirt_container to false. See story 2009858 for details.

  • Adds support for configuring HTTP(S) proxy settings using the http_proxy, https_proxy and no_proxy variables in proxy.yml. These variables are passed down to Kolla Ansible which uses them to configure Docker, allowing container image pull operations and container networking to use HTTP(S) proxies.

  • Adds a new variable seed_hypervisor_enable_snat that allows users to enable SNAT service on the seed hypervisor. The default value is false.

  • Adds support for Rocky Linux 8 as Host OS.

  • Adds support for running package updates on Ubuntu hosts via the following existing commands:

    • kayobe seed host package update --packages <packages>

    • kayobe seed hypervisor host package update --packages <packages>

    • kayobe infra vm host package update --packages <packages>

    • kayobe overcloud host package update --packages <packages>

  • Updates base CentOS Stream cloud image to CentOS-Stream-GenericCloud-8-20210603.0.x86_64.

Upgrade Notes

  • Updates the maximum supported version of Ansible from 4.x (ansible-core 2.11) to 5.x (ansible-core 2.12). The minimum supported version is updated from 2.10 to 4.x. This is true for both Kayobe and Kolla Ansible. Note that environments with Python 3.7 or lower (e.g. CentOS Stream 8) will be limited to Ansible 4.x (ansible-core 2.11).

  • Upgrading directly from Ansible 2.10 to Ansible 4 or later is known to cause problems. You should uninstall ansible and ansible-base before upgrading your Kayobe virtual environment:

    pip uninstall ansible ansible-base

  • The default value of dnf_install_epel has been changed to false. This means that the EPEL DNF repository is no longer installed by default. Neither existing EPEL repositories nor the epel-release package will be removed. If necessary, EPEL may be enabled by setting dnf_install_epel to true in dnf.yml. See story 2009757 for details.

  • Removes support for deploying vmtp, following its removal from Kolla.

  • Starting with Yoga, Ironic has changed the default PXE from plain PXE to iPXE. Kayobe follows this upstream decision but allows users to revert to the previous default of plain PXE. For details, please refer to Kolla Ansible’s documentation.

  • The variable kolla_tls_cert which was deprecated in the Train release has been removed. Use kolla_external_tls_cert instead.

Security Issues

  • Fixes an issue where any passwords in kolla_ansible_custom_passwords were exposed in Ansible logs. When using verbosity level 3 (-vvv), they were also exposed in Ansible output.

Bug Fixes

  • Ironic inspection through Bifrost now work even if DHCP-relay is used. The dhcp-range in dnsmasq.conf corrctly configured with network mask.

  • In production environments, the provision network may be separated from the other networks, so in this case, if you want Bifrost’s DHCP service provides the correct gateway for the clients the inspection_gateway should be used instead of the gateway attribute for the provision network. This also avoids configuring the multiple IP gateways on a single host which leads to unpredictable results.

  • Fixes an issue where the Neutron SR-IOV agent image is not built when the service is enabled.

  • Fixes an issue with idempotence of local Kolla Ansible configuration generation.

  • Fixes an issue with the seed’s configdrive when the admin network is a VLAN. See story 2008089 for details.

  • Enables deployment of Grafana when Monasca is enabled, as a replacement for the retired monasca-grafana image. See story 2009717 for details.

  • Fixes some issues seen when using the --check argument with kayobe overcloud host configure. See story 2004798.

  • public-openrc.sh is now only generated if the admin-openrc.sh file generated by Kolla Ansible exists. This fixes an issue where the task would fail, when running in a clean environment, with a set of Kolla Ansible tags that did not include the generation of admin-openrc.sh. See story 2009323.

  • Fixes Ansible inventory generation with some custom group mappings using the same group names for Kayobe and Kolla Ansible. See story 2009927 for details.

  • The set of commands starting with kayobe overcloud database now generate the kolla configuration necessary to login to the nodes running the database.

  • Fixes an issue with config drive generation for infrastructure and seed VMs when using untagged interfaces. The symptom of this issue is that kayobe cannot login to the instance. If you check the libvirt console log, you will see KeyError: 'vlan_link'. See story 2009910 for details.

  • Fixes a failure to detect the Kayobe installation prefix when lib is present multiple times in the installation path. See story 2009721 for details.

  • Fixes an issue with IPA image builds which used the master branch of ironic-python-agent, even on stable releases of Kayobe, or when explicitly setting ipa_build_source_version.

  • Fixes an issue seen when using Jinja2 3.1.0.

  • Fixes an issue where any passwords in kolla_ansible_custom_passwords were exposed in Ansible logs. When using verbosity level 3 (-vvv), they were also exposed in Ansible output.

  • Fixes an issue where patch links could be erroneously created on hosts not in the overcloud group. See Story 2009911 for details.

  • Deployment image (IPA) build no longer uses master version of upper-constraints. Instead, it defaults to using the constraints for the OpenStack release associated with the version of Kayobe being used. See story 2009810 for details.

  • Fixes failures to run kayobe overcloud bios raid configure by upgrading the stackhpc.drac role to version 1.1.6.

  • Fixes an issue where cached seed VM images are unnecessarily owned by root. See story 2009277 for details.

  • Fixes an issue with masking NTP services which are not found. See story 2009821 for details.