2023.2 Series Release Notes

15.6.0

New Features

  • Adds the internal VIP to the NOPROXY/noproxy environment variables.

  • Adds support for using Cumulus switches (NCLU and NVUE) with Networking Generic Switch.

Security Issues

  • When running API requests from a host configured with kayobe, traffic destined for the internal VIP is sent via the default proxy. This can be a security issue if not using TLS as the proxy will be able to intercept the traffic. If using an untrusted proxy, with TLS disabled on the internal VIP, it is recommended that you run kayobe overcloud host configure -t proxy, kayobe seed hypervisor host configure -t proxy, kayobe seed host configure -t proxy, and kayobe infra vm host configure -t proxy, to add the internal VIP to the no proxy configuration. This is considered a minor issue as traffic between containers will not use the proxy by default. LP#2087556

Bug Fixes

  • The proxysql image is now built when kolla_enable_proxysql is set to true.

  • Changes the default cloud image for seed and infra VMs to use Rocky Linux 9.3 when using bios boot mode, to fix boot failures with newer cloud images. When deploying new VMs, it is recommended to set infra_vm_boot_firmware and seed_vm_boot_firmware to efi.

15.5.0

New Features

  • Adds support for specifying boot_firmware and machine variables to seed and infra VMs. This can be used to launch VMs in UEFI boot mode with Q35 machine type.

  • Bumps stackhpc.libvirt-vm Ansible role to v1.16.1.

Bug Fixes

  • eos_config does not support the provider parameter since Ansible 7. Users are required to update their configuration according to Arista EOS documentation.

  • Fixes a bug where systemd-networkd was not permanently enabled when the unit was already in state runtime-enabled. LP#2073100

15.4.0

Bug Fixes

  • Added fix for the custom RabbitMQ configuration. Fixed incorrect path and glob, so now you can template also all other configuration files such as advanced.config, definitions.json, enabled_plugins, and erl_inetrc together with rabbitmq.conf and rabbitmq-env.conf.

  • Fixes an issue where Dell OS6 and Dell OS9 switch configuration was not applied correctly. LP#2061102.

  • Fixes issue of ironic files being left behind after node deprovision which prevents it from being enrolled and provisioned again.

  • Fixes default Ubuntu Apt keyrings location to the recommended /etc/apt/keyrings.

  • Fixes the bug where /etc/hosts was not populated correctly when running Kayobe using a host limit. LP#2051714

  • Fixes an issue with overcloud service destroy where it failed to remove the inspection store docker volume. See LP#2050092.

  • Fixes bugs with the kolla_enable_letsencrypt variable which were causing overcloud container image build to fail, or to include letsencrypt images when disabled.

15.3.0

Bug Fixes

  • letsencrypt and haproxy-ssh images are now built when kolla_enable_letsencrypt is set to true.

  • Fixes the wipe-disks role which was failing on supported host operating systems due to a change in the output format of lsblk -J in util-linux version 2.37. LP#2051859

15.2.0

Deprecation Notes

  • Support for the devicemapper Docker storage driver is deprecated following its removal from Docker Engine 25.0. Support will be fully removed in the Caracal 16.0.0 release. Operators using devicemapper should ensure that a compatible version of Docker Engine is installed (i.e. release 24.x or below).

15.1.0

Bug Fixes

  • Fixes gateway assignment when seed SNAT is disabled. In this circumstance Bifrost was generating ConfigDrive data with the default gateway unset even when one is available on the admin network.

  • Fixes a bug where NetworkManager would overwrite resolv.conf when resolv_is_managed is set to True. LP#2044537

15.0.0

New Features

  • The Spanning Tree Protocol (STP) can now be configured on bridge interfaces. Enable or disable STP by setting the bridge_stp attribute for a network. Note that STP is not set by default on Ubuntu, but it is disabled on Rocky Linux 9 for compatibility with network scripts, as NetworkManager enables STP on all bridges by default.

  • Adds a new kolla_bifrost_deploy_image_user_data_content variable used to define custom user_data content used by cloud-init for overcloud provision.

  • Adds initial support for configuring NVIDIA vGPUs. For more information please see the new documentation for this feature.

  • Kolla Ansible inventories in the Kayobe configuration are now passed through without modification. Previously, only group_vars were passed through. When using multiple environments, the Kolla inventory from the base configuration layer and the Kolla inventory from the Kayobe environment layer will be passed through. The inventory from the environment takes precedence over the inventory from the base layer. This allows you to put any shared configuration in the base layer.

  • Adds support for saving configuration to switches when running kayobe physical network configure. Currently only supported on Dell OS9 and OS10 devices.

  • Adds support for using Dell OS10 switches with Networking Generic Switch.

  • Adds experimental support for layering multiple environments using a .kayobe-environment file.

  • Adds new functionality to merge Kolla custom service configuration in a Kayobe environment with Kolla configuration in the base configuration layer.

  • Adds the following variables to allow more fine-grained placement of services:

    • controller_ironic_conductor_group

    • controller_ironic_inspector_group

    • controller_loadbalancer_group

    • controller_network_group

  • Adds support for building the neutron-ovn-agent image when Neutron and OVN are enabled. LP#2039558

  • Attempts to log in to the kolla docker registry can be skipped by setting deploy_containers_registry_attempt_login to false.

    This is required for deployments using a non-standard registry deployed on the seed during the deploy-container step, since it takes place after the registry login attempt.

Upgrade Notes

  • For Rocky Linux 9, Kayobe now disables STP on a bridge by default. This action will cause the bridge interface to restart during the host configuration process.

  • As Kolla Ansible inventories are now passed through without modification, the inventory directory in Kayobe configuration (etc/kayobe/kolla/inventory/) must be a valid Ansible inventory, although *.j2 files used as Kolla Ansible inventory templates are ignored. For cases where only group_vars or hosts_vars are required, a blank inventory file in the same directory may be used.

  • It is no longer possible to create an environment named kayobe. This is reserved for internal use.

  • Updates the maximum supported version of Ansible from 7.x (ansible-core 2.14) to 8.x (ansible-core 2.15). The minimum supported version is updated from 6.x to 7.x. This is true for both Kayobe and Kolla Ansible.

  • If the admin network does not have a gateway defined and seed_enable_snat is false, which is the default, overcloud hosts will not have a default gateway immediately after provisioning anymore. A default gateway on another network can still be applied during the host configuration step.

  • Removes the disable-selinux element from the default lists of DIB elements. This makes host images more compatible with the default SELinux configuration applied by Kayobe, which is permissive since the Zed release.

  • Environment-specific Kolla custom service configuration is now merged with Kolla configuration in the base configuration layer. Config options duplicated in the base layer and the environment will need to be de-deduplicated to avoid the config option showing up multiple times in the generated output (although in general this should not be a problem).

    Set kolla_openstack_custom_config_environment_merging_enabled to false to revert back to the previous behavior where only the config in the environment was considered.

  • Introduces a new variable kolla_ansible_extra_custom_passwords to avoid the need to combine kolla_ansible_default_custom_passwords and kolla_ansible_custom_passwords when adding or overriding passwords.

  • Removes the kolla_docker_registry_insecure variable from etc/kayobe/kolla.yml as it is not used since the 2023.1 (Antelope) release. The replacement docker_registry_insecure variable has been added to etc/kayobe/docker.yml.

  • Adds an introspection rule to update the location of the deployment kernel registered in existing Ironic nodes. Nodes discovered on a deployment running the Train release or earlier may still be using the ipa.vmlinuz kernel, which stays unchanged when deployment images get updated. If only default introspection rules are in use, existing nodes may be updated from the Bifrost container with the following command:

    OS_CLOUD=bifrost baremetal introspection reprocess $NODE_UUID_OR_NAME

    If non-default rules are used, reprocessing may revert any customisation done by the operator. In this case, a more cautious approach is to update the deployment kernel location manually:

    OS_CLOUD=bifrost baremetal node set --driver-info deploy_kernel=<http://url/to/ipa.kernel> $NODE_UUID_OR_NAME

    If the kolla_bifrost_inspector_rules list is customised, the rule inspector_rule_legacy_deploy_kernel should be added to it.

Deprecation Notes

  • Deprecates the following variables for removal in the Bobcat release:

    • kolla_extra_global

    • kolla_extra_aodh

    • kolla_extra_barbican

    • kolla_extra_blazar

    • kolla_extra_ceilometer

    • kolla_extra_cinder

    • kolla_extra_cloudkitty

    • kolla_extra_designate

    • kolla_extra_gnocchi

    • kolla_extra_grafana

    • kolla_extra_heat

    • kolla_extra_ironic

    • kolla_extra_inspector

    • kolla_extra_keystone

    • kolla_extra_magnum

    • kolla_extra_mariabackup

    • kolla_extra_mariadb

    • kolla_extra_manila

    • kolla_extra_masakari

    • kolla_extra_murano

    • kolla_extra_neutron

    • kolla_extra_neutron_ml2

    • kolla_extra_nova

    • kolla_extra_octavia

    • kolla_extra_placement

    • kolla_extra_sahara

    Kolla custom service configuration files in etc/kayobe/kolla/config and etc/kayobe/environments/<environment>/kolla/config should be used instead.

Bug Fixes

  • Fixes failure to run kayobe overcloud deprovision after Bifrost is redeployed. LP#2038889

  • Improves performance of Bifrost operations by preventing unnecessary requests to the Ironic API.

  • Fixes an issue where local configuration generation would be skipped when running in check mode. This would lead to Kolla Ansible checking with stale configuration. See story 2010526 for details.

  • Fixes an issue where kayobe configuration dump would fail when variables are encrypted using Ansible Vault. Encrypted variables are now sanitised in the dump output. LP#2031390

  • Fixes slow fact gathering in some environments by not configuring the seed host as the initial default gateway for overcloud hosts when seed_enable_snat is false, which is the default. LP#2039461

  • Fixes download of roles from Ansible Galaxy following the renaming of the mrlesmithjr.manage_lvm role. LP#2023502

  • Fixes an issue where the Kolla Ansible variable kolla_admin_openrc_cacert was not set to the value of kolla_internal_fqdn_cacert.

  • Fixes detection of data file path when using editable installations with a recent pip.

  • When merging Kolla configuration, fallback to templating when only one source file exists. This allows you to use configuration that templates to invalid YAML/INI as long as there is no environment override. This improves backwards compatibility where it was permitted to use such constructs.

  • Fixes the regression in configuring additional route options on CentOS / Rocky.

  • When determining whether or not a host needs bootstrapping, we attempt to connect to the host using ansible_user, if the login fails, we then assume that the host needs bootstrapping. In previous releases we used a manually crafted ssh command. This did not respect any customisations to the SSH arguments made through Ansible configuration. We now use the raw module so that these customisations are used when connecting to the host. One possible use case is to configure a jump host between the control host and the target hosts. If bootstrapping was needed, hosts will now show as unreachable in the summary stats at the end of the run. This can safely be ignored.

  • Fixed issue of seed containers being unable to use password protected registry by adding docker login function to kayobe deploy-containers role.

  • Fixes an issue when user forgot to combine kolla_ansible_custom_passwords, kolla_ansible_default_custom_passwords and own dictionary with custom passwords in configuration files. Now kolla_ansible_extra_custom_passwords should provide only user custom passwords to add or override in kolla/passwords.yml.

  • Installs ncclient dependency for Juniper switch configuration when using Ansible check mode.

  • Adds a workaround to avoid NetworkManager setting the MTU of bridge VLAN interfaces to an incorrect value. LP#2039947

  • Removes the kolla_docker_registry_insecure variable from etc/kayobe/kolla.yml as it is not used since the 2023.1 (Antelope) release. The replacement docker_registry_insecure variable has been added to etc/kayobe/docker.yml.

  • Fixes conflicts between NetworkManager nmconnection files generated by cloud-init and those generated by Kayobe by upgrading the MichaelRigart.interfaces role to version 1.14.4. LP#2039975

  • Stop merging config when a symlink points to the same file. This was a common pattern before the merging of kolla config was introduced and results in duplicated config options.