2024.1 Series Release Notes¶
18.3.0-8¶
New Features¶
Generates a system-scoped
public-openrc-system.sh
file. This allows running Ironic commands against the public API, which is useful when access to the internal API is unavailable. LP#2051837
Bug Fixes¶
Fixes busy libvirt’s secret volume while secrets are changing. LP#2073678
Adds a check to stop deploying/upgrading the RabbitMQ containers if it will result in downgrading the version of RabbitMQ running.
Fixes a bug where the RabbitMQ version check would fail to pull the new image due to lack of auth. LP#2086171
18.3.0¶
New Features¶
Adds the ability to provide the NTP (time source) server for multiple DHCP ranges in the Ironic Inspector DHCP server.
Upgrade Notes¶
Support for failing execution early if fact collection fails on any of the hosts by setting
kolla_ansible_setup_any_errors_fatal
totrue
has been removed. This is due to Ansible’sany_errors_fatal
parameter not being templated, resulting in the value always being interpreted astrue
, even though the default value ofkolla_ansible_setup_any_errors_fatal
isfalse
.Equivalent behaviour is possible by setting the maximum failure percentage to 0. This may be done specifically for fact gathering using
gather_facts_max_fail_percentage
or globally usingkolla_max_fail_percentage
.
Bug Fixes¶
Fixes an issue with ironic dnsmasq failing to start in deployments using podman because it requires the NET_RAW capability. See LP#2055282.
Fixes kolla-ansible removing inventory file placed in
/etc/kolla/<inventory>
. See LP#2052706 for more details.
Fixes keystone service configuration for haproxy when using federation. LP#2058656
Fixes mariadb’s backup failure due to missing CREATE privileges on the mariadb_backup_history table. LP#2061889
Fixes the MariaDB recovery issue when kolla-ansible is running from a docker container. LP#2073370
Fixes #2080552.
openvswitch
role will now setexternal-ids:hostname
to{{ ansible_facts.fqdn }}
instead of{{ ansible_facts.hostname }}
due to Neutron using FQDN based hostnames inrequested-chassis
field. LP#2080552
Fix ProxySQL unable to bind due to incorrect format of IPv6 addresses in the mysql_ifaces configuration. LP#2081106
Add missing logrotate config for redis. LP#2084523
Fixes an issue during fact gathering when using the
--limit
argument where a host that fails to gather facts could cause another host to fail during delegated fact gathering.
Add
skip_kpartx yes
to multipath.confdefaults
section to prevent kpartx scanning multipath devices and unlockmultipathd del map
operation of os-brick for volume detaching oprtaions. LP#2078973 <https://launchpad.net/bugs/2078973>`__
Fixes 2067036. Added
octavia_interface_wait_timeout
to control octavia-interface.service timeout to be able wait openvswitch agent sync has been finished and octavia-lb-net is reachable from the host. Also set restart policy for this unit to on-failure LP#2067036
Fixes Octavia service upgrade issue where it can fail when Octavia persistence database user is missing. LP#2065591
Fixes unreliable health checks for neutron_ovn_agent and neutron_ovn_metadata_agent bug. Changed to check OVS DB connection instead of OVN southbound DB connection. LP#2084128
Fixes an issue, when using podman, with named volumes that use a mode specifier. See LP#2054834 for more details.
Fixes parsing of JSON output of inner modules called by
kolla-toolbox
when data was returned on standard error. LP#2080544
Updates
proxysql.yaml.j2
to use mariadb_port for backends. This fixes setups wheredatabase_port
andmariadb_port
differ.
18.2.0¶
New Features¶
Modifies public API firewalld rules to be applied immediately to a running firewalld service. This requires firewalld to be running, but avoids reloading firewalld, which is disruptive due to the way in which firewalld builds its firewall chains.
Added a command to upgrade to a target version of RabbitMQ. This is required before a SLURP upgrade. See the docs for more details: https://docs.openstack.org/kolla-ansible/latest/reference/message-queues/rabbitmq.html#slurp
Bug Fixes¶
Fixes an deploy opensearch with enable TLS on the internal VIP.
Fixes handling of openvswitch on
manila-share
nodes. LP#1993285
Adds database configuration necessary for barbican. LP#2072554
Fixes behaviour of Change Password screen in Horizon until bug #2073639 is resolved. LP#2073159
Fixes the Python requests library issue when using custom CA by adding the REQUESTS_CA environment variable to the kolla-toolbox container. See LP#1967132
Fixes the detection of the Nova Compute Ironic service when a custom host option is set in the service config file. See LP#2056571
Removes the default /tmp/ mountpoint from the horizon container. This change is made to harden the container and prevent potential security issues. For more information, see the Bug Report: LP#2068126.
Fixed an issue with the
prometheus.yml
template which would break when deploying alertmanager.
Fixes an issue where OVN northbound or southbound database deployment could fail when a new leader is elected. LP#2059124
18.1.0¶
Upgrade Notes¶
MariaDB backup now uses the same image as the running MariaDB server. The following variables relating to MariaDB backups are no longer used and have been removed:
mariabackup_image
mariabackup_tag
mariabackup_image_full
Bug Fixes¶
Fixes trove module imports. Path to the modules needed by trove-api changed in source trove package so the configuration was updated. LP#1937120
Modifies the MariaDB procedure to use the same container image as the running MariaDB server container. This should prevent compatibility issues that may cause the backup to fail.
Fixes a bug in kolla_podman_worker, where missing commas in list of strings create implicit concatenation of items that should be separate. LP#2067278
Fixes configuration of CloudKitty when internal TLS is enabled. LP#1998831
Fixes the dimensions comparison when we set values like 1g in the container dimensions configuration, making the docker container getting restarted even with no changes, as we are comparing 1g with 1073741824, which is displayed in the docker inspect while 1g is in the configuration.
Fixes keystone port in skyline-console pointing to wrong endpoint port. LP#2069855
Fixes 2065168. Fix kolla systemd unit template to prevent restart all kolla services with docker.service restart. LP#[2065168]
All stable RabbitMQ feature flags are now enabled during deployments, reconfigures, and upgrades. As such, the variable
rabbitmq_feature_flags
is no longer required. This is a partial fix to RabbitMQ SLURP support. LP#2049512
Fixes skyline’s old format of stop task. It used docker_container which would cause problems with podman deployments.
18.0.0¶
New Features¶
Added a new variable
prometheus_ceph_exporter_interval
for controlling the scrape interval of Ceph metrics.
Exposed a flag, bifrost_enable_ironic_inspector, to enable ironic-inspector in Bifrost. This option defaults to True as it can be useful for backwards compatibility. It still allows using native in-band inspection when Ironic Inspector is enabled by setting inspect_interface to agent. Please see the Ironic documentation for more details.
Implemented [Configure tap-as-a-service plugin on neutron containers]. Added the necessary changes and configurations to use the neutron plugin, tap-as-a-service, for creating port mirrors using openstack tap commands. For more information, refer to the blueprint configure-taas-plugin.
Implemented [Enable Fluentd Plugin Systemd]. Added the necessary changes and configurations to use the fluentd plugin, systemd, to read logs from /var/log/journal by default. This allows us to read and manipulate these logs for monitoring purposes.
These logs will be sent to OpenSearch by default. To disable this behavior, set the value of the variable
enable_fluentd_systemd
tofalse
in the configuration file/etc/kolla/globals.yml
.By default, when enabling central logging, we also enable the
systemd
plugin. To disable this behavior when central logging is enabled, set the value of the variableenable_fluentd_systemd
tofalse
in the configuration file/etc/kolla/globals.yml
.fluent-plugin-systemd source: https://github.com/fluent-plugin-systemd/fluent-plugin-systemd
For more information, refer to the blueprint enable-fluent-plugin-systemd.
New variables have been added to be used by the neutron role,
neutron_dns_integration
andneutron_dns_domain
. They allow enabling/disabling internal/external DNS integrations, or their combinations.
Configures the log level field for the Grafana OpenSearch datasource. This allows for logs to be coloured based on log level. To apply this you need to delete the datasource and reconfigure grafana.
Removed configuration and deployment of
prometheus-haproxy-exporter
as its repository is now archived. We now use the native support for Prometheus which is now built into HAProxy. For consistency this is exposed on theprometheus_haproxy_exporter_port
port.prometheus-haproxy-exporter
containers and config are automatically removed.
Elevated access for project-scoped service role in Ironic has been enabled. Ironic recently started to enforce new policies and scopes, and it is one of the few OpenStack projects that require system scope for some admin-related API calls. However, Ironic has also begun to allow project-scoped behavior for service roles by setting
rbac_service_role_elevated_access
. This change enables this setting to achieve similar behavior for service role as other OpenStack projects.
The service role has been added to Ironic service users. Ironic recently enforced new policy validation and added support for service roles.
Support has been added for setting the max fail percentage for Ansible plays via
kolla_max_fail_percentage
. It can also be set on a per-service basis, e.g.,nova_max_fail_percentage
.
Set a log retention policy for OpenSearch via Index State Management (ISM). Documentation.
Support for neutron-fwaas v2 has been re-added. Set enable_neutron_fwaas: yes to enable.
Configuration has been added to connect Skyline’s Prometheus to make the Monitor Center work. The latest Skyline Console now includes a Monitor Center in the administrator view that displays information from Prometheus. For this to work, the Prometheus connection needs to be set up in skyline.yaml.
The ability to override Skyline configuration files has been added. You can supply your own versions of nginx.conf for Skyline Console, gunicorn.py, and skyline.yaml for the Skyline API Server. Place the files in the skyline subfolder of your Kolla config directory; skyline.yaml will be merged with the Kolla-provided version.
More services supported by Skyline have been added to the configuration, making them accessible to Skyline’s frontend console. The new services include Barbican, Designate, Masakari, and Swift or Ceph RGW. If both Swift and Ceph RGW are enabled, only Swift is configured.
Allow to overwrite Skyline Console logos. Some of the Skyline logos can be replaced. You can now do this. See the reference documentation for details.
Single Sign-On (SSO) is enabled in Skyline Console if Keystone federation is enabled and at least one identity provider with the openid protocol is set up. Skyline Console’s redirect URI is added to Keystone’s trusted dashboards.
Upgrade Notes¶
The minimum supported Ansible version is now
8
(ansible-core 2.15), and the maximum supported is9
(ansible-core 2.16).
Support for deploying
Freezer
has been dropped.
Support for deploying
Murano
has been dropped. Additionally, support for deploying outward RabbitMQ (only used for Murano) has been dropped as well.
Support for deploying
Sahara
has been dropped.
Support for deploying
Senlin
has been dropped.
Support for deploying
Solum
has been dropped.
Support for deploying
Vitrage
has been dropped.
The configuration variable
designate_enable_notifications_sink
has been changed tono
, configuring notifications for designate in neutron, nova, and control deployment ofdesignate-sink
which is now optional.Operators who want to keep the previous behavior should set this to
true
.
The
grafana
volume is no longer used. If you wish to automatically remove the old volume, setgrafana_remove_old_volume
totrue
. Note that doing this will lose any plugins installed via the CLI directly and not through Kolla. If you have previously installed Grafana plugins via the Grafana UI or CLI, you must change to installing them at image build time. The Grafana volume, which contains existing custom plugins, will be automatically removed in the D release.
Due to the change from using the
prometheus-haproxy-exporter
to using the native support for Prometheus which is now built into HAProxy, metric names may have been replaced and/or removed, and in some cases the metric names may have remained the same but the labels may have changed. Alerts and dashboards may also need to be updated to use the new metrics. Please review any configuration that references the old metrics as this is not a backwards compatible change.
The Horizon role has been reworked to the preferred local_settings.d configuration model. Files
local_settings
andcustom_local_settings
have been renamed to_9998-kolla-settings.py
and_9999-custom-settings.py
respectively. Users who use Horizon’s custom configuration must change the names of those files in/etc/kolla/config/horizon
as well.
Added log retention in OpenSearch, previously handled by Elasticsearch Curator. By default the soft and hard retention periods are 30 and 60 days respectively. If you are upgrading from Elasticsearch, and have previously configured
elasticsearch_curator_soft_retention_period_days
orelasticsearch_curator_hard_retention_period_days
, those variables will be used instead of the defaults. You should migrate your configuration to use the new variable names before the Caracal release.
If credentials are updated in
passwords.yml
kolla-ansible is now able to update these credentials in the keystone database and in the on disk config files.The changes to
passwords.yml
are applied oncekolla-ansible -i INVENTORY
reconfigure has been run.If you want to revert to the old behavior - credentials not automatically updating during reconfigure if they changed in
passwords.yml
- you can specify this by settingupdate_keystone_service_user_passwords: false
in your globals.yml.Notice that passwords are only changed if you change them in
passwords.yml
. This mechanism is not a complete solution for automatic credential rollover. No passwords are changed if you do not change them insidepasswords.yml
.
Deprecation Notes¶
Support for deploying Masakari is no longer deprecated. The Masakari CI scenarios are now working again, and commitment has been made to improve the health of the project.
Bug Fixes¶
Introduced a separate role for managing sysctl settings. This role automatically detects support for IPv6 and, if unsupported, skips the IPv6 sysctl settings. This role expands the previous backportable fix of this issue, reviewed at https://review.opendev.org/c/openstack/kolla-ansible/+/905831. For more details, see LP#1906306.
Add conditionals for IPv6 sysctl settings that have IPV6 disabled in kernel. Changing sysctl settings related to IPv6 on those systems lead to errors. LP#1906306
Fixed nova-cell not updating the cell0 database address when VIP changes. For more details, refer to LP#1915302.
Fixes non-persistent Neutron agent state data. LP2009884
Fixes
ovs-dpdk
images pull. LP#[2041864]
Starting with ansible-core 2.13, list concatenation format is changed which resulted in inability to override horizon policy files. See LP#2045660 for more details.
Fixes long service restarts while using systemd LP#2048130.
Fixes an issue with high CPU usage of the cAdvisor container by setting the per-container housekeeping interval to the same value as the Prometheus scrape interval. LP#2048223
Fixes glance image import LP#2048525.
Fixes Nova operations using the
scp
command, such as cold migration or resize, on Debian Bookworm. LP#2048700
Fixes configuration of nova-compute and nova-compute-ironic, that will enable exposing vendordata over configdrive. LP#2049607
Fixes mariadb role deployment when using Ansible check mode. LP#2052501
Fixed an issue with
openvswitch
bridge creation whenneutron_bridge_name
was specified as two bridges. For details, see LP#2056332.
Fixed the use of Redis as coordination backend. For details, see LP#2056667.
Fixed the wrong configuration of the ovs-dpdk service, which broke the deployment of Kolla-Ansible. For details, see bug 2058372.
Incorrect condition in Podman part prevented the retrieval of facts of all the containers when no names were provided. LP#2058492
Fixes a bug where loadbalancer upgrade task fails, when podman was used as container engine. LP#2063896
Updated configuration of service user tokens for all Nova and Cinder services to stop using admin role for service_token and use service role.
See LP#[2004555] and LP#[2049762] for more details.
Fixes enabled usage audit notifications when they are not needed. See LP##2049503.
Fixed ‘cinder-backup’ service when Swift with TLS enabled. LP#2051986
The configuration of DNS integrations has been fixed. See LP##2049503, DNS Integration, and the DNS Integration with an External Service for details.
Fixes an idempotency issue in the OpenSearch upgrade tasks where subsequent runs of kolla-ansible upgrade would leave shard allocation disabled. LP#2049512
Fixes Docker health check for the
sahara_engine
container. LP#2046268
Fixed a trove deployment bug where the trove guest-agent failed to connect to RabbitMQ due to the absence of the oslo_messaging_rabbit config in guest-agent.conf. See bug 2048822 for details.
Fixed trove failing to discover the swift endpoint due to the absence of service_credentials in guest-agent.conf. See bug 2048829 for details.
Fixed an issue where the MariaDB Cluster recovery process would fail if the sequence number was not found in the logs. The recovery process now checks the complete log file for the sequence number and recovers the cluster. See LP#1821173 for details.
Fix the Octavia jobboard boolean value. See https://bugs.launchpad.net/kolla-ansible/+bug/2058046 for details.
Updated the default Grafana OpenSearch datasource configuration to use values for OpenSearch that work out of the box. Replaced the Elasticsearch values that were previously being used. The new configuration can be applied by deleting your datasource and reconfiguring Grafana through Kolla Ansible. To prevent dashboards from breaking when the datasource is deleted, one should use datasource variables <https://grafana.com/docs/grafana/latest/dashboards/variables/add-template-variables/#add-a-data-source-variable> in Grafana. See bug 2039500 <https://bugs.launchpad.net/kolla-ansible/+bug/2039500>.
Fixed bug #2039498 where the Grafana docker volume was bind mounted over Grafana plugins installed at image build time. This was fixed by copying the dashboards into the container from an existing bind mount instead of using the
grafana
volume. However, this leaves behind the volume, which can be removed by settinggrafana_remove_old_volume
totrue
. Please note that any plugins installed via the CLI directly and not through Kolla will be lost when doing this. In a future release,grafana_remove_old_volume
will default totrue
.
Added log retention in OpenSearch, previously handled by Elasticsearch Curator, now using Index State Management (ISM) OpenSearch bundled plugin. LP#2047037.
Support for friendly labels has been added for Prometheus Ironic exporter and Alertmanager metrics. See LP#2041855 for details.
A precheck has been added to catch when
om_enable_rabbitmq_quorum_queues
is set toTrue
, but quorum queues have not been configured on all appropriate queues. A manual migration is required, see here for details: https://docs.openstack.org/kolla-ansible/latest/reference/message-queues/rabbitmq.html#high-availability LP#2045887
Changes to service user passwords in
passwords.yml
will now be applied when reconfiguring services.This behaviour can reverted by setting
update_keystone_service_user_passwords: false
.Fixes LP#2045990