2024.2 Series Release Notes

26.1.1-8

Bug Fixes

• Fixes newly added policy rules, baremetal:node:set_provision_state:clean_steps and baremetal:node:set_provision_state:service_steps``which impacted ``project scoped users utilizing the 2024.2 release of Ironic where they were attempting to invoke service or clean provision state commands. This was due to a misunderstanding of the correct policy checker to invoke, and additional testing has been added around these functions to ensure they work as expected moving forward.

• The configuration option [inspector]power_off is now actually ignored for nodes with fast track enabled, as documented in its help.

• Fixes the built-in in-band inspection implementation to power off the node after aborting inspection on user’s request, unless the node is in the fast track mode or [inspector]power_off is set to False.

• Update the node cache after a successful servicing and cleaning. This ensures the node information is correctly updated in the database.

26.1.1

Security Issues

• An issue in Ironic has been resolved where image checksums would not be checked prior to the conversion of an image to a raw format image from another image format.

  With default settings, this normally would not take place, however the image_download_source option, which is available to be set at a node level for a single deployment, by default for that baremetal node in all cases, or via the [agent]image_download_source configuration option when set to local. By default, this setting is http.

  This was in concert with the [DEFAULT]force_raw_images when set to True, which caused Ironic to download and convert the file.

  In a fully integrated context of Ironic’s use in a larger OpenStack deployment, where images are coming from the Glance image service, the previous pattern was not problematic. The overall issue was introduced as a result of the capability to supply, cache, and convert a disk image provided as a URL by an authenticated user.

  Ironic will now validate the user supplied checksum prior to image conversion on the conductor. This can be disabled using the [conductor]disable_file_checksum configuration option.

Bug Fixes

• Fixes a security issue where Ironic would fail to checksum disk image files it downloads when Ironic had been requested to download and convert the image to a raw image format. This required the image_download_source to be explicitly set to local, which is not the default.

  This fix can be disabled by setting [conductor]disable_file_checksum to True, however this option will be removed in new major Ironic releases.

  As a result of this, parity has been introduced to align Ironic to Ironic-Python-Agent’s support for checksums used by standalone users of Ironic. This includes support for remote checksum files to be supplied by URL, in order to prevent breaking existing users which may have inadvertently been leveraging the prior code path. This support can be disabled by setting [conductor]disable_support_for_checksum_files to True.

• Fixes aborting in-band inspection. Previously, it would fail with Can not transition from state 'inspect failed' on event 'abort'.

26.1.0

Prelude

The Ironic project is pleased to announce the release Ironic 26.1, also known as the 2024.2 or Dalmatian release. This release adds functionality to enable the creation and use of runbooks of repeated steps in cleaning and service states. Furthermore, cloud admins are now able to lease baremetal nodes to Nova users automatically through the use of metadata provided by Nova. Firmware updates have improved capability and ease-of-use on BMCs which manage multiple hosts. Also users of the redfish hardware type can now invoke firmware updates as a service step. Operators seeking to migrate away from BIOS booting now have additional options to help enforce migration mandates. Coupled with numerous bug fixes and other minor enhancements, this release of Ironic will improve the life of operators for years to come. We hope you enjoy!

New Features

• Adds a new API concept, runbooks, to enable self-service of maintenance items on nodes by project members.

  Runbooks are curated lists of steps that can be run on nodes only associated via traits and used in lieu of an explicit list of steps for manual cleaning or servicing.

• Adds a new top-level REST API endpoint /v1/runbooks/ with basic CRUD support.

• Extends the /v1/nodes/<node>/states/provision API to accept a runbook ident (name or UUID) instead of clean_steps or service_steps for servicing or manual cleaning.

• Implements RBAC-aware lifecycle management for runbooks, allowing projects to limit who can CRUD and use a runbook.

• Ironic now supports automatically setting node.lessee at deployment time using metadata provided at deploy time, typically by OpenStack Nova. When [conductor]/automatic_lessee_source is set to instance, Ironic will set the lessee field on the node and remove it before cleaning.

• Adds a new capability allowing to fetch the list of virtual media devices attached to a node by making a GET request.

• Make the idrac hardware type inherit from the redfish hardware type since the idrac hardware type is an extension of the redfish with Dell specific overrides. This will ensure that features available to the redfish hardware type will always be available to idrac. Added redfish interface as available for the bios, power and vendor interfaces of the idrac hardware type.

Upgrade Notes

• When upgrading Ironic to address the qemu-img image conversion security issues, the ironic-python-agent ramdisks will also need to be upgraded.

• When upgrading Ironic to address the qemu-img image conversion security issues, the [conductor]conductor_always_validates_images setting may be set to True as a short term remedy while ironic-python-agent ramdisks are being updated. Alternatively it may be advisable to also set the [agent]image_download_source setting to local to minimize redundant network data transfers.

• As a result of security fixes to address qemu-img image conversion security issues, a new configuration parameter has been added to Ironic, [conductor]permitted_image_formats with a default value of “raw,qcow2,iso”. Raw and qcow2 format disk images are the image formats the Ironic community has consistently stated as what is supported and expected for use with Ironic. These formats also match the formats which the Ironic community tests. Operators who leverage other disk image formats, may need to modify this setting further.

• [conductor]/automatic_lessee has been deprecated in favor of [conductor]/automatic_lessee_source.

  Standalone Ironic deployments previously setting automatic_lessee to True now may want to set automatic_lessee_source to request to retain existing behavior.

  Deployers explicitly setting automatic_lessee to false may want to set automatic_lessee_source to none to retain existing behavior. The old configuration option, when explicitly set, will be honored until fully removed.

• Ironic will now automatically set the node.lessee field for all deployments by default when provided in node instance_info at deployment time. Deployers are encouraged to review their security settings and Ironic Secure RBAC documentation to ensure no unexpected access is granted.

• Ironic now requires rescue passwords to be hashed. Operators who would like to continue using unhashed passwords must set [conductor]/require_rescue_password_hashed to false.

Deprecation Notes

• The [[agent]]manage_agent_boot configuration directive is being deprecated. It is completely untested, and requires operators to manually configure significant parts of infrastructure typically handled by Ironic. Operators using this configuration are advised to migrate away from it before it’s scheduled removal during the 2025.2 cycle releases, coming out late 2025.

• Deprecates the idrac-redfish interfaces in favor of the redfish interfaces for the bios, power, and vendor interfaces. This is a no-op change as these interfaces wrapped the redfish interface with no change already.

Security Issues

• Ironic now checks the supplied image format value against the detected format of the image file, and will prevent deployments should the values mismatch. If being used with Glance and a mismatch in metadata is identified, it will require images to be re-uploaded with a new image ID to represent corrected metadata. This is the result of CVE-2024-44082 tracked as bug 2071740.

• Ironic always inspects the supplied user image content for safety prior to deployment of a node should the image pass through the conductor, even if the image is supplied in raw format. This is utilized to identify the format of the image and the overall safety of the image, such that source images with unknown or unsafe feature usage are explicitly rejected. This can be disabled by setting [conductor]disable_deep_image_inspection to True. This is the result of CVE-2024-44082 tracked as bug 2071740.

• Ironic can also inspect images which would normally be provided as a URL for direct download by the ironic-python-agent ramdisk. This is not enabled by default as it will increase the overall network traffic and disk space utilization of the conductor. This level of inspection can be enabled by setting [conductor]conductor_always_validates_images to True. Once the ironic-python-agent ramdisk has been updated, it will perform similar image security checks independently, should an image conversion be required. This is the result of CVE-2024-44082 tracked as bug 2071740.

• Ironic now explicitly enforces a list of permitted image types for deployment via the [conductor]permitted_image_formats setting, which defaults to “raw”, “qcow2”, and “iso”. While the project has classically always declared permissible images as “qcow2” and “raw”, it was previously possible to supply other image formats known to qemu-img, and the utility would attempt to convert the images. The “iso” support is required for “boot from ISO” ramdisk support.

• Ironic now explicitly passes the source input format to executions of qemu-img to limit the permitted qemu disk image drivers which may evaluate an image to prevent any mismatched format attacks against qemu-img.

• The ansible deploy interface example playbooks now supply an input format to execution of qemu-img. If you are using customized playbooks, please add “-f {{ ironic.image.disk_format }}” to your invocations of qemu-img. If you do not do so, qemu-img will automatically try and guess which can lead to known security issues with the incorrect source format driver.

• Operators who have implemented any custom deployment drivers or additional functionality like machine snapshot, should review their downstream code to ensure they are properly invoking qemu-img. If there are any questions or concerns, please reach out to the Ironic project developers.

• Operators are reminded that they should utilize cleaning in their environments. Disabling any security features such as cleaning or image inspection are at your own risk. Should you have any issues with security related features, please don’t hesitate to open a bug with the project.

• The [conductor]disable_deep_image_inspection setting is conveyed to the ironic-python-agent ramdisks automatically, and will prevent those operating ramdisks from performing deep inspection of images before they are written.

• The [conductor]permitted_image_formats setting is conveyed to the ironic-python-agent ramdisks automatically. Should a need arise to explicitly permit an additional format, that should take place in the Ironic service configuration.

Bug Fixes

• Adds microversion headers to the root (‘/’) endpoint.

• Fixes multiple issues in the handling of images as it relates to the execution of the qemu-img utility, which is used for image format conversion, where a malicious user could craft a disk image to potentially extract information from an ironic-conductor process’s operating environment.

  Ironic now explicitly enforces a list of approved image formats as a [conductor]permitted_image_formats list, which mirrors the image formats the Ironic project has historically tested and expressed as known working. Testing is not based upon file extension, but upon content fingerprinting of the disk image files. This is tracked as CVE-2024-44082 via bug 2071740.

• Fixes inspection failure when bmc_address or bmc_v6address is null in the inventory received from the ramdisk.

• The network_data fetched from Neutron contained ‘links’, ‘networks’ but was missing ‘services’. This patch brings in ‘services’ to include dns nameservers that can be configured by Glean or cloud-init during cleaning and provisioning operations, especially when virtual media boot is used without DHCP.

• Set node “alive” and make it fast trackable as soon as inspection is finished, in addition add a wait for the agent to callback should it not be available when fast track is attempted.

• Replaces ari/aki format references with appropriate artifacts for kernel and ramdisk in the documentation and Ironic DevStack plugin.

• Update kernel_append_params to match the [pxe] configuration, addressing a TODO from the Xena release.

Other Notes

• Removes support for pre-SQLAlchemy 2.0 query objects from the internal database API. Downstream plug-ins must be adjusted to use the new-style queries.

26.0.0

New Features

• Adds support for updating BIOS in configurations where a single BMC is managing multiple systems (e.g. sushy-tools emulator with multiple VMs). In such cases, Targets parameter is added to SimpleUpdate API call.

• Adds configuration options for operators to specify any or what boot modes to disallow for enrollment (disallowed_enrollment_boot_modes) and/or deployment (disallowed_deployment_boot_modes). Defaults are empty lists, indicating all boot modes are allowed.

• Adds a new configuration option store_cred_in_env to allow switching between file-based and environment variable persistence for IPMI credentials. Defaults to False.

• Makes redfish driver firmware update feature a service step, enabling operators to perform firmware updates on active nodes.

Upgrade Notes

• Adds upgrade checks for the following situations:

  • Error on unknown hardware types or interfaces in the configuration.

  • Warning on deprecated hardware types or interfaces in the configuration.

  • Warning on unknown hardware types or interfaces used on any nodes.

• The deprecated ibmc hardware type has been removed from Ironic.

• The deprecated idrac-wsman and related idrac interface aliases have been removed from the idrac hardware type.

• The deprecated xclarity hardware type has been removed from Ironic.

Security Issues

• Log the node UUID instead of the full node object in ironic/conductor/cleaning.py, to avoid logging the node’s driver_info property (containing its BMC username and password).

• Agent communication now requires an HTTPS url by default. This can be changed using the [agent]require_tls setting.

Bug Fixes

• [bug 2069413] Fixes an issue with node servicing that caused node to be put into ‘service failed’ state when Ironic configuration option [pxe]enable_netboot_fallback was enabled.

• [bug 2069430] Fixes an issue that prevented Ironic from being able to execute node servicing steps exposed by IPA’s HardwareManager

• Fixes an issue in Redfish generic attach and detach virtual media where the virtual media devices were not recognized causing a failure when attaching or detaching a single virtual media device.

• Fixes the default grub initial configuration to be simpler and directly load the generated configuration file. The template also includes output which also help operators understand the context as to where the node is booting from, should issues be encountered.

• Replaces deprecated Storage.StorageControllers in Redfish RAID with Storage.Controllers, which provides an array of links to controller objects instead of embedding the full controller objects. The old field is now used as a fallback.

• During node deployment, unless explicitly configured otherwise, Ironic now only creates PXE link files for ports with pxe_enabled=True, preventing unintended booting from disabled ports.

Other Notes

• The Redfish interoperability profile 1.0.0 has been replaced with version 1.1.0 that adds a lot of missing resources and fields, and clarifies their purpose.

25.0.0

New Features

• Adds additional validation to the agent callback_url.

• Delegate parsing of version headers in API requests to the microversion-parse library which also adds support for the new standard singular header: ‘OpenStack-API-Version: baremetal <version>’.

Upgrade Notes

• python-cinderclient is no longer a dependency, all OpenStack Cinder operations are now done using openstacksdk.

• python-glanceclient is no longer a dependency, all OpenStack Glance operations are now done using openstacksdk.

• The default value for [redfish]use_swift has been changed to false. This is to limit URL validation challenges presented by some baseboard management controllers where characters in the Swift temporary URL form are rejected by Baseboard Management Controllers.

• API version 1.91 removes special treatment given to URLs ending in “.json”. Operators desiring the previous behaviour can request API version 1.90 or earlier.

Security Issues

• Additional validation of the callback_url which is supplied to Ironic by the agent has been added. In addition to any standardized formatting checks included in Python urllib, we will also reject requests which have an invalid URL schema formatting.

Bug Fixes

• Fixes an issue with units tests that show this DeprecationWarning: The metaschema specified by $schema was not found. Using the latest draft to validate, but this will raise an error in the future. cls = validator_for(schema) Removed the warning for deprecated schema by using a new template.

• Previously the conductors online database column is not considered when displaying the “baremetal conductor list” Alive status. This means that when a conductor is stopped gracefully it will be shown as (inaccurately) alive for the duration of [conductor]heartbeat_timeout.

  A conductor is now considered alive if online is true and there is a recent enough heartbeat.

• Fixes the issue of service steps not starting due to servicing states (states.SERVICING and states.SERVICEWAIT) missing from _FASTTRACK_HEARTBEAT_ALLOWED constant.

• Fixes issue with configuring virtual media boot for executing service steps by adding missing entries for states.SERVICING and states.SERVICEWAIT in the whitelist of the states allowed by this method.

• [bug 2011053] Fix issue with boot from volume feature. Convert lun field from decimal to hexadecimal when generating iscsi url so that ipxe firmware could be able to identify the iSCSI SAN URI correctly, according to SAN URIs description at https://ipxe.org/sanuri.

• Fixes usage of redfish detach virtual media feature to be conform to the general implementation. Before the detach virtual media API call using redfish driver was not working as intended and caused the operation to fail.

• Fixes an issue in redfish attach/detach generic virtual media where the attached devices are not correctly recognized causing the attach operation to fail.

• No longer falls back to unmanaged inspection for virtual media and UEFI HTTP boot interfaces. Previously, if the validation of the boot interface failed before in-band inspection, Ironic would switch to unmanaged inspection, which involved PXE-booting. It is not expected that nodes explicitly configured to use virtual media start booting over PXE, so the fallback has been removed.

• Service step validation no longer requires a priority field, which is not supported for servicing.

• Fixes service steps that rely on a reboot. Previously, the reboot was not properly recognized in the conductor logic.

• Ironic now stops any active IPMI Serial-Over-LAN console sessions when initializing a console session. This resolves and issue where console support would fail if a previous console session was not properly disconnected.

• Special treatment of .json is now disabled for nodes with .json extension in URL field.

  See bug 1748224 for more details.

• Adds an ISO publisher value to ISO images which are mastered as part of cleaning/deployment/service operations in support of a fix for bug 2032377.

• Fixes generated URL when using the virtual media attachment API. Previously, it missed the node UUID, causing conflicts between different nodes.

Other Notes

• Moving forward, Ironic will discourage the use of “partition” images, in favor of “whole disk images”, largely due to the underlying complexity in the code to maintain partition image support. This is not the deprecation of the functionality as the community has agreed to keep the functionality and fix any issues we become aware of. UEFI Partition images, where EFI assets are copied from a partition image, are the only partition images Ironic intends to test on a regular basis moving forward.