2023.1 Series (11.2.0 - 11.4.x) Release Notes

2023.1-eom

Bug Fixes

  • Fixes an issue where inspection would fail if an IPv6 address wrapped in brackets is used for the redfish BMC address. See bug: 2036455.

  • In case the LLDP raw data collected by the inspection process includes non UTF-8 information, the parser fails breaking the inspection process. This patch works around that excluding the malformed data and adding an entry in the logs to provide information on the failed tlv.

  • Fixes the Role Based Access Control state and capabilities to align with OpenStack Community RBAC goals which includes support for a service role by default to enable inter-service communication to be configured without an admin username. In large part, these changes were missed as the Inspector service is considered an “admin-only” service.

    Also in alignment with overall community position changes, where the admin role is sufficient without an explicit system scope. To help ensure a high level of security, explicit testing was also added for the manager role, which is unavailable as that role is reserved for administrative functions inside of a tenant’s project.

11.3.0

Upgrade Notes

  • The minimum version of SQLAlchemy is now 1.4.0, in preparation for the future anticipated release of SQLAlchemy 2.0.0.

  • The minimum version of Oslo.DB is now 12.1.0, in preparation for the future anticipated release of SQLAlchemy 2.0.0.

  • Database schema upgrades from versions prior to 7.3.0 are not supported. Please upgrade to an intermediate release prior to upgrading to this release.

Deprecation Notes

  • Plugin maintainers should be aware that the Node Cache object field version_id filed is no longer in use. It is still returned by the data model if stored for the purposes of compatibility, but Inspector will not update the field through the normal course of its operation.

Bug Fixes

  • Fixes an issue where database responses of nodes would get orphaned in inspector process RAM, and would not be garbage collected. We were able to discover and reproduce this issue while working on database connectivity locks remaining in place. Please see story 2009727 for more details.

Other Notes

  • Plugin maintainers who are directly working with the database will need to update their plugins. Specifically, the Database API has been delineated into using enginefacade with a dedicated reader and writer model, in anticipation of support for SQLAlchemy 2.0 and an eventual merge of Inspector into Ironic at some point in the future. Database actions are now performed through the ironic_inspector.db.api module, where previously they were spread across ironic_inspector.db and ironic_inspector.node_cache.

11.1.0

New Features

  • Follow the same process for determining the root device as Ironic Python Agent which has been changed to accommodate for the feature enabling users to specify a list of devices that should be skipped during cleaning/deployment The field skip_block_devices is one of the properties of a node

10.11.0

New Features

  • Supports listening on a Unix socket instead of a normal TCP socket. This is useful with an HTTP server such as nginx in proxy mode.

10.10.0

Known Issues

  • The response headers for empty body HTTP 204 replies, at present, violate RFC7230. This was not intentional, but underlying libraries also make inappropriate changes to the headers, which can cause clients to experience odd failures. This is anticipated to be corrected once an underlying issue in eventlet is resolved.

Upgrade Notes

  • The rootwrap rule to allow restarting the systemd service openstack-ironic-inspector-dnsmasq.service has been removed. No known tooling requires this rule since before Train. Any configuration tool which is setting [dnsmasq_pxe_filter]dnsmasq_start_command also needs to be writing an appropriate rootwrap.d file, as the inspector devstack plugin does.

Bug Fixes

  • Fixes HTTP responses so the Eventlet library, which is used to support the operation of the WSGI application, does not incorrectly inject a Transfer-Encoding header into the HTTP response, even on HTTP 204 replies, which is a violation of RFC7230. This header ultimately can cause varying client reactions which are not expected and can raise exceptions. For now, this has been remedied via an explicit return of a Content-Length header, which is also an RFC7230 violation, but it appears to be the lesser of known evils at this time.

10.9.0

New Features

  • Adds support for filtering by state in the list introspection API. See story 1625183.

    • GET /v1/introspection?state=starting,...

10.8.0

New Features

  • The new [healthcheck] enabled option has been added. When this option is set to True, the healthcheck middleware is enabled in the API pipeline and the additional API endpoint to monitor service availability becomes available at /healthcheck path.

Bug Fixes

  • Inspector now ignores failures to list ironic ports during PXE filter driver sync, and just skips the sync in this case. Previously such errors resulted in the PXE filter driver being stuck in an uninitialized state until Ironic inspector was restarted. See bug 2008971.

  • Fixes issues in Inspector where various tasks would not have retry logic applied to them and may sporadically fail. This is because the OpenStack SDK does not comprehend the NodeLocked error, which previously python-ironicclient silently handled. Basic operations such as “power reboot” and “set boot device” will now be retried automatically if they fail. For more information, please see story 2009107.

10.7.0

Bug Fixes

  • Fixes an issue where a failed inspection due to a transient failure can prevent retry attempts to inspect to be perceived as a failure. If a prior inspection fails and is in error state, when a new introspection is requested, the state is now appropriately set to starting.

10.6.0

New Features

  • The default policy will be replaced with one which aligns with the Secure-RBAC scopes and roles. Since ironic-inspector is a tool used only by system-level admins, only the system scope is supported, and the only roles in the policy rules are admin and reader.

Upgrade Notes

  • [DEFAULT]/ipmi_address_fields now has ibmc_address in the default configuration, allowing introspection to try and match the BMC address if no ports are defined when using the ibmc driver.

  • The default value of [oslo_policy] policy_file config option has been changed from policy.json to policy.yaml. Operators who are utilizing customized policy files or previously generated static policy files (which are not needed by default), should generate new policy files and modify them to meet their needs in the event of any new policies or rules have been added. Please consult the oslopolicy-convert-json-to-yaml tool to convert a JSON to YAML formatted policy file in backward compatible way.

  • The new policy is only enforced when [oslo_policy] config is changed to enforce_new_defaults=True and enforce_scope=True, otherwise the existing deprecated policy is used. User accounts which rely on having the baremetal_admin or baremetal_observer roles will need to have system-scoped admin or reader roles to use the API when the new policy is enforced.

Deprecation Notes

  • The use of legacy policy files was deprecated by the oslo.policy library during the Victoria development cycle. As a result, this deprecation is being noted in the Wallaby with an anticipated future removal of support by oslo.policy. As such operators will need to convert to YAML policy files. Please see the upgrade notes for details on the migration of any custom policy files.

  • The previous policy is still enforced by default, but is now deprecated and will be removed in a future release.

10.5.0

New Features

  • Adds a possibility to setup ironic inspector behind a proxy, while allowing the links of the resources API returns to remain correct. Inspector now respects the following headers that are passed with API requests: X-Forwarded-For, X-Forwarded-Proto, X-Forwarded-Host, X-Forwarded-Port, X-Forwarded-Prefix. If the API is run providing SCRIPT_NAME environment variable, it is now also respected, and it allows to return the correct links in response to requests, even if inspector API is not placed at the web server root resource.

Bug Fixes

  • Fixes database migrations with SQLAlchemy 1.3.20.

10.4.0

New Features

  • Adds an accelerators plugin to identify accelerator devices and update the bare metal node for future scheduling. The accelerator devices will be saved to node properties under the key accelerators. Introduces a configuration option [accelerators]known_devices to specify a configuration file which contains required information to identify accelerator devices, by default it uses the in-tree configuration file named known_accelerators.yaml.

  • The dnsmasq pxe-filter now supports mapping between host InfiniBand MAC to EthernetOverInfiniBand MAC. (This was previously only supported by the iptables pxe-filter.)

  • By default the DHCP filtering will open the DHCP server for any node when introspection is active. It will only block DHCP for enrolled nodes that are not being introspected. Doing so is required to support interface discovery (which by default will enrol the PXE port to Ironic if not present). This behaviour is not always wanted, as nodes not managed by Ironic may boot the inspection image.

    A new option was added [pxe_filter]deny_unknown_macs which allow changing this behaviour so that the DHCP server only allow enrolled nodes being introspected and deny everything else.

    Note

    If this option is True, nodes must have at least one enrolled port prior to introspection.

Bug Fixes

  • Fixes the node identification logic to enable a user to list the redfish_address label for driver_info field values for identification of a machine using the [DEFAULT]ipmi_address_fields configuration option. Previously the host would just not be matched as the full URL would be evaluated instead of what the URL may resolve to.

10.3.0

New Features

  • The new API GET /v1/introspection/<node>/data/unprocessed allows retrieving raw (unprocessed) data if data store is enabled.

Upgrade Notes

  • API now listens on :: by default, change the listen_address configuration option to modify.

Bug Fixes

  • The extra_hardware processing hook no longer refuses to parse extra data if some records are empty or have unexpected length. These records are now discarded.

    The previous behaviour can be returned by setting the new option [extra_hardware]strict to True.

  • The extra_hardware processing hook no longer removes the incoming data object if it has unexpected data format, assuming that this object is used for something else.

    The previous behaviour can be returned by setting the new option [extra_hardware]strict to True.

  • Using auth_strategy=http_basic incorrectly required authentication for public paths such as / and /v1. These paths are now public.

  • Fixes an issue which may occur with Apache httpd webservers acting as a proxy where the server may report Bad Gateway, however inspector continues operating as if there was no problem. This was due to a lack of a Content-Type header on HTTP 202 and 204 replies, and lack of message body with HTTP 202 messages which Apache httpd can error upon.

  • No longer tries to set local_gb to -1 if the matched root device has size of zero.

10.2.0

New Features

  • Adds the ability for periodic clean-up and synchronization tasks with ironic to be able to be disabled by setting the [DEFAULT]clean_up_period to a value of 0. This is intended for “stand-alone” operators only as it may result in unexpected behaviours if used in a non-standalone environment.

  • Adds a new configuration option [discovery]enroll_node_fields that specifies additional fields to set on a node (e.g. driver interfaces).

  • Enable Basic HTTP authentication middleware.

    When the config option [DEFAULT]auth_strategy is set to http_basic then non-public API calls require a valid HTTP Basic authentication header to be set. The config option [DEFAULT]http_basic_auth_user_file defaults to /etc/ironic-inspector/htpasswd and points to a file that supports the Apache htpasswd syntax[1]. This file is read for every request, so no service restart is required when changes are made.

    The only password digest supported is bcrypt, and the bcrypt Python library is used for password checks since it supports $2y$ prefixed bcrypt passwords as generated by the Apache htpasswd utility.

    To try basic authentication, the following can be done:

    • Set /etc/ironic-inspector/inspector.conf [DEFAULT]auth_strategy to http_basic

    • Populate the htpasswd file with entries, for example: htpasswd -nbB myName myPassword >> /etc/ironic-inspector/htpasswd

    • Make basic authenticated HTTP requests, for example: curl --user myName:myPassword http://localhost:6385/v1/introspection

    [1] https://httpd.apache.org/docs/current/misc/password_encryptions.html

  • Adds periodic leader election for the cleanup sync with Ironic. The election interval is configured by the new leader_election_interval config option.

  • Adds a configuration option [processing]update_pxe_enabled to control whether the pxe_enabled should be updated according to introspection data for ports. The default value is True which is backwards compatible.

Upgrade Notes

  • Remove upper constraint for Python construct library and use the latest version available. The minimum compatible version for Python construct is now 2.9.39

  • The raw data from the extra_hardware processing hook is no longer stored in Swift in an object named extra_hardware-<node UUID>. The same information is already available as part of the unprocessed introspection data without a hard dependency on Swift.

Deprecation Notes

  • The deprecated [swift]max_retries parameter has been removed.

Bug Fixes

  • Fixes an issue where IPv6 link local addresses are ignored during interface validation, making introspection fail.

  • Fixes AttributeError: 'Node' object has no attribute 'uuid' when trying to introspect an active node that is not currently in the cache.

  • No longer aborts the whole process if one periodic task fails.

  • Fixes accessing API endpoints with trailing slashes. Now they’re treated the same way as without slashes, although the latter remain canonical URLs.

  • No longer uses introspection delay for nodes with manage_boot==False (i.e. boot is managed by ironic). It is useless and may actually break introspection if a node boots before it gets whitelisted in the PXE filter.

  • The introspection start API is now synchronous when manage_boot==False. This means that any failures will be propagated to Ironic, preventing it from powering a node on and booting it without the PXE filter updated.

10.1.0

New Features

  • Added the capability to define a scope for the inspection process. Previously, all introspection rules were applied when inspecting any node. There was no mechanism to apply only a selected set of rules. This change introduces a scope field to introspection rules. If a scope is set on an introspection rule, it will only apply to nodes that have a matching inspection_scope property. If not set, it will apply to all nodes.

  • Added physnet_cidr_map processing plugin, the plugin uses the IP address of interfaces returned during inspection and set the port physical_network via lookup from a CIDR to physical network mapping in config option [port_physnet]/cidr_map.

Upgrade Notes

  • The python-ironicclient package has been removed as a dependency in favour of openstacksdk. Third party modules and plugins will require an update if they previously invoked ironicclient.

Other Notes

  • The devstack plugin for ironic-inspector has been changed to utilise pre-built ironic-python-agent images based on Centos8 instead of legacy CoreOS based images.

  • Added base class (BasePhysnetHook) for plugins that assign a physical network to ports.

10.0.0

Upgrade Notes

  • Python 2.7 support has been dropped. Last release of ironic-inspector to support Python 2.7 is OpenStack Train. The minimum version of Python now supported by ironic-inspector is Python 3.6.

Bug Fixes

  • Fixes an issue during manual inspection of active nodes where the node UUID was not passed back to the inspector when it tried to identify a matching port.

  • No longer tries to power off nodes after introspection if manage_boot is False.

  • Introspection now respects the force_persistent_boot_device parameter in a node’s driver_info.

  • Fixes an issue happening during manual inspection of active nodes where the code attempts to delete or update ports, while the only modification allowed for active nodes is updating the MAC address if the node is in maintenance.

9.2.0

Prelude

The Train release of Ironic Inspector features support for running separate API and conductor services.

New Features

  • Allows splitting the ironic-inspector service into ironic-inspector-api and ironic-inspector-conductor which coordinate via Tooz and its underlying backend. A new configuration option [DEFAULT]standalone is introduced to enable this feature. The configuration defaults to True, and ironic-inspector runs as a single service, which is compatible with the old behaviour. When set to False, ironic-inspector-api-wsgi is used to start the API service, and ironic-inspector-conductor is used to start the conductor service. For ironic-inspector running in non-standalone mode, the user needs to set the new configuration option [coordination]backend_url, which specifies the backend used for coordination.

Upgrade Notes

  • Updates the default Ironic API version to 1.56, which is the most recent version in the Stein series Bare Metal release (12.1.0).

Bug Fixes

  • Fixes introspection of active nodes that are not in the lookup cache, see story 2006233.

9.1.0

New Features

  • Adds the capability for introspection data to be posted to the API when a baremetal node is in active or rescue states. This feature may be useful for data centre operators who wish to update introspection data periodically.

    To enable this feature, set [processing]permit_active_introspection to True. When this is set, the value of [processing]power_off is overridden for nodes in active or rescue states.

  • Adds support to enroll node with IPv6 BMC address. Introduces a configuration option [discovery]enabled_bmc_address_version to specify the order of preferred IP version of the BMC address.

Upgrade Notes

  • The deprecated options from the ironic section os_region, auth_strategy, ironic_url, os_service_type and os_endpoint_type have been removed. Please use keystoneauth options instead.

  • The deprecation configuration options os_service_type, os_region and os_endpoint_type from the [swift] section have been removed.

Deprecation Notes

  • The configuration option [swift]max_retries is deprecated. It has been doing nothing for a few releases already.

Bug Fixes

  • No longer fails introspection if memory or CPU information is not provided in the inventory. These are no longer required for scheduling, introspection should not require them either.

9.0.0

New Features

  • A new option enable_mdns allows to enable publishing the baremetal introspection API endpoint via mDNS as specified in the API SIG guideline.

  • Adds support to reapply with provided unprocessed introspection data. The introspection data is supplied in the body of POST request to /v1/introspection/<node_id>/data/unprocessed. The introspection data will also be saved to storage backend.

Upgrade Notes

  • The deprecated SSL configuration options [DEFAULT]ssl_cert_path and [DEFAULT]ssl_key_path were removed, please use configuration options from [ssl] section.

  • The deprecated configuration option [processing]store_data_location was removed.

Security Issues

  • Fixes insufficient input filtering when looking up a node by information from the introspection data. It could potentially allow SQL injections via the /v1/continue API endpoint. See story 2005678 for details.

Bug Fixes

  • Fixes an issue when extra_hardware plugin failed to save extra hardware information to Swift, the collected information is not processed and consumed.

  • Fixes an issue while mapping port InfiniBand MAC address to EthernetOverInfiniBand MAC. Prior to this fix, it will fail to map and raise an exception.

8.2.0

Prelude

The Stein release of ironic-inspector features support of storing introspection data in the database instead of the Object Store service, as well as fixes for IPv6.

New Features

  • Adds the support to store introspection data in ironic-inspector database. Set the option [processing]store_data to database to use this feature.

  • Adds a migration tool ironic-inspector-migrate-data to facilitate the introspection data migration between supported introspection data storage backends. Currently the available introspection data storage backends are: database and swift. For example, to migrate existing introspection data stored in the swift to database, execute following command:

    $ ironic-inspector-migrate-data --from swift --to database --config-file /etc/ironic-inspector/inspector.conf

    Storage backends involved in the migration should have been properly configured in the ironic inspector configuration file. Before the introspection data migration can be started. The ironic inspector database should be upgraded to have the latest schema.

  • Adds support to use latest as the microversion value in the request to the ironic-inspector API.

Upgrade Notes

  • The set-attribute action now automatically sets reset_interfaces to True if the driver is updated. If it’s not desired, set it explicitly to False.

Deprecation Notes

  • Deprecates the configuration option [processing]store_data_location. The introspection data can be retrieved from the ironic-inspector API, there is no need to keep an extra link in Ironic.

Bug Fixes

  • Fixes inspection of nodes with IPv6 BMC address. Inspection could not be initiated because an IPv6 address was treated as a hostname, which could not be resolved.

  • Remove debug logging for PXE filter driver which tends to fill up inspector logs when debug is enabled.

  • Fixes updating a driver with the set-attribute introspection rule action by providing reset_interfaces.

8.1.0

New Features

  • Adds a configuration option [iptables]ip_version to specify the desired ip version for the iptables pxe filter, possible values are 4 and 6, the default value is 4. When set to 6, the iptables pxe filter will use ip6tables command to manage rules for the DHCPv6 port 547.

  • Adds new introspection rules actions to add or remove traits on nodes: add-trait and remove-trait.

Upgrade Notes

  • The deprecated configuration option [DEFAULT]node_status_keep_time was removed.

  • Adds rpc related configuration options for the communication between ironic-inspector API and worker. It needs to be configured properly during upgrade. Set [DEFAULT]transport_url to fake:// if a rpc backend is not available or not desired.

Deprecation Notes

  • Configuration options [DEFAULT]ssl_cert_path and [DEFAULT]ssl_key_path are deprecated for ironic-inspector now uses oslo.service as underlying HTTP service instead of Werkzeug. Please use [ssl]cert_file and [ssl]key_file.

Bug Fixes

  • A new rootwrap filter is now included to allow control of the systemd dnsmasq service used by ironic-inspector. This fixes a permission issue when systemctl commands are used as dnsmasq_start_command and dnsmasq_stop_command in the configuration for the dnsmasq pxe filter. See bug 2002818.

    Note

    The filter uses the systemd service name used by the RDO distribution (openstack-ironic-inspector-dnsmasq.service).

  • Fixes issue that can result in introspection failure when a network switch sends incomplete information for LLDP switch_id or port_id. The validation expects these fields when a port is updated, this fix now handles the validation exception.

  • Allows the set-attribute introspection rule action to accept None as value for a property.

  • Fixes the issue that ports were not collected when there were only IPv6 addresses (no IPv4), and the configuration option [processing]add_ports was not set to all. Inspector will report “No suitable interfaces found” if no interface is collected. For more information see Story 1744073

8.0.0

New Features

  • Adds new parameter manage_boot to the introspection API to allow disabling boot management (setting the boot device and rebooting) for a specific node. If it is set to False, the boot is supposed to be managed by a 3rd party.

    If the new option can_manage_boot is set to False (the default is True), then ``manage_boot must be explicitly set to False.

  • Modifies introspection rules to allow formatting to be applied to strings nested in dicts and lists in the actions.

Upgrade Notes

  • Updates the default Ironic API version to 1.38.

    This version is used by default within the Bare Metal Inspection service when communicating with the Bare Metal API. It is the default used by processing plugins, which may override the version, and by introspection rules, which may not override the version.

    1.38 was the API version at the time of the most recent Queens series Bare Metal service release (10.1.0).

    See story 2002166.

Bug Fixes

  • The dnsmasq PXE filter no longer whitelists the MAC addresses of ports deleted from the Bare Metal service. Instead they are blacklisted unless introspection is active or the node_not_found_hook is set in the configuration. This ensures that no previously enrolled node accidentally boot the inspection image when no node introspection is active. Bug #2001979.

  • Stops introspection when setting boot device is failed, as the node is not guaranteed to perform a PXE boot in this case.

Other Notes

  • The deprecated configuration option [iptables]manage_firewall was removed, use [pxe_filter]driver to set filtering driver.

7.3.0

New Features

  • Adds wildcard ignore entry to dnsmasq PXE filter. When node introspection is active, or if node_not_found_hook is set in the configuration the ignore is removed from the wildcard entry. This ensures that unknown nodes do not accidentally boot into the introspection image when no node introspection is active.

    This brings dnsmasq PXE filter driver feature parity with the iptables PXE filter driver, which uses a firewall rule to block any DHCP request on the interface where Ironic Inspector’s DHCP server is listening.

  • Issuing a SIGHUP to the Ironic-Inspector service will cause the service to reload and use any changed values for mutable configuration options.

    Mutable configuration options are indicated as such in the sample configuration file by Note: This option can be changed without restarting.

    A warning is logged for any changes to immutable configuration options.

Upgrade Notes

  • The [discovery]enroll_node_driver option, specifying the hardware type or driver to use for newly discovered nodes, was changed from fake classic driver to fake-hardware hardware type.

  • Adds dependency on the retrying Python library.

Bug Fixes

  • Fixes bug in which the switch_id field in a port’s local_link_connection can be set to a non-MAC address if the processed LLDP has a value other than a MAC address for ChassisID. The Bare Metal API requires the switch_id field to be a MAC address, and will return an error otherwise. See bug 1748022 for details.

  • Ironic introspection no longer tries to access the Identity service if the auth_strategy option is set to noauth and the auth_type option is not set to none.

  • The periodic PXE filter update task now retries fetching port list from the Bare Metal service 5 times (with 1 second delay) before giving up. This ensures that a temporary networking glitch will not result in the ironic-inspector service stopping.

7.1.0

Deprecation Notes

  • Several configuration options related to ironic API access are deprecated and will be removed in the Rocky release. These include:

    • [ironic]/os_region - use [ironic]/region_name option instead

    • [ironic]/auth_strategy - set [ironic]/auth_type option to none to access ironic API in noauth mode

    • [ironic]/ironic_url - use [ironic]/endpoint_override option to set specific ironic API endpoint address if discovery of Ironic API endpoint is not desired or impossible (for example in standalone mode)

    • [ironic]/os_service_type - use [ironic]/service_type option

    • [ironic]/os_endpoint_type - use [ironic]/valid_interfaces option to set Ironic endpoint types that will be attempted to be used

  • Several configuration options related to Swift API access are deprecated and will be removed in Rocky release. These include:

    • [swift]/os_service_type - use [swift]/service_type option

    • [swift]/os_endpoint_type - use [swift]/valid_interfaces option

    • [swift]/os_region - use [swift]region_name option

Other Notes

7.0.0

New Features

  • Introduces the dnsmasq PXE filter driver. This driver takes advantage of the inotify facility to reconfigure the dnsmasq service in real time to implement a caching black-/white-list of port MAC addresses.

Upgrade Notes

  • A new state aborting was introduced to distinguish between the node introspection abort precondition (being able to perform the state transition from the waiting state) from the activities necessary to abort an ongoing node introspection (power-off, set finished timestamp etc.)

  • Handling of local_gb property was moved from the scheduler hook to root_disk_selection.

Bug Fixes

  • The node_info.finished(<transition>, error=<error>) now updates node state together with other status attributes in a single DB transaction.

Other Notes

6.1.0

New Features

  • The PXE filter drivers mechanism is now enabled. The firewall-based filtering was re-implemented as the iptables PXE filter driver.

  • Adds an API access policy enforcment based on oslo.policy rules. Similar to other OpenStack services, operators now can configure fine-grained access policies using policy.yaml file. See policy.yaml.sample in the code tree for the list of available policies and their default rules. This file can also be generated from the code tree with the following command:

    tox -egenpolicy

    See the oslo.policy package documentation for more information on using and configuring API access policies.

Upgrade Notes

  • Due to the choice of default values for API access policies rules, some API parts of the ironic-inspector service will become available to wider range of users after upgrade:

    • general access to the whole API is by default granted to a user with either admin, administrator or baremetal_admin role (previously it allowed access only to a user with admin role)

    • listing of current introspection statuses and showing a given introspection is by default also allowed to a user with the baremetal_observer role

    If these access policies are not appropriate for your deployment, override them in a policy.json file in the ironic-inspector configuration directory (usually /etc/ironic-inspector).

    See the oslo.policy package documentation for more information on using and configuring API access policies.

Deprecation Notes

  • The firewall-specific configuration options were moved from the firewall to the iptables group. All options in the iptables group are now deprecated.

  • The generic firewall options firewall_update_period and manage_firewall were moved under the pxe_filter group as sync_period and driver=iptables/noop respectively.

Bug Fixes

  • The older ipmi_address field in the introspection data no longer has priority over the newer bmc_address inventory field during lookup. This fixes lookup based on MAC addresses, when the BMC address is reported as 0.0.0.0 for any reason (see bug 1714944).

  • Should the iptables PXE filter encounter an unexpected exception in the periodic sync call, the exception will be logged and the filter driver will be reset in order to make subsequent sync calls fail (and propagate the failure, exiting the ironic-inspector process eventually).

Other Notes

  • Allows a periodic task to shut down an ironic-inspector process upon a failure.

6.0.0

New Features

  • Querying ironic-inspector rules API now also returns the invert and multiple attributes of the associated conditions.

  • Add disabled option to add_ports, so discovered nodes can be created without creating ports.

  • Add a check from the link_local_connection plugin to use data stored by the lldp_basic; this avoids parsing the LLDP packets twice.

  • Adds node state to the GET /v1/introspection/<node UUID or name> and GET /v1/introspection API response data.

  • Processing hooks can now define dependencies on other processing hooks. ironic-inspector start up fails when required hooks are not enabled before the hook that requires them.

  • Update pxe_enabled field on ports. It is set to True for the PXE-booting port and False for the remaining ports. Both newly discovered and existing ports are affected.

Upgrade Notes

  • Experimental setting IPMI credentials support was removed from all versions of the API. The current ironic-inspector API version was bumped to 1.12 to mark this change.

  • The default API version was synchronised with the current API version again after removal of the IPMI credentials setting.

  • Ports creating logic was moved from core processing code to the validate_interfaces processing hook. This may affect deployments that disable this hook or replace it with something else. Also make sure to place this hook before any hooks expecting ports to be created.

  • Bare metal API version 1.19 is now required.

  • Removes deprecated configuration options: introspection_delay_drivers from the default section and log_bmc_address from the processing section.

  • Support for rollback actions in introspection rules was removed.

  • Old status records are no longer removed by default. They are still removed if a node is removed from Ironic.

Deprecation Notes

  • The node_status_keep_time configuration option is deprecated. Now that we can remove status information about nodes removed from ironic, this option does not make much sense, and may be confusing

Bug Fixes

  • Timeout in an active state led to an undefined transition error. This is fixed and an introspection finishes now with Timeout error.

  • 0.0.0.0 and an empty string in the bmc_address inventory field are now correctly treated as missing BMC address.

  • For postgreSQL, the database migration command ironic-inspector-dbsync upgrade always failed (with enum NODE_STATE does not exist). This is fixed and the migration now works.

  • Do not fail the whole introspection due to a value formatting error during introspection rules rollback. See bug 1686942 for an example and detailed investigation.

5.1.0

Bug Fixes

  • The POST /v1/introspection/<Node ID>/data/unprocessed API updates the started_at time when ironic inspector begins processing the node.

  • Exception CalledProcessError is raised when running iptables cmd on start up. The issue is caused by eventlet bug, see: https://github.com/eventlet/eventlet/issues/357 The issue affects ironic-inspector only if it manages firewall - configured with manage_firewall = True configuration option.

  • Wrong provision state name ‘inspectfail’ in ironic-inspector valid states for node inspection. This issue leads to state inconsistency between ironic and ironic-inspector. For example, if ironic inspection timeout is lower than ironic-inspector’s, and inspection timeout occurs, ironic will transition node into ‘inspect failed’ provision state. In such case when node inspection finishes without errors the node will be in ‘inspect failed’ provision state with inspection in ‘finished’ state.

5.0.0

New Features

  • Extend the introspection status returned from GET@/v1/introspection/<Node Id> to contain the uuid, started_at and finished_at fields.

  • Add a plugin to parse raw LLDP Basic Management, 802.1, and 802.3 TLVs and store the data in Swift.

  • Add an API endpoint for listing introspection statuses. Operators can use this to get the status for all running or previously run introspection processing.

  • Introduce a new configuration option api_max_limit that defines the maximum number of items per page when API results are paginated.

  • InfiniBand interface discovery is now supported through introspection. The ironic-inspector will add the client-id to the corresponding ironic port that represents the InfiniBand interface. The ironic-inspector should be configured with a list of interfaces firewall.ethoib_interfaces to indicate which Ethernet Over InfiniBand Interfaces are used for DHCP.

  • Node introspection state is now kept in a dedicated database column. The introspection is now using a finite state machine. The state isn’t exposed to the user yet.

  • Adds support for using operators with the root device hints mechanism. The supported operators are =, ==, !=, >=, <=, >, <, s==, s!=, s>=, s>, s<=, s<, <in>, <all-in> and <or>.

  • Looking up nodes during introspection or discovery now supports multiple attributes matching. For example, two nodes can use the same bmc_address and still can be distinguished by MAC addresses.

  • Avoid failing introspection on diskless nodes. The node property local_gb == 0 is set in that case.

Known Issues

  • Due to the nature of the NodeInfo.state attribute (being updated independently from the rest of the node_info attributes) if a (DB) connection was lost before the Node.state column was updated, Node.finished_at and Node.error columns may not be in sync with the Node.state column.

Upgrade Notes

  • Add a new dependency, pytz.

  • A database migration is required to change some columns from Float to DateTime type. This may take some time based on the number of introspection statuses in the DB.

  • Removed previously deprecated authentication options from “ironic”, “swift”, and “keystone_authtoken” sections.

  • Removed long deprecated support for “discoverd” section in configuration file.

  • The default value for the configuration option “introspection_delay_drivers” was changed to .*, which means that by default “introspection_delay” is now applied to all drivers. Set “introspection_delay” to 0 to disable the delay.

  • Node.state and Node.version_id database columns are introduced.

  • The introspection state column defaults to the state finished unless the introspection error column value on a node row isn’t null, then node state is set to error.

  • Uniqueness of a node bmc_address isn’t enforced any more.

  • The primary key of the attributes table is relaxed from the attributes.name, attributes.value column pair to a new column attributes.uuid.

Deprecation Notes

  • The configuration option “log_bmc_address” is deprecated.

  • The configuration option “introspection_delay_drivers” is deprecated.

Bug Fixes

  • Change database columns started_at and finished_at to type DateTime from type Float so that timestamps fit into these columns correctly.

  • Fix bug where periodic clean up failed with DBDeadlock if introspection timed out.

  • Ensure the configuration options firewall.firewall_update_period and clean_up_period are applied to the periodic_clean_up and periodic_update tasks after the config file is read.

  • LLC hook now formats the chassis ID and port ID MAC addresses into Unix format as expected by ironic.

  • LLC hook ensures that correct port information is passed to the patch_port function

  • LLC hook no longer assumes all inspected ports are added to ironic

  • Loopback BMC addresses (useful e.g. with virtualbmc) are no longer used for lookup.

  • Introspection fails on nodes with the same IPMI address but different IPMI ports.

Other Notes

  • Default API version is temporary pinned to 1.8 (before deprecating setting IPMI credentials). It will be reset to the latest version again when support for setting IPMI credentials is removed.

4.2.0

New Features

  • Adds new processing hook pci_devices for setting node capabilities based on PCI devices present on a node and rules in the [pci_devices] aliases configuration option. Requires “pci-devices” collector to be enabled in IPA.

Bug Fixes

  • Use only single quotes for strings inside SQL statements. Fixes a crash when PostgreSQL is used as a database backend.

  • Set the node to the error state when it failed get data from Swift.

4.1.0

New Features

  • Added GenericLocalLinkConnectionHook processing plugin to process LLDP data returned during inspection and set port ID and switch ID in an Ironic node’s port local link connection information using that data.

  • Add configuration option processing.power_off defaulting to True, which allows to leave nodes powered on after introspection.

Bug Fixes

  • Fix setting non string ‘value’ field for rule’s actions. As non string value is obviously not a formatted value, add the check to avoid AttributeError exception.

4.0.0

Prelude

Starting with this release only ironic-python-agent (IPA) is supported as an introspection ramdisk.

New Features

  • Added a new “capabilities” processing hook detecting the CPU and boot mode capabilities (the latter disabled by default).

  • File name for stored ramdisk logs can now be customised via “ramdisk_logs_filename_format” option.

Upgrade Notes

  • The default file name for stored ramdisk logs was change to contain only node UUID (if known) and the current date time. A proper “.tar.gz” extension is now appended.

  • API “POST /v1/rules” returns 201 response code instead of 200 on creating success. API version was bumped to 1.6. API less than 1.6 continues to return 200.

  • Default API version was changed from minimum to maximum which Inspector can support.

  • Support for the old bash-based ramdisk was removed. Please switch to IPA before upgrading.

  • Removed the deprecated “root_device_hint” alias for the “raid_device” hook.

Bug Fixes

  • Fixed “/v1/continue” to return HTTP 500 on unexpected exceptions, not HTTP 400.

  • Fix response return code for rule creating endpoint, it returns 201 now instead of 200 on success.

  • The “size” root device hint is now always converted to an integer for consistency with IPA.

3.3.0

New Features

  • Ironic-Inspector is now using keystoneauth and proper auth_plugins instead of keystoneclient for communicating with Ironic and Swift. It allows to finely tune authentication for each service independently. For each service, the Keystone session is created and reused, minimising the number of authentication requests to Keystone.

  • Add support for using Ironic node names in API instead of UUIDs. Note that using node names in the introspection status API will require a call to Ironic to be made by the service.

  • Introduced API “POST /v1/introspection/UUID/data/unprocessed” for reapplying the introspection over stored data.

Upgrade Notes

  • Operators are advised to specify a proper keystoneauth plugin and its appropriate settings in [ironic] and [swift] config sections. Backward compatibility with previous authentication options is included. Using authentication information for Ironic and Swift from [keystone_authtoken] config section is no longer supported.

  • Handling ramdisk logs was moved out of the “ramdisk_error” plugin, so disabling it will no longer disable handling ramdisk logs. As before, you can set “ramdisk_logs_dir” option to an empty value (the default) to disable storing ramdisk logs.

Deprecation Notes

  • Most of current authentication options for either Ironic or Swift are deprecated and will be removed in a future release. Please configure the keystoneauth auth plugin authentication instead.

Bug Fixes

  • Fixes a problem which caused an unhandled TypeError exception to bubble up when inspector was attempting to convert some eDeploy data to integer.

  • Fixed a regression in the firewall code, which causes re-running introspection for an already inspected node to fail.

  • Fixed the “is-empty” condition to return True on missing values.

  • The lookup procedure now uses all valid MACs, not only the MAC(s) that will be used for creating port(s).

  • The “enroll” node_not_found_hook now uses all valid MACs to check node existence, not only the MAC(s) that will be used for creating port(s).

  • The ramdisk logs are now stored on all preprocessing errors, not only ones reported by the ramdisk itself. This required moving the ramdisk logs handling from the “ramdisk_error” plugin to the generic processing code.

3.2.0

New Features

Bug Fixes

  • Don’t fail on finish power off if node in ‘enrol’ state. Nodes in ‘enrol’ state are not expected to have power credentials.

3.1.0

New Features

  • Introduced API “POST /v1/introspection/<UUID>/abort” for aborting the introspection process.

  • New condition plugins “contains” and “matches” allow to match value against regular expressions.

  • Added new condition plugin “is-empty”, which allows to match empty string, list, dictionary or None.

  • Add a new node_not_found hook - enroll, which allows automatic discovery of Ironic’s node.

  • Conditions now support comparing fields from node info;

  • Introspection rules conditions got a new generic “invert” parameter that inverts the result of the condition.

Upgrade Notes

  • Switch required Ironic API version to ‘1.11’, which supports ‘enroll’ state.

  • Minimum possible value for the “max_concurrency” setting is now 2.

  • Removed deprecated support for passing “node_patches” and “ports_patches” arguments to processing hooks.

  • Ramdisk logs are no longer part of data stored to Swift and returned by the API.

  • Introspection rules actions ‘set-attribute’, ‘set-capability’ and ‘extend-attribute’ no longer have the opposite effect on nodes that do not match a rule.

Deprecation Notes

  • The rollback actions for introspection rules are deprecated. No in-tree actions are using them, 3rdpart should stop using them as soon as possible.

  • Using the root_device_hint alias for the raid_device plugin is deprecated.

Bug Fixes

  • Fixed extra_hardware plugin connection to Swift.

  • Only issue iptables calls when list of active MAC addresses changes.

  • Dropped rollback actions from ‘set-attribute’, ‘set-capability’ and ‘extend-attribute’ introspection rules actions, as they were confusing, completely undocumented and broke some real world use cases (e.g. setting driver field).

  • Introspection rules (e.g. set-attribute action) now accept ‘path’ field without leading forward slash as Ironic CLI does.

Other Notes

  • Switched to Futurist library for asynchronous tasks.

  • Log level for error when node was not found in Inspector cache was changed from error to info level. It was done because not_found_hook may handle this case, so this wouldn’t be error any more.

3.0.0

Prelude

Starting with this release, ironic-python-agent becomes the default introspection ramdisk, with the old bash-based ramdisk being deprecated.

New Features

  • Inspector no longer requires old-style “local_gb”, “memory_mb”, “cpus” and “cpu_arch” fields from the introspection ramdisk. They are still supported, though, for compatibility with the old ramdisk.

Upgrade Notes

  • Removed support for introspecting nodes in maintenance mode, deprecated in the liberty cycle. Use “inspecting”, “manageable” or “enroll” states instead.

  • The root_disk_selection processing hook will now error out if root device hints are specified on ironic node, but ironic-python-agent is not used as an introspection ramdisk.

Deprecation Notes

  • Using old bash-based ramdisk is deprecated, please switch to ironic-python-agent as soon as possible.

Bug Fixes

  • The data processing API endpoint now validates that data received from the ramdisk is actually a JSON object instead of failing the internal error later (issue https://bugs.launchpad.net/bugs/1525876).

Other Notes

  • Make debug-level logging more compact by removing newlines from firewall logging and disabling some 3rdparty debug messages by default.

  • Improve logging for ramdisk logs collection.

  • Logging during processing is now more consistent in terms of how it identifies the node. Now we try to prefix the log message with node UUID, BMC address and PXE MAC address (if available). Logging BMC addresses can be disabled via new “log_bmc_address” option in the “processing” section.

2.3.0

Prelude

This release includes automatic docs generation via Sphinx.

Critical Issues

Security Issues

Bug Fixes

  • Log a warning when add_ports is set to pxe, but no PXE MAC is returned from the ramdisk.

  • Acquire a lock on a node UUID when handling it.

Other Notes

  • IPA (ironic-python-agent) is now fully supported in the Devstack plugin and will become the default ramdisk in the next release.

  • Allow auto-generation of database migrations.

  • Introduced new docs generation via Sphinx and ReST.

    • Separate doc folder includes source and build

    • Integration with tox as docs target

    • makefile for manual building

    • OpenStack Theme support