2023.1 Series (11.2.0 - 11.4.x) Release Notes¶
2023.1-eom¶
Bug Fixes¶
Fixes an issue where inspection would fail if an IPv6 address wrapped in brackets is used for the redfish BMC address. See bug: 2036455.
In case the lldp raw data collected by the inspection process includes non utf-8 information, the parser fails breaking the inspection process. This patch works around that excluding the malformed data and adding an entry in the logs to provide information on the failed tlv.
Fixes the Role Based Access Control state and capabilities to align with OpenStack Community RBAC goals which includes support for a
service
role by default to enable inter-service communication to be configured without anadmin
username. In large part, these changes were missed as the Inspector service is considered an “admin-only” service.Also in alignment with overall community position changes, where the
admin
role is sufficent without an explicitsystem
scope. To help ensure a high level of security, explicit testing was also added for themanager
role, which is unavailable as that role is reserved for administrative functions inside of a tenant’s project.
11.3.0¶
Upgrade Notes¶
The minimum version of SQLAlchemy is now
1.4.0
, in preparation for the future anticipated release of SQLAlchemy2.0.0
.
The minimum version of Oslo.DB is now
12.1.0
, in preparation for the future anticipated release of SQLAlchemy2.0.0
.
Database schema upgrades from versions prior to
7.3.0
are not supported. Please upgrade to an intermediate release prior to upgrading to this release.
Deprecation Notes¶
Plugin maintainers should be aware that the Node Cache object field
version_id
filed is no longer in use. It is still returned by the data model if stored for the purposes of compatability, but Inspector will not update the field through the normal course of it’s operation.
Bug Fixes¶
Fixes an issue where database responses of nodes would get orphaned in inspector process RAM, and would not be garbage collected. We were able to discover and reproduce this issue while working on database connectivity locks remaining in place. Please see story 2009727 for more details.
Other Notes¶
Plugin maintainers who are directly working with the database will need to update their plugins. Specifically the Database API has been delineated into using
enginefacade
with a dedicated reader and writer model, in anticipation of support for SQLAlchemy 2.0 and an eventual merge of Inspector into Ironic at some point in the future. Database actions are now performed through theironic_inspector.db.api
module, where previously they were spread acrossironic_inspector.db
andironic_inspector.node_cache
.
11.1.0¶
New Features¶
Follow the same process for determining root device as Ironic Python Agent which has been changed to accommodate for the feature enabling users to specify a list of devices that should be skipped during cleaning/deployment The field
skip_block_devices
is one of the properties of a node
10.11.0¶
New Features¶
Supports listening on a Unix socket instead of a normal TCP socket. This is useful with an HTTP server such as nginx in proxy mode.
10.10.0¶
Known Issues¶
The response headers for empty body HTTP 204 replies, at present, violate RFC7230. This was not intentional, but underlying libraries also make inappropriate changes to the headers, which can cause clients to experience odd failures. This is anticipated to be corrected once an underlying issue in eventlet is resolved.
Upgrade Notes¶
The rootwrap rule to allow restarting the systemd service openstack-ironic-inspector-dnsmasq.service has been removed. No known tooling requires this rule since before Train. Any configuration tool which is setting [dnsmasq_pxe_filter]dnsmasq_start_command also needs to be writing an appropriate rootwrap.d file, as the inspector devstack plugin does.
Bug Fixes¶
Fixes HTTP responses so the Eventlet library, which is used to support the operation of the WSGI application, does not incorrectly inject a
Transfer-Encoding
header into the HTTP response, even on HTTP 204 replies, which is a violation of RFC7230. This header ultimately can cause varying client reactions which are not expected and can raise exceptions. For now, this has been remedied via an explicit return of aContent-Length
header, which is also an RFC7230 violation, but it appears to be the lesser of known evils at this time.
10.9.0¶
New Features¶
Adds support for filter by state in the list introspection API. See story 1625183.
GET /v1/introspection?state=starting,...
10.8.0¶
New Features¶
The new
[healthcheck] enabled
option has been added. When this option is set toTrue
, the healthcheck middleware is enabled in API pipeline and the additional API endpoint to monitor service availability becomes available at/healthcheck
path.
Bug Fixes¶
Inspector now ignores failures to list ironic ports during pxe filter driver sync, and just skips the sync in this case. Previously such errors resulted in pxe filter driver being stuck in an uninitialized state until ironic inspector was restarted. See bug 2008971.
Fixes issues in Inspector where various tasks would not have retry logic applied to them and may sporadically fail. This is because the OpenStack SDK does not comprehend the NodeLocked error, which previously python-ironicclient silently handled. Basic operations such as “power reboot” and “set boot device” will now be retried automatically if they fail. For more information, please see story 2009107.
10.7.0¶
Bug Fixes¶
Fixes an issue where a failed inspection due to a transient failure can prevent retry attempts to inspect to be perceived as a failure. If a prior inspection fails and is in
error
state, when a new introspection is requested, the state is now appropriately set tostarting
.
10.6.0¶
New Features¶
The default policy will been replaced with one which aligns with the Secure-RBAC scopes and roles. Since ironic-inspector is a tool used only by system-level admins, only the
system
scope is supported, and the only roles in the policy rules areadmin
andreader
.
Upgrade Notes¶
[DEFAULT]/ipmi_address_fields
now hasibmc_address
in the default configuration, allowing introspection to try and match the BMC address if no ports are defined when using the ibmc driver.
The default value of
[oslo_policy] policy_file
config option has been changed frompolicy.json
topolicy.yaml
. Operators who are utilizing customized policy files or previously generated static policy files (which are not needed by default), should generate new policy files and modify them to meet their needs in the event of any new policies or rules have been added. Please consult the oslopolicy-convert-json-to-yaml tool to convert a JSON to YAML formatted policy file in backward compatible way.
The new policy is only enforced when
[oslo_policy]
config is changed toenforce_new_defaults=True
andenforce_scope=True
, otherwise the existing deprecated policy is used. User accounts which rely on having thebaremetal_admin
orbaremetal_observer
roles will need to have system-scopedadmin
orreader
roles to use the API when the new policy is enforced.
Deprecation Notes¶
Use of legacy policy files was deprecated by the
oslo.policy
library during the Victoria development cycle. As a result, this deprecation is being noted in the Wallaby with an anticipated future removal of support byoslo.policy
. As such operators will need to convert to YAML policy files. Please see the upgrade notes for details on migration of any custom policy files.
The previous policy is still enforced by default, but is now deprecated and will be removed in a future release.
10.5.0¶
New Features¶
Adds a possibility to setup ironic inspector behind a proxy, while allowing the links of the resources API returns to remain correct. Inspector now respects the following headers that are passed with API requests:
X-Forwarded-For
,X-Forwarded-Proto
,X-Forwarded-Host
,X-Forwarded-Port
,X-Forwarded-Prefix
. If the API is run providingSCRIPT_NAME
environment variable, it is now also respected, and it allows to return the correct links in response to requests, even if inspector API is not placed at the web server root resource.
Bug Fixes¶
Fixes database migrations with SQLAlchemy 1.3.20.
10.4.0¶
New Features¶
Adds an
accelerators
plugin to identify acclerator devices and update the bare metal node for future scheduling. The accelerator devices will be saved to node properties under the keyaccelerators
. Introduces a configuration option[accelerators]known_devices
to specify a configuration file which contains required information to identify accelerator devices, by default it uses the in-tree configuration file namedknown_accelerators.yaml
.
The dnsmasq pxe-filter now supports mapping between host InfiniBand MAC to EthernetOverInfiniBand MAC. (This was previously only supported by the iptables pxe-filter.)
By default the DHCP filtering will open the DHCP server for any node when introspection is active. It will only block DHCP for enrolled nodes that are not being introspected. Doing so is required to support interface discovery (which by default will enroll the pxe port to ironic if not present). This behaviour is not always wanted, as nodes not managed by ironic may boot the inspection image.
A new option was added
[pxe_filter]deny_unknown_macs
which allow changeing this behaviour so that the DHCP server only allow enrolled nodes being introspected and deny everything else.Note
If this option is
True
, nodes must have at least one enrolled port prior to introspection.
Bug Fixes¶
Fixes the node identification logic to enable a user to list the
redfish_address
label fordriver_info
field values for identification of a machine using the[DEFAULT]ipmi_address_fields
configuration option. Previously the host would just not be matched as the full URL would be evaluated instead of what the URL may resolve to.
10.3.0¶
New Features¶
The new API
GET /v1/introspection/<node>/data/unprocessed
allows retrieving raw (unprocessed) data if data store is enabled.
Upgrade Notes¶
API now listens on
::
by default, change thelisten_address
configuration option to modify.
Bug Fixes¶
The
extra_hardware
processing hook no longer refuses to parse extra data if some records are empty or have unexpected length. These records are now discared.The previous behavior can be returned by setting the new option
[extra_hardware]strict
toTrue
.
The
extra_hardware
processing hook no longer removes the incomingdata
object if it has unexpected data format, assuming that this object is used for something else.The previous behavior can be returned by setting the new option
[extra_hardware]strict
toTrue
.
Using auth_strategy=http_basic incorrectly required authentication for public paths such as / and /v1. These paths are now public.
Fixes an issue which may occur with Apache httpd webservers acting as a proxy where the server may report
Bad Gateway
, however inspector continues operating as if there was no problem. This was due to a lack of aContent-Type
header on HTTP 202 and 204 replies, and lack of message body with HTTP 202 messages which Apache httpd can error upon.
No longer tries to set
local_gb
to -1 if the matched root device has size of zero.
10.2.0¶
New Features¶
Adds the ability for periodic clean-up and synchronization tasks with
ironic
to be able to be disabled by setting the[DEFAULT]clean_up_period
to a value of0
. This is intended for “stand-alone” operators only as it may result in unexpected behaviors if used in a non-standalone environment.
Adds a new configuration option
[discovery]enroll_node_fields
that specifies additional fields to set on a node (e.g. driver interfaces).
Enable Basic HTTP authentication middleware.
When the config option
[DEFAULT]auth_strategy
is set tohttp_basic
then non-public API calls require a valid HTTP Basic authentication header to be set. The config option[DEFAULT]http_basic_auth_user_file
defaults to/etc/ironic-inspector/htpasswd
and points to a file that supports the Apache htpasswd syntax[1]. This file is read for every request, so no service restart is required when changes are made.The only password digest supported is bcrypt, and the
bcrypt
python library is used for password checks since it supports$2y$
prefixed bcrypt passwords as generated by the Apache htpasswd utility.To try basic authentication, the following can be done:
Set
/etc/ironic-inspector/inspector.conf
[DEFAULT]auth_strategy
tohttp_basic
Populate the htpasswd file with entries, for example:
htpasswd -nbB myName myPassword >> /etc/ironic-inspector/htpasswd
Make basic authenticated HTTP requests, for example:
curl --user myName:myPassword http://localhost:6385/v1/introspection
[1] https://httpd.apache.org/docs/current/misc/password_encryptions.html
Adds periodic leader election for the cleanup sync with Ironic. The election interval is configured by the new
leader_election_interval
config option.
Adds a configuration option
[processing]update_pxe_enabled
to control whether the pxe_enabled should be updated according to introspection data for ports. The default value is True which is backwards compatible.
Upgrade Notes¶
Remove upper constraint for python construct library and use the latest version available. The minimum compatible version for python construct is now 2.9.39
The raw data from the
extra_hardware
processing hook is no longer stored in Swift in an object namedextra_hardware-<node UUID>
. The same information is already available as part of the unprocessed introspection data without a hard dependency on Swift.
Deprecation Notes¶
The deprecated
[swift]max_retries
parameter has been removed.
Bug Fixes¶
Fixes an issue where IPv6 link local addresses are ignored during interface validation, making introspection fail.
Fixes
AttributeError: 'Node' object has no attribute 'uuid'
when trying to introspect an active node that is not currently in the cache.
No longer aborts the whole process if one periodic task fails.
Fixes accessing API endpoints with trailing slashes. Now they’re treated the same way as without slashes, although the latter remain canonical URLs.
No longer uses introspection delay for nodes with
manage_boot==False
(i.e. boot is managed by ironic). It is useless and may actually break introspection if a node boots before it gets whitelisted in the PXE filter.
The introspection start API is now synchronous when
manage_boot==False
. This means that any failures will be propagated to ironic, preventing it from powering a node on and booting it without the PXE filter updated.
10.1.0¶
New Features¶
Added the capability to define a scope for the inspection process. Previously, all introspection rules were applied when inspecting any node. There was no mechanism to apply only a selected set of rules. This change introduces a
scope
field to introspection rules. If a scope is set on an introspection rule, it will only apply to nodes that have a matchinginspection_scope
property. If not set, it will apply to all nodes.
Added
physnet_cidr_map
processing plugin, the plugin uses the IP address of interfaces returned during inspection and set the portphysical_network
via lookup from a CIDR to physical network mapping in config option[port_physnet]/cidr_map
.
Upgrade Notes¶
The python-ironicclient package has been removed as a dependency in favor of openstacksdk. Third party modules and plugins will require an update if they previously invoked ironicclient.
Other Notes¶
The devstack plugin for
ironic-inspector
has been changed to utilize pre-builtironic-python-agent
images based on Centos8 instead of legacy CoreOS based images.
Added base class (
BasePhysnetHook
) for plugins that assign a physical network to ports.
10.0.0¶
Upgrade Notes¶
Python 2.7 support has been dropped. Last release of ironic-inspector to support Python 2.7 is OpenStack Train. The minimum version of Python now supported by ironic-inspector is Python 3.6.
Bug Fixes¶
Fixes an issue during manual inspection of active nodes where the node uuid was not passed back to the inspector when it tried to identify a matching port.
No longer tries to power off nodes after introspection if
manage_boot
isFalse
.
Introspection now respects the
force_persistent_boot_device
parameter in a node’sdriver_info
.
Fixes an issue happening during manual inspection of active nodes where the code attempts to delete or update ports, while the only modification allowed for active nodes is updating the MAC address if the node is in maintenance.
9.2.0¶
Prelude¶
The Train release of Ironic Inspector features support for running separate API and conductor services.
New Features¶
Allows splitting the ironic-inspector service into ironic-inspector-api and ironic-inspector-conductor which coordinate via tooz and its underlying backend. A new configuration option
[DEFAULT]standalone
is introduced to enable this feature. The configuration defaults to True, and ironic-inspector runs as a single service, which is compatible with the old behavior. When set to False,ironic-inspector-api-wsgi
is used to start the API service, andironic-inspector-conductor
is used to start the conductor service. For ironic-inspector running in non-standalone mode, the user needs to set the new configuration option[coordination]backend_url
, which specifies the backend used for coordination.
Upgrade Notes¶
Updates the default Ironic API version to 1.56, which is the most recent version in the Stein series Bare Metal release (12.1.0).
Bug Fixes¶
Fixes introspection of active nodes that are not in the lookup cache, see story 2006233.
9.1.0¶
New Features¶
Adds the capability for introspection data to be posted to the API when a baremetal node is in
active
orrescue
states. This feature may be useful for data center operators who wish to update introspection data periodically.To enable this feature, set
[processing]permit_active_introspection
toTrue
. When this is set, the value of[processing]power_off
is overridden for nodes inactive
orrescue
states.
Adds support to enroll node with IPv6 BMC address. Introduces a configuration option
[discovery]enabled_bmc_address_version
to specify the order of preferred IP version of the BMC address.
Upgrade Notes¶
The deprecated options from the
ironic
sectionos_region
,auth_strategy
,ironic_url
,os_service_type
andos_endpoint_type
have been removed. Please use keystoneauth options instead.
The deprecation configuration options
os_service_type
,os_region
andos_endpoint_type
from the[swift]
section have been removed.
Deprecation Notes¶
The configuration option
[swift]max_retries
is deprecated. It has been doing nothing for a few releases already.
Bug Fixes¶
No longer fails introspection if memory or CPU information is not provided in the inventory. These are no longer required for scheduling, introspection should not require them either.
9.0.0¶
New Features¶
A new option
enable_mdns
allows to enable publishing the baremetal introspection API endpoint via mDNS as specified in the API SIG guideline.
Adds support to reapply with provided unprocessed introspection data. The introspection data is supplied in the body of POST request to
/v1/introspection/<node_id>/data/unprocessed
. The introspection data will also be saved to storage backend.
Upgrade Notes¶
The deprecated SSL configuration options
[DEFAULT]ssl_cert_path
and[DEFAULT]ssl_key_path
were removed, please use configuration options from[ssl]
section.
The deprecated configuration option
[processing]store_data_location
was removed.
Security Issues¶
Fixes insufficient input filtering when looking up a node by information from the introspection data. It could potentially allow SQL injections via the
/v1/continue
API endpoint. See story 2005678 for details.
Bug Fixes¶
Fixes an issue when extra_hardware plugin failed to save extra hardware information to Swift, the collected information is not processed and consumed.
Fixes an issue while mapping port InfiniBand MAC address to EthernetOverInfiniBand MAC. Prior to this fix, it will fail to map and raise an exception.
8.2.0¶
Prelude¶
The Stein release of ironic-inspector features support of storing introspection data in the database instead of the Object Store service, as well as fixes for IPv6.
New Features¶
Adds the support to store introspection data in ironic-inspector database. Set the option
[processing]store_data
todatabase
to use this feature.
Adds a migration tool
ironic-inspector-migrate-data
to facilitate the introspection data migration between supported introspection data storage backends. Currently the available introspection data storage backends are:database
andswift
. For example, to migrate existing introspection data stored in the swift to database, execute following command:$ ironic-inspector-migrate-data --from swift --to database --config-file /etc/ironic-inspector/inspector.conf
Storage backends involved in the migration should have been properly configured in the ironic inspector configuration file. Before the introspection data migration can be started. The ironic inspector database should be upgraded to have the latest schema.
Adds support to use
latest
as the microversion value in the request to the ironic-inspector API.
Upgrade Notes¶
The
set-attribute
action now automatically setsreset_interfaces
toTrue
if the driver is updated. If it’s not desired, set it explicitly toFalse
.
Deprecation Notes¶
Deprecates the configuration option
[processing]store_data_location
. The introspection data can be retrieved from the ironic-inspector API, there is no need to keep an extra link in ironic.
Bug Fixes¶
Fixes inspection of nodes with IPv6 BMC address. Inspection could not be initiated because an IPv6 address was treated as a hostname, which could not be resolved.
Remove debug logging for PXE filter driver which tends to fill up inspector logs when debug is enabled.
Fixes updating a driver with the
set-attribute
introspection rule action by providingreset_interfaces
.
8.1.0¶
New Features¶
Adds a configuration option
[iptables]ip_version
to specify the desired ip version for the iptables pxe filter, possible values are4
and6
, the default value is4
. When set to6
, the iptables pxe filter will useip6tables
command to manage rules for the DHCPv6 port547
.
Adds new introspection rules actions to add or remove traits on nodes:
add-trait
andremove-trait
.
Upgrade Notes¶
The deprecated configuration option
[DEFAULT]node_status_keep_time
was removed.
Adds rpc related configuration options for the communication between ironic-inspector API and worker. It needs to be configured properly during upgrade. Set
[DEFAULT]transport_url
tofake://
if a rpc backend is not available or not desired.
Deprecation Notes¶
Configuration options
[DEFAULT]ssl_cert_path
and[DEFAULT]ssl_key_path
are deprecated for ironic-inspector now uses oslo.service as underlying HTTP service instead of Werkzeug. Please use[ssl]cert_file
and[ssl]key_file
.
Bug Fixes¶
A new rootwrap filter is now included to allow control of the systemd dnsmasq service used by ironic-inspector. This fixes a permission issue when systemctl commands are used as
dnsmasq_start_command
anddnsmasq_stop_command
in the configuration for the dnsmasq pxe filter. See bug 2002818.Note
The filter uses the systemd service name used by the RDO distribution (
openstack-ironic-inspector-dnsmasq.service
).
Fixes issue that can result in introspection failure when a network switch sends incomplete information for LLDP switch_id or port_id. The validation expects these fields when a port is updated, this fix now handles the validation exception.
Allows the
set-attribute
introspection rule action to acceptNone
as value for a property.
Fixes the issue that ports were not collected when there were only IPv6 addresses (no IPv4), and the configuration option
[processing]add_ports
was not set toall
. Inspector will report “No suitable interfaces found” if no interface is collected. For more information see Story 1744073
8.0.0¶
New Features¶
Adds new parameter
manage_boot
to the introspection API to allow disabling boot management (setting the boot device and rebooting) for a specific node. If it is set toFalse
, the boot is supposed to be managed by a 3rd party.If the new option
can_manage_boot
is set toFalse
(the default isTrue), then ``manage_boot
must be explicitly set toFalse
.
Modifies introspection rules to allow formatting to be applied to strings nested in dicts and lists in the actions.
Upgrade Notes¶
Updates the default Ironic API version to 1.38.
This version is used by default within the Bare Metal Inspection service when communicating with the Bare Metal API. It is the default used by processing plugins, which may override the version, and by introspection rules, which may not override the version.
1.38 was the API version at the time of the most recent Queens series Bare Metal service release (10.1.0).
See story 2002166.
Bug Fixes¶
The
dnsmasq
PXE filter no longer whitelists the MAC addresses of ports deleted from the Bare Metal service. Instead they are blacklisted unless introspection is active or thenode_not_found_hook
is set in the configuration. This ensures that no previously enrolled node accidentally boot the inspection image when no node introspection is active. Bug #2001979.
Stops introspection when setting boot device is failed, as the node is not guaranteed to perform a PXE boot in this case.
Other Notes¶
The deprecated configuration option
[iptables]manage_firewall
was removed, use[pxe_filter]driver
to set filtering driver.
7.3.0¶
New Features¶
Adds wildcard ignore entry to
dnsmasq
PXE filter. When node introspection is active, or ifnode_not_found_hook
is set in the configuration the ignore is removed from the wildcard entry. This ensures that unknown nodes do not accidentally boot into the introspection image when no node introspection is active.This brings
dnsmasq
PXE filter driver feature parity with theiptables
PXE filter driver, which uses a firewall rule to block any DHCP request on the interface where Ironic Inspector’s DHCP server is listening.
Issuing a SIGHUP to the ironic-inspector service will cause the service to reload and use any changed values for mutable configuration options.
Mutable configuration options are indicated as such in the sample configuration file by
Note: This option can be changed without restarting
.A warning is logged for any changes to immutable configuration options.
Upgrade Notes¶
The
[discovery]enroll_node_driver
option, specifying the hardware type or driver to use for newly discovered nodes, was changed fromfake
classic driver tofake-hardware
hardware type.
Adds dependency on the retrying python library.
Bug Fixes¶
Fixes bug in which the
switch_id
field in a port’slocal_link_connection
can be set to a non-MAC address if the processed LLDP has a value other than a MAC address forChassisID
. The bare metal API requires theswitch_id
field to be a MAC address, and will return an error otherwise. See bug 1748022 for details.
Ironic introspection no longer tries to access the Identity service if the
auth_strategy
option is set tonoauth
and theauth_type
option is not set tonone
.
The periodic PXE filter update task now retries fetching port list from the Bare Metal service 5 times (with 1 second delay) before giving up. This ensures that a temporary networking glitch will not result in the ironic-inspector service stopping.
7.1.0¶
Deprecation Notes¶
Several configuration options related to ironic API access are deprecated and will be removed in the Rocky release. These include:
[ironic]/os_region
- use[ironic]/region_name
option instead[ironic]/auth_strategy
- set[ironic]/auth_type
option tonone
to access ironic API in noauth mode[ironic]/ironic_url
- use[ironic]/endpoint_override
option to set specific ironic API endpoint address if discovery of ironic API endpoint is not desired or impossible (for example in standalone mode)[ironic]/os_service_type
- use[ironic]/service_type
option[ironic]/os_endpoint_type
- use[ironic]/valid_interfaces
option to set ironic endpoint types that will be attempted to be used
Several configuration options related to swift API access are deprecated and will be removed in Rocky release. These include:
[swift]/os_service_type
- use[swift]/service_type
option[swift]/os_endpoint_type
- use[swift]/valid_interfaces
option[swift]/os_region
- use[swift]region_name
option
Other Notes¶
The sample configuration file located at
example.conf
and the sample policy file located atpolicy.yaml.sample
were removed in this release, as they are now published with documentation. See the sample configuration file and the sample policy file.
7.0.0¶
New Features¶
Introduces the dnsmasq PXE filter driver. This driver takes advantage of the
inotify
facility to reconfigure the dnsmasq service in real time to implement a caching black-/white-list of port MAC addresses.
Upgrade Notes¶
A new state
aborting
was introduced to distinguish between the node introspection abort precondition (being able to perform the state transition from thewaiting
state) from the activities necessary to abort an ongoing node introspection (power-off, set finished timestamp etc.)
Handling of
local_gb
property was moved from thescheduler
hook toroot_disk_selection
.
Bug Fixes¶
The
node_info.finished(<transition>, error=<error>)
now updates node state together with other status attributes in a single DB transaction.
Other Notes¶
The tempest plugin code that was in
ironic_inspector/test/inspector_tempest_plugin/
has been removed. Tempest plugin code has been migrated to the project openstack/ironic-tempest-plugin. This was an OpenStack wide goal for the Queens cycle.
6.1.0¶
New Features¶
The PXE filter drivers mechanism is now enabled. The firewall-based filtering was re-implemented as the
iptables
PXE filter driver.
Adds an API access policy enforcment based on oslo.policy rules. Similar to other OpenStack services, operators now can configure fine-grained access policies using
policy.yaml
file. See policy.yaml.sample in the code tree for the list of available policies and their default rules. This file can also be generated from the code tree with the following command:tox -egenpolicy
See the oslo.policy package documentation for more information on using and configuring API access policies.
Upgrade Notes¶
Due to the choice of default values for API access policies rules, some API parts of the ironic-inspector service will become available to wider range of users after upgrade:
general access to the whole API is by default granted to a user with either
admin
,administrator
orbaremetal_admin
role (previously it allowed access only to a user withadmin
role)listing of current introspection statuses and showing a given introspection is by default also allowed to a user with the
baremetal_observer
role
If these access policies are not appropriate for your deployment, override them in a
policy.json
file in the ironic-inspector configuration directory (usually/etc/ironic-inspector
).See the oslo.policy package documentation for more information on using and configuring API access policies.
Deprecation Notes¶
The firewall-specific configuration options were moved from the
firewall
to theiptables
group. All options in theiptables
group are now deprecated.
The generic firewall options
firewall_update_period
andmanage_firewall
were moved under thepxe_filter
group assync_period
anddriver=iptables/noop
respectively.
Bug Fixes¶
A
version_id
is now explicitly generated during thenode_cache.start_introspection/.add_node
call to avoid race conditions such as in case of the two concurrent introspection calls bug.
The older
ipmi_address
field in the introspection data no longer has priority over the newerbmc_address
inventory field during lookup. This fixes lookup based on MAC addresses, when the BMC address is reported as0.0.0.0
for any reason (see bug 1714944).
Should the
iptables
PXE filter encounter an unexpected exception in the periodicsync
call, the exception will be logged and the filter driver will be reset in order to make subsequentsync
calls fail (and propagate the failure, exiting the ironic-inspector process eventually).
Other Notes¶
Allows a periodic task to shut down an ironic-inspector process upon a failure.
6.0.0¶
New Features¶
Querying ironic-inspector rules API now also returns the
invert
andmultiple
attributes of the associated conditions.
Add
disabled
option toadd_ports
, so discovered nodes can be created without creating ports.
Add a check from the
link_local_connection
plugin to use data stored by thelldp_basic
; this avoids parsing the LLDP packets twice.
Adds node state to the
GET /v1/introspection/<node UUID or name>
andGET /v1/introspection
API response data.
Processing hooks can now define dependencies on other processing hooks. ironic-inspector start up fails when required hooks are not enabled before the hook that requires them.
Update
pxe_enabled
field on ports. It is set toTrue
for the PXE-booting port andFalse
for the remaining ports. Both newly discovered and existing ports are affected.
Upgrade Notes¶
Experimental setting IPMI credentials support was removed from all versions of the API. The current ironic-inspector API version was bumped to 1.12 to mark this change.
The default API version was synchronized with the current API version again after removal of the IPMI credentials setting.
Ports creating logic was moved from core processing code to the
validate_interfaces
processing hook. This may affect deployments that disable this hook or replace it with something else. Also make sure to place this hook before any hooks expecting ports to be created.
Bare metal API version 1.19 is now required.
Removes deprecated configuration options:
introspection_delay_drivers
from the default section andlog_bmc_address
from theprocessing
section.
Support for rollback actions in introspection rules was removed.
Old status records are no longer removed by default. They are still removed if a node is removed from Ironic.
Deprecation Notes¶
The
node_status_keep_time
configuration option is deprecated. Now that we can remove status information about nodes removed from ironic, this option does not make much sense, and may be confusing
Bug Fixes¶
Timeout in an active state led to an undefined transition error. This is fixed and an introspection finishes now with
Timeout
error.
0.0.0.0
and an empty string in thebmc_address
inventory field are now correctly treated as missing BMC address.
For postgreSQL, the database migration command
ironic-inspector-dbsync upgrade
always failed (with enum NODE_STATE does not exist). This is fixed and the migration now works.
Do not fail the whole introspection due to a value formatting error during introspection rules rollback. See bug 1686942 for an example and detailed investigation.
5.1.0¶
Bug Fixes¶
The POST /v1/introspection/<Node ID>/data/unprocessed API updates the started_at time when ironic inspector begins processing the node.
Exception CalledProcessError is raised when running iptables cmd on start up. The issue is caused by eventlet bug, see: https://github.com/eventlet/eventlet/issues/357 The issue affects ironic-inspector only if it manages firewall - configured with
manage_firewall = True
configuration option.
Wrong provision state name ‘inspectfail’ in ironic-inspector valid states for node inspection. This issue leads to state inconsistency between ironic and ironic-inspector. For example, if ironic inspection timeout is lower than ironic-inspector’s, and inspection timeout occurs, ironic will transition node into ‘inspect failed’ provision state. In such case when node inspection finishes without errors the node will be in ‘inspect failed’ provision state with inspection in ‘finished’ state.
5.0.0¶
New Features¶
Extend the introspection status returned from
GET@/v1/introspection/<Node Id>
to contain theuuid
,started_at
andfinished_at
fields.
Add a plugin to parse raw LLDP Basic Management, 802.1, and 802.3 TLVs and store the data in Swift.
Add an API endpoint for listing introspection statuses. Operators can use this to get the status for all running or previously run introspection processing.
Introduce a new configuration option
api_max_limit
that defines the maximum number of items per page when API results are paginated.
InfiniBand interface discovery is now supported through introspection. The ironic-inspector will add the client-id to the corresponding ironic port that represents the InfiniBand interface. The ironic-inspector should be configured with a list of interfaces
firewall.ethoib_interfaces
to indicate which Ethernet Over InfiniBand Interfaces are used for DHCP.
Node introspection state is now kept in a dedicated database column. The introspection is now using a finite state machine. The state isn’t exposed to the user yet.
Adds support for using operators with the root device hints mechanism. The supported operators are
=
,==
,!=
,>=
,<=
,>
,<
,s==
,s!=
,s>=
,s>
,s<=
,s<
,<in>
,<all-in>
and<or>
.
Looking up nodes during introspection or discovery now supports multiple attributes matching. For example, two nodes can use the same
bmc_address
and still can be distinguished by MAC addresses.
Avoid failing introspection on diskless nodes. The node property
local_gb == 0
is set in that case.
Known Issues¶
Due to the nature of the NodeInfo.state attribute (being updated independently from the rest of the node_info attributes) if a (DB) connection was lost before the Node.state column was updated, Node.finished_at and Node.error columns may not be in sync with the Node.state column.
Upgrade Notes¶
Add a new dependency,
pytz
.
A database migration is required to change some columns from Float to DateTime type. This may take some time based on the number of introspection statuses in DB.
Removed previously deprecated authentication options from “ironic”, “swift”, and “keystone_authtoken” sections.
Removed long deprecated support for “discoverd” section in configuration file.
The default value for the configuration option “introspection_delay_drivers” was changed to
.*
, which means that by default “introspection_delay” is now applied to all drivers. Set “introspection_delay” to 0 to disable the delay.
Node.state and Node.version_id database columns are introduced.
The introspection state column defaults to the state
finished
unless the introspection error column value on a node row isn’t null, then node state is set toerror
.
Uniqueness of a node
bmc_address
isn’t enforced any more.
The primary key of the
attributes
table is relaxed from theattributes.name, attributes.value
column pair to a new columnattributes.uuid
.
Deprecation Notes¶
The configuration option “log_bmc_address” is deprecated.
Support for setting IPMI credentials via ironic-inspector is deprecated and will be removed completely in Pike. A new API version 1.9 was introduced with this feature de-activated. For reasoning see https://bugs.launchpad.net/ironic-python-agent/+bug/1654318.
The configuration option “introspection_delay_drivers” is deprecated.
Bug Fixes¶
Change database columns
started_at
andfinished_at
to type DateTime from type Float so that timestamps fit into these columns correctly.
Fix bug where periodic clean up failed with DBDeadlock if introspection timed out.
Ensure the configuration options
firewall.firewall_update_period
andclean_up_period
are applied to theperiodic_clean_up
andperiodic_update
tasks after the config file is read.
LLC hook now formats the chassis ID and port ID MAC addresses into Unix format as expected by ironic.
LLC hook ensures that correct port information is passed to the patch_port function
LLC hook no longer assumes all inspected ports are added to ironic
Loopback BMC addresses (useful e.g. with virtualbmc) are no longer used for lookup.
Introspection fails on nodes with the same IPMI address but different IPMI ports.
Other Notes¶
Default API version is temporary pinned to 1.8 (before deprecating setting IPMI credentials). It will be reset to the latest version again when support for setting IPMI credentials is removed.
4.2.0¶
New Features¶
Adds new processing hook pci_devices for setting node capabilities based on PCI devices present on a node and rules in the [pci_devices] aliases configuration option. Requires “pci-devices” collector to be enabled in IPA.
Bug Fixes¶
Use only single quotes for strings inside SQL statements. Fixes a crash when PostgreSQL is used as a database backend.
Set the node to the error state when it failed get data from swift.
4.1.0¶
New Features¶
Added GenericLocalLinkConnectionHook processing plugin to process LLDP data returned during inspection and set port ID and switch ID in an Ironic node’s port local link connection information using that data.
Add configuration option processing.power_off defaulting to True, which allows to leave nodes powered on after introspection.
Bug Fixes¶
Fix setting non string ‘value’ field for rule’s actions. As non string value is obviously not a formatted value, add the check to avoid AttributeError exception.
4.0.0¶
Prelude¶
Starting with this release only ironic-python-agent (IPA) is supported as an introspection ramdisk.
New Features¶
Added a new “capabilities” processing hook detecting the CPU and boot mode capabilities (the latter disabled by default).
File name for stored ramdisk logs can now be customized via “ramdisk_logs_filename_format” option.
Upgrade Notes¶
The default file name for stored ramdisk logs was change to contain only node UUID (if known) and the current date time. A proper “.tar.gz” extension is now appended.
API “POST /v1/rules” returns 201 response code instead of 200 on creating success. API version was bumped to 1.6. API less than 1.6 continues to return 200.
Default API version was changed from minimum to maximum which Inspector can support.
Support for the old bash-based ramdisk was removed. Please switch to IPA before upgrading.
Removed the deprecated “root_device_hint” alias for the “raid_device” hook.
Bug Fixes¶
Fixed “/v1/continue” to return HTTP 500 on unexpected exceptions, not HTTP 400.
Fix response return code for rule creating endpoint, it returns 201 now instead of 200 on success.
The “size” root device hint is now always converted to an integer for consistency with IPA.
3.3.0¶
New Features¶
Ironic-Inspector is now using keystoneauth and proper auth_plugins instead of keystoneclient for communicating with Ironic and Swift. It allows to finely tune authentification for each service independently. For each service, the keystone session is created and reused, minimizing the number of authentification requests to Keystone.
Add support for using Ironic node names in API instead of UUIDs. Note that using node names in the introspection status API will require a call to Ironic to be made by the service.
Database migrations downgrade was removed. More info about database migration/rollback could be found here https://docs.openstack.org/openstack-ops/content/ops_upgrades-roll-back.html
Introduced API “POST /v1/introspection/UUID/data/unprocessed” for reapplying the introspection over stored data.
Upgrade Notes¶
Operators are advised to specify a proper keystoneauth plugin and its appropriate settings in [ironic] and [swift] config sections. Backward compatibility with previous authentification options is included. Using authentification informaiton for Ironic and Swift from [keystone_authtoken] config section is no longer supported.
Handling ramdisk logs was moved out of the “ramdisk_error” plugin, so disabling it will no longer disable handling ramdisk logs. As before, you can set “ramdisk_logs_dir” option to an empty value (the default) to disable storing ramdisk logs.
Deprecation Notes¶
Most of current authentification options for either Ironic or Swift are deprecated and will be removed in a future release. Please configure the keystoneauth auth plugin authentification instead.
Bug Fixes¶
Fixes a problem which caused an unhandled TypeError exception to bubble up when inspector was attempting to convert some eDeploy data to integer.
Fixed a regression in the firewall code, which causes re-running introspection for an already inspected node to fail.
Fixed the “is-empty” condition to return True on missing values.
The lookup procedure now uses all valid MAC’s, not only the MAC(s) that will be used for creating port(s).
The “enroll” node_not_found_hook now uses all valid MAC’s to check node existence, not only the MAC(s) that will be used for creating port(s).
The ramdisk logs are now stored on all preprocessing errors, not only ones reported by the ramdisk itself. This required moving the ramdisk logs handling from the “ramdisk_error” plugin to the generic processing code.
3.2.0¶
New Features¶
Added CORS support middleware to Ironic Inspector, allowing a deployer to optionally configure rules under which a javascript client may break the single-origin policy and access the API directly.
- OpenStack CrossProject Spec:
https://specs.openstack.org/openstack/openstack-specs/specs/cors-support.html
- Oslo_Middleware Docs:
https://docs.openstack.org/developer/oslo.middleware/cors.html
- OpenStack Cloud Admin Guide:
https://docs.openstack.org/admin-guide-cloud/cross_project_cors.html
Bug Fixes¶
DHCP is now disabled completely when no nodes are on introspection and the “node_not_found_hook” is not set. This reduces probability of serving DHCP to wrong nodes, if their NIC is not registered in Ironic. See https://bugs.launchpad.net/ironic-inspector/+bug/1557979 and https://bugzilla.redhat.com/show_bug.cgi?id=1317695 for details.
Don’t fail on finish power off if node in ‘enroll’ state. Nodes in ‘enroll’ state are not expected to have power credentials.
3.1.0¶
New Features¶
Introduced API “POST /v1/introspection/<UUID>/abort” for aborting the introspection process.
New condition plugins “contains” and “matches” allow to match value against regular expressions.
Added new condition plugin “is-empty”, which allows to match empty string, list, dictionary or None.
Add a new node_not_found hook - enroll, which allows automatically discover Ironic’s node.
Conditions now support comparing fields from node info;
Actions support formatting to fetch values from introspection data. See https://docs.openstack.org/developer/ironic-inspector/usage.html#introspection-rules
Introspection rules conditions got a new generic “invert” parameter that inverts the result of the condition.
Upgrade Notes¶
Switch required Ironic API version to ‘1.11’, which supports ‘enroll’ state.
Minimum possible value for the “max_concurrency” setting is now 2.
Removed deprecated support for passing “node_patches” and “ports_patches” arguments to processing hooks.
Ramdisk logs are no longer part of data stored to Swift and returned by the API.
Introspection rules actions ‘set-attribute’, ‘set-capability’ and ‘extend-attribute’ no longer have the opposite effect on nodes that do not match a rule.
Deprecation Notes¶
The rollback actions for introspection rules are deprecated. No in-tree actions are using them, 3rdpart should stop using them as soon as possible.
Using the root_device_hint alias for the raid_device plugin is deprecated.
Bug Fixes¶
Fixed extra_hardware plugin connection to Swift.
Only issue iptables calls when list of active MAC’s changes.
Dropped rollback actions from ‘set-attribute’, ‘set-capability’ and ‘extend-attribute’ introspection rules actions, as they were confusing, completely undocumented and broke some real world use cases (e.g. setting driver field).
Introspection rules (e.g. set-attribute action) now accept ‘path’ field without leading forward slash as Ironic cli does.
Other Notes¶
Switched to Futurist library for asynchronous tasks.
Log level for error when node was not found in Inspector cache was changed from error to info level. It was done because not_found_hook may handle this case, so this wouldn’t be error anymore.
3.0.0¶
Prelude¶
Starting with this release, ironic-python-agent becomes the default introspection ramdisk, with the old bash-based ramdisk being deprecated.
New Features¶
Inspector no longer requires old-style “local_gb”, “memory_mb”, “cpus” and “cpu_arch” fields from the introspection ramdisk. They are still supported, though, for compatibility with the old ramdisk.
Upgrade Notes¶
Removed support for introspecting nodes in maintenance mode, deprecated in the liberty cycle. Use “inspecting”, “manageable” or “enroll” states instead.
The root_disk_selection processing hook will now error out if root device hints are specified on ironic node, but ironic-python-agent is not used as an introspection ramdisk.
Deprecation Notes¶
Using old bash-based ramdisk is deprecated, please switch to ironic-python-agent as soon as possible.
Bug Fixes¶
Fixed confusing error message shown to user when something bad happens during preprocessing (https://launchpad.net/bugs/1523907).
The data processing API endpoint now validates that data received from the ramdisk is actually a JSON object instead of failing the internal error later (issue https://bugs.launchpad.net/bugs/1525876).
Other Notes¶
Make debug-level logging more compact by removing newlines from firewall logging and disabling some 3rdparty debug messages by default.
Improve logging for ramdisk logs collection.
Logging during processing is now more consistent in terms of how it identifies the node. Now we try to prefix the log message with node UUID, BMC address and PXE MAC address (if available). Logging BMC addresses can be disabled via new “log_bmc_address” option in the “processing” section.
2.3.0¶
Prelude¶
This release includes automatic docs generation via Sphinx.
Critical Issues¶
Fixed several issues with MySQL database support:
Security Issues¶
Never enable Flask debug mode as it may allow remote code execution. See https://bugs.launchpad.net/bugs/1506419 for details.
Bug Fixes¶
Log a warning when add_ports is set to pxe, but no PXE MAC is returned from the ramdisk.
Acquire a lock on a node UUID when handling it.
Other Notes¶
IPA (ironic-python-agent) is now fully supported in the devstack plugin and will become the default ramdisk in the next release.
Allow autogeneration of database migrations.
Introduced new docs generation via Sphinx and ReST.
Separate doc folder includes source and build
Integration with tox as docs target
makefile for manual building
Openstack Theme support