Usage¶

Using from Ironic API¶

Ironic Kilo introduced support for hardware introspection under name of “inspection”. ironic-inspector introspection is supported for some generic drivers, please refer to Ironic inspection documentation for details.

Node States¶

• The nodes should be moved to MANAGEABLE provision state before introspection (requires python-ironicclient of version 0.5.0 or newer):

  ironic node-set-provision-state <UUID> manage
  

• After successful introspection and before deploying nodes should be made available to Nova, by moving them to AVAILABLE state:

  ironic node-set-provision-state <UUID> provide
  

  Note

  Due to how Nova interacts with Ironic driver, you should wait 1 minute before Nova becomes aware of available nodes after issuing this command. Use nova hypervisor-stats command output to check it.

Introspection Rules¶

Inspector supports a simple JSON-based DSL to define rules to run during introspection. Inspector provides an API to manage such rules, and will run them automatically after running all processing hooks.

A rule consists of conditions to check, and actions to run. If conditions evaluate to true on the introspection data, then actions are run on a node.

Available conditions and actions are defined by plugins, and can be extended, see How To Contribute for details. See api for specific calls to define introspection rules.

{"field": "node://property.path", "op": "eq", "value": "val"}
{"field": "data://introspection.path", "op": "eq", "value": "val"}

{"action": "set-attribute", "path": "/driver_info/ipmi_address",
 "value": "{data[inventory][bmc_address]}"}

Plugins¶

ironic-inspector heavily relies on plugins for data processing. Even the standard functionality is largely based on plugins. Set processing_hooks option in the configuration file to change the set of plugins to be run on introspection data. Note that order does matter in this option, especially for hooks that have dependencies on other hooks.

These are plugins that are enabled by default and should not be disabled, unless you understand what you’re doing:

scheduler

validates and updates basic hardware scheduling properties: CPU number and architecture, memory and disk size.

Note

Diskless nodes have the disk size property local_gb == 0. Always use node driver root_device hints to prevent unexpected HW failures passing silently.

validate_interfaces validates network interfaces information. Creates new

ports, optionally deletes ports that were not present in the introspection data. Also sets the pxe_enabled flag for the PXE-booting port and unsets it for all the other ports to avoid nova picking a random port to boot the node.

The following plugins are enabled by default, but can be disabled if not needed:

ramdisk_error

reports error, if error field is set by the ramdisk, also optionally stores logs from logs field, see api for details.

capabilities

detect node capabilities: CPU, boot mode, etc. See Capabilities Detection for more details.

pci_devices

gathers the list of all PCI devices returned by the ramdisk and compares to those defined in alias field(s) from pci_devices section of configuration file. The recognized PCI devices and their count are then stored in node properties. This information can be later used in nova flavors for node scheduling.

Here are some plugins that can be additionally enabled:

example

example plugin logging it’s input and output.

raid_device

gathers block devices from ramdisk and exposes root device in multiple runs.

extra_hardware

stores the value of the ‘data’ key returned by the ramdisk as a JSON encoded string in a Swift object. The plugin will also attempt to convert the data into a format usable by introspection rules. If this is successful then the new format will be stored in the ‘extra’ key. The ‘data’ key is then deleted from the introspection data, as unless converted it’s assumed unusable by introspection rules.

local_link_connection

Processes LLDP data returned from inspection specifically looking for the port ID and chassis ID, if found it configures the local link connection information on the nodes Ironic ports with that data. To enable LLDP in the inventory from IPA ipa-collect-lldp=1 should be passed as a kernel parameter to the IPA ramdisk. In order to avoid processing the raw LLDP data twice, the lldp_basic plugin should also be installed and run prior to this plugin.

lldp_basic

Processes LLDP data returned from inspection and parses TLVs from the Basic Management (802.1AB), 802.1Q, and 802.3 sets and stores the processed data back to the Ironic inspector data in Swift.

Refer to How To Contribute for information on how to write your own plugin.

Discovery¶

Starting from Mitaka, ironic-inspector is able to register new nodes in Ironic.

The existing node-not-found-hook handles what happens if ironic-inspector receives inspection data from a node it can not identify. This can happen if a node is manually booted without registering it with Ironic first.

For discovery, the configuration file option node_not_found_hook should be set to load the hook called enroll. This hook will enroll the unidentified node into Ironic using the fake driver (this driver is a configurable option, set enroll_node_driver in the ironic-inspector configuration file, to the Ironic driver you want).

The enroll hook will also set the ipmi_address property on the new node, if its available in the introspection data we received, see ramdisk_callback.

Once the enroll hook is finished, ironic-inspector will process the introspection data in the same way it would for an identified node. It runs the processing plugins, and after that it runs introspection rules, which would allow for more customisable node configuration, see rules.

A rule to set a node’s Ironic driver to the agent_ipmitool driver and populate the required driver_info for that driver would look like:

[{
    "description": "Set IPMI driver_info if no credentials",
    "actions": [
        {"action": "set-attribute", "path": "driver", "value": "agent_ipmitool"},
        {"action": "set-attribute", "path": "driver_info/ipmi_username",
         "value": "username"},
        {"action": "set-attribute", "path": "driver_info/ipmi_password",
         "value": "password"}
    ],
    "conditions": [
        {"op": "is-empty", "field": "node://driver_info.ipmi_password"},
        {"op": "is-empty", "field": "node://driver_info.ipmi_username"}
    ]
},{
    "description": "Set deploy info if not already set on node",
    "actions": [
        {"action": "set-attribute", "path": "driver_info/deploy_kernel",
         "value": "<glance uuid>"},
        {"action": "set-attribute", "path": "driver_info/deploy_ramdisk",
         "value": "<glance uuid>"}
    ],
    "conditions": [
        {"op": "is-empty", "field": "node://driver_info.deploy_ramdisk"},
        {"op": "is-empty", "field": "node://driver_info.deploy_kernel"}
    ]
}]

All nodes discovered and enrolled via the enroll hook, will contain an auto_discovered flag in the introspection data, this flag makes it possible to distinguish between manually enrolled nodes and auto-discovered nodes in the introspection rules using the rule condition eq:

{
    "description": "Enroll auto-discovered nodes with fake driver",
    "actions": [
        {"action": "set-attribute", "path": "driver", "value": "fake"}
    ],
    "conditions": [
        {"op": "eq", "field": "data://auto_discovered", "value": true}
    ]
}

Reapplying introspection on stored data¶

To allow correcting mistakes in introspection rules the API provides an entry point that triggers the introspection over stored data. The data to use for processing is kept in Swift separately from the data already processed. Reapplying introspection overwrites processed data in the store. Updating the introspection data through the endpoint isn’t supported yet. Following preconditions are checked before reapplying introspection:

• no data is being sent along with the request
• Swift store is configured and enabled
• introspection data is stored in Swift for the node UUID
• node record is kept in database for the UUID
• introspection is not ongoing for the node UUID

Should the preconditions fail an immediate response is given to the user:

• 400 if the request contained data or in case Swift store is not enabled in configuration
• 404 in case Ironic doesn’t keep track of the node UUID
• 409 if an introspection is already ongoing for the node

If the preconditions are met a background task is executed to carry out the processing and a 202 Accepted response is returned to the endpoint user. As requested, these steps are performed in the background task:

• preprocessing hooks
• post processing hooks, storing result in Swift
• introspection rules

These steps are avoided, based on the feature requirements:

• node_not_found_hook is skipped
• power operations
• roll-back actions done by hooks

Limitations:

• there’s no way to update the unprocessed data atm.
• the unprocessed data is never cleaned from the store
• check for stored data presence is performed in background; missing data situation still results in a 202 response

Capabilities Detection¶

Starting with the Newton release, Ironic Inspector can optionally discover several node capabilities. A recent (Newton or newer) IPA image is required for it to work.

cpu_flags = aes:cpu_aes,svm:cpu_vt,vmx:cpu_vt

# EMAC=<ethernet mac of the ethoib> IMAC=<qp number:lid:GUID>
# For example:
IMAC=97:fe:80:00:00:00:00:00:00:7c:fe:90:03:00:29:26:52
qp number=97:fe
lid=80:00:00:00:00:00:00
GUID=7c:fe:90:03:00:29:26:52

EMAC=02:00:02:97:00:01 IMAC=97:fe:80:00:00:00:00:00:00:7c:fe:90:03:00:29:26:52
EMAC=02:00:00:61:00:02 IMAC=61:fe:80:00:00:00:00:00:00:7c:fe:90:03:00:29:24:4f