Advanced features

Specifying the disk for deployment (root device hints)

The Bare Metal service supports passing hints to the deploy ramdisk about which disk it should pick for the deployment. The list of supported hints is:

  • model (STRING): device identifier

  • vendor (STRING): device vendor

  • serial (STRING): disk serial number

    Note

    Some RAID controllers will generate serial numbers to represent volumes provided to the operating system which do not match or align to physical disks in a system.

  • size (INT): size of the device in GiB

    Note

    A node’s ‘local_gb’ property is often set to a value 1 GiB less than the actual disk size to account for partitioning (this is how DevStack, TripleO and Ironic Inspector work, to name a few). However, in this case size should be the actual size. For example, for a 128 GiB disk local_gb will be 127, but size hint will be 128.

  • wwn (STRING): unique storage identifier and typically mapping to a device. This can be a single device, or a SAN storage controller, or a RAID controller.

  • wwn_with_extension (STRING): unique storage identifier with the vendor extension appended

  • wwn_vendor_extension (STRING): unique vendor storage identifier

  • rotational (BOOLEAN): whether it’s a rotational device or not. This hint makes it easier to distinguish HDDs (rotational) and SSDs (not rotational) when choosing which disk Ironic should deploy the image onto.

  • hctl (STRING): the SCSI address (Host, Channel, Target and Lun), e.g ‘1:0:0:0’

  • by_path (STRING): the alternate device name corresponding to a particular PCI or iSCSI path, e.g /dev/disk/by-path/pci-0000:00

    Note

    Device identification by-path may not be reliable on Linux kernels using asynchronous device initialization.

  • name (STRING): the device name, e.g /dev/md0

    Warning

    The root device hint name should only be used for devices with constant names (e.g RAID volumes). For SATA, SCSI and IDE disk controllers this hint is not recommended because the order in which the device nodes are added in Linux is arbitrary, resulting in devices like /dev/sda and /dev/sdb switching around at boot time.

    Warning

    Furthermore, recent move to asynchronous device initialization among some Linux distribution kernels means that the actual device name string is entirely unreliable when multiple devices are present in the host, as the device name is claimed by the device which responded first, as opposed to the previous pattern where it was the first initialized device in a synchronous process.

To associate one or more hints with a node, update the node’s properties with a root_device key, for example:

baremetal node set <node-uuid> --property root_device='{"wwn": "0x4000cca77fc4dba1"}'

That will guarantee that Bare Metal service will pick the disk device that has the wwn equal to the specified wwn value, or fail the deployment if it can not be found.

Note

Starting with the Ussuri release, root device hints can be specified per-instance, see Using Bare Metal service as a standalone service.

The hints can have an operator at the beginning of the value string. If no operator is specified the default is == (for numerical values) and s== (for string values). The supported operators are:

  • For numerical values:

    • = equal to or greater than. This is equivalent to >= and is supported for legacy reasons

    • == equal to

    • != not equal to

    • >= greater than or equal to

    • > greater than

    • <= less than or equal to

    • < less than

  • For strings (as python comparisons):

    • s== equal to

    • s!= not equal to

    • s>= greater than or equal to

    • s> greater than

    • s<= less than or equal to

    • s< less than

    • <in> substring

  • For collections:

    • <all-in> all elements contained in collection

    • <or> find one of these

Examples are:

  • Finding a disk larger or equal to 60 GiB and non-rotational (SSD):

    baremetal node set <node-uuid> --property root_device='{"size": ">= 60", "rotational": false}'

  • Finding a disk whose vendor is samsung or winsys:

    baremetal node set <node-uuid> --property root_device='{"vendor": "<or> samsung <or> winsys"}'

Note

If multiple hints are specified, a device must satisfy all the hints.

Appending kernel parameters to boot instances

The Bare Metal service supports passing custom kernel parameters to boot instances to fit users’ requirements. The way to append the kernel parameters is depending on how to boot instances.

Network boot

Currently, the Bare Metal service supports assigning unified kernel parameters to PXE booted instances by:

  • Modifying the [pxe]/kernel_append_params configuration option, for example:

    [pxe]
kernel_append_params = quiet splash

    Note

    The option was called pxe_append_params before the Xena cycle.

  • Copying a template from shipped templates to another place, for example:

    https://opendev.org/openstack/ironic/src/branch/master/ironic/drivers/modules/pxe_config.template

    Making the modifications and pointing to the custom template via the configuration options: [pxe]/pxe_config_template and [pxe]/uefi_pxe_config_template.

Local boot

For local boot instances, users can make use of configuration drive (see Enabling the configuration drive (configdrive)) to pass a custom script to append kernel parameters when creating an instance. This is more flexible and can vary per instance. Here is an example for grub2 with ubuntu, users can customize it to fit their use case:

#!/usr/bin/env python
import os

# Default grub2 config file in Ubuntu
grub_file = '/etc/default/grub'
# Add parameters here to pass to instance.
kernel_parameters = ['quiet', 'splash']
grub_cmd = 'GRUB_CMDLINE_LINUX'
old_grub_file = grub_file+'~'
os.rename(grub_file, old_grub_file)
cmdline_existed = False
with open(grub_file, 'w') as writer, \
       open(old_grub_file, 'r') as reader:
       for line in reader:
           key = line.split('=')[0]
           if key == grub_cmd:
               #If there is already some value:
               if line.strip()[-1] == '"':
                   line = line.strip()[:-1] + ' ' + ' '.join(kernel_parameters) + '"'
               cmdline_existed = True
           writer.write(line)
       if not cmdline_existed:
           line = grub_cmd + '=' + '"' + ' '.join(kernel_parameters) + '"'
           writer.write(line)

os.remove(old_grub_file)
os.system('update-grub')
os.system('reboot')

Console

In order to change default console configuration in the Bare Metal service configuration file ([pxe] section in /etc/ironic/ironic.conf), include the serial port terminal and serial speed. Serial speed must be the same as the serial configuration in the BIOS settings, so that the operating system boot process can be seen in the serial console or web console. Following examples represent possible parameters for serial and web console respectively.

  • Node serial console. The console parameter console=ttyS0,115200n8 uses ttyS0 for console output at 115200bps, 8bit, non-parity, e.g.:

    [pxe]

# Additional append parameters for baremetal PXE boot.
kernel_append_params = nofb vga=normal console=ttyS0,115200n8

  • For node web console configuration is similar with the addition of ttyX parameter, see example:

    [pxe]

# Additional append parameters for baremetal PXE boot.
kernel_append_params = nofb vga=normal console=tty0 console=ttyS0,115200n8

For detailed information on how to add consoles see the reference documents kernel params and serial console. In case of local boot the Bare Metal service is not able to control kernel boot parameters. To configure console locally, follow ‘Local boot’ section above.

Boot mode support

Some of the bare metal hardware types (namely, redfish, ilo and generic ipmi) support setting boot mode (Legacy BIOS or UEFI).

Note

Setting boot mode support in generic ipmi driver is coupled with setting boot device. That makes boot mode support in the ipmi driver incomplete.

Note

In this chapter we will distinguish ironic node from bare metal node. The difference is that ironic node refers to a logical node, as it is configured in ironic, while bare metal node indicates the hardware machine that ironic is managing.

The following rules apply in order when ironic manages node boot mode:

  • If the hardware type (or bare metal node) does not implement reading current boot mode of the bare metal node, then ironic assumes that boot mode is not set on the bare metal node

  • If boot mode is not set on ironic node and bare metal node boot mode is unknown (not set, can’t be read etc.), ironic node boot mode is set to the value of the [deploy]/default_boot_mode option

  • If boot mode is set on a bare metal node, but is not set on ironic node, bare metal node boot mode is set on ironic node

  • If boot mode is set on ironic node, but is not set on the bare metal node, ironic node boot mode is attempted to be set on the bare metal node (failure to set boot mode on the bare metal node will not fail ironic node deployment)

  • If different boot modes appear on to be set ironic node and on the bare metal node, ironic node boot mode is attempted to be set on the bare metal node (failure to set boot mode on the bare metal node will fail ironic node deployment)

Warning

If a bare metal node does not support setting boot mode, then the operator needs to make sure that boot mode configuration is consistent between ironic node and the bare metal node.

The boot modes can be configured in the Bare Metal service in the following way:

  • Only one boot mode (either uefi or bios) can be configured for the node.

  • If the operator wants a node to boot always in uefi mode or bios mode, then they may use capabilities parameter within properties field of an bare metal node. The operator must manually set the appropriate boot mode on the bare metal node.

    To configure a node in uefi mode, then set capabilities as below:

    openstack baremetal node set <node-uuid> --property capabilities='boot_mode:uefi'

    Conversely, to configure a node in bios mode, then set the capabilities as below:

    openstack baremetal node set <node-uuid> --property capabilities='boot_mode:bios'

Note

The Ironic project changed the default boot mode setting for nodes from bios to uefi during the Yoga development cycle.

Nodes having boot_mode set to uefi may be requested by adding an extra_spec to the Compute service flavor:

openstack flavor set ironic-test-3 --property capabilities:boot_mode="uefi"
openstack server create boot --flavor ironic-test-3 --image test-image instance-1

If capabilities is used in extra_spec as above, nova scheduler (ComputeCapabilitiesFilter) will match only bare metal nodes which have the boot_mode set appropriately in properties/capabilities. It will filter out rest of the nodes.

The above facility for matching in the Compute service can be used in heterogeneous environments where there is a mix of uefi and bios machines, and operator wants to provide a choice to the user regarding boot modes. If the flavor doesn’t contain boot_mode and boot_mode is configured for bare metal nodes, then nova scheduler will consider all nodes and user may get either bios or uefi machine.

Some hardware support setting secure boot mode, see UEFI secure boot mode for details.

Choosing the disk label

Note

The term disk label is historically used in Ironic and was taken from parted. Apparently everyone seems to have a different word for disk label - these are all the same thing: disk type, partition table, partition map and so on…

Ironic allows operators to choose which disk label they want their bare metal node to be deployed with when Ironic is responsible for partitioning the disk; therefore choosing the disk label does not apply when the image being deployed is a whole disk image.

There are some edge cases where someone may want to choose a specific disk label for the images being deployed, including but not limited to:

  • For machines in bios boot mode with disks larger than 2 terabytes it’s recommended to use a gpt disk label. That’s because a capacity beyond 2 terabytes is not addressable by using the MBR partitioning type. But, although GPT claims to be backward compatible with legacy BIOS systems that’s not always the case.

  • Operators may want to force the partitioning to be always MBR (even if the machine is deployed with boot mode uefi) to avoid breakage of applications and tools running on those instances.

The disk label can be configured in two ways; when Ironic is used with the Compute service or in standalone mode. The following bullet points and sections will describe both methods:

  • When no disk label is provided Ironic will configure it according to the boot mode (see Boot mode support); bios boot mode will use msdos and uefi boot mode will use gpt.

  • Only one disk label - either msdos or gpt - can be configured for the node.

Warning

If the host is in UEFI boot mode, use of disk_label is redundant, and may cause deployments to fail unexpectedly if the node is not explicitly set to boot in UEFI mode. Use of appropriate boot mode is highly recommended.

When used with Compute service

When Ironic is used with the Compute service the disk label should be set to node’s properties/capabilities field and also to the flavor which will request such capability, for example:

baremetal node set <node-uuid> --property capabilities='disk_label:gpt'

As for the flavor:

openstack flavor set baremetal --property capabilities:disk_label="gpt"

When used in standalone mode

When used without the Compute service, the disk label should be set directly to the node’s instance_info field, as below:

baremetal node set <node-uuid> --instance-info capabilities='{"disk_label": "gpt"}'

Notifications

The Bare Metal service supports the emission of notifications, which are messages sent on a message broker (like RabbitMQ or anything else supported by the oslo messaging library) that indicate various events which occur, such as when a node changes power states. These can be consumed by an external service reading from the message bus. For example, Searchlight is an OpenStack service that uses notifications to index (and make searchable) resources from the Bare Metal service.

Notifications are disabled by default. For a complete list of available notifications and instructions for how to enable them, see the Notifications.

Configuring node web console

See Configuring Web or Serial Console.