Add InfiniBand Support

https://bugs.launchpad.net/ironic/+bug/1532534

Today, Ironic supports Ethernet interfaces for Hardware inspection and PXE boot. Ironic should have the ability to inspect and PXE boot over InfiniBand network as well.

Problem description

  • Hardware inspection for InfiniBand - A InfiniBand GUID is similar in concept to a MAC address because it consists of a 24-bit manufacturer’s prefix and a 40-bit device identifier (64 bits total).

  • PXE Boot over InfiniBand - To allow DHCP over IPoIB interface the DHCP client must send the client-id with a unique identifying value for the client. The value is consist of the vendor prefix 12 bytes and GUID 8 bytes. Today ironic doesn’t send update the neutron port with client-id option.

Proposed change

  • Hardware inspection for InfiniBand - In order to use InfiniBand with PXE you have to flash the NIC with a vendor specific firmware. The vendor firmware defined the conversion of GUID to “InfiniBand MAC” (48 bits). To resource complicity of the change, the ironic address will contain the “InfiniBand MAC”.

  • PXE/iPXE Boot over InfiniBand changes: To allow DHCP over InfiniBand we need the following:

    1. The dhcp-server must use the BROADCAST flag in the dhcp-server. This already support in neutron-dhcp-agent by config file.

    2. Updating the ironic port extra attribute to contains the InfiniBand port client-id extra e.g:

      {
    'client-id':
    'ff:00:00:00:00:00:02:00:00:02:c9:00:00:02:c9:03:00:00:10:39'
}

      The client-id update can be done manually or with IPA and ironic-inspector.

    3. The neutron port that represent the ironic port should be updated with client-id option in the extra_dhcp_opts attribute. The client-id consists of a vendor prefix and the port GUID. The client id for Mellanox ConnectX Family Devices is consists of a prefix (ff:00:00:00:00:00:02:00:00:02:c9:00) and 8 byte port GUID. The prefix in the client-id is vendor specific.

    4. The PXE MAC file name consists of the <Hardware Type>-<MAC>. For InfiniBand the hardware Type is 20 and the mac is the InfiniBand truncate GUID.

    5. The iPXE MAC file name consists of the <MAC>. For InfiniBand the MAC is the InfiniBand truncate GUID.

Other projects changes:

  • ironic-python-agent changes:

    1. Update the ironic agent to calculate the InfiniBand truncate GUID and the Client ID.

    2. Update coreos and tinyipa with ib_ipoib driver.

  • ironic-inspector changes:

    1. Update the ironic-inspector to update port.extra with client-id

  • diskimage-builder changes:

    1. Update the mellanox element to load ib_ipoib driver.

Alternatives

  • Extend the ironic port to support GUID which is 8 bytes and calculate the client-id in the ironic code from the GUID. This will require updating the ironic model and API. This will require updating the nova ironic driver to truncate the GUID to MAC.

Data model impact

None

State Machine Impact

None

REST API impact

None

Client (CLI) impact

None

“ironic” CLI

None

“openstack baremetal” CLI

None

RPC API impact

None

Driver API impact

None

Nova driver impact

None

Ramdisk impact

N/A

Security impact

None

Other end user impact

None

Scalability impact

None

Performance Impact

None

Other deployer impact

When using IPA, the deployer needs IPA that provides the InfiniBand MAC and client-id.

Developer impact

None

Implementation

Assignee(s)

Primary assignee:

moshele

Other contributors:

None

Work Items

  • Add Client-ID option to the neutron port to allow DHCP.

  • Update the generation of the iPXE/PXE file.

  • Update documentation.

Dependencies

None

Testing

  • Adding unit tests.

  • Adding Third-party CI which will test Mellanox hardware.

Upgrades and Backwards Compatibility

None

Documentation Impact

  • We will update the ironic documentation on how to allow pxe boot from IPoIB.

References