Filtering hosts by isolating aggregates

Background

I want to set up an aggregate ABC with hosts that allow you to run only certain licensed images. I could tag the aggregate with metadata such as <LICENSED=WINDOWS>. Then if I boot an instance with an image containing the property <LICENSED=WINDOWS>, it will land on one of the hosts in aggregate ABC. But if the user creates a new image which does not include <LICENSED=WINDOWS> metadata, an instance booted with that image could still land on a host in aggregate ABC as reported in launchpad bug 1677217. The AggregateImagePropertiesIsolation scheduler filter passes even though the aggregate metadata <LICENSED=WINDOWS> is not present in the image properties.

Solution

The above problem is addressed by blueprint placement-req-filter-forbidden-aggregates which was implemented in the 20.0.0 Train release.

The following example assumes you have configured aggregate ABC and added hosts HOST1 and HOST2 to it in Nova, and that you want to isolate those hosts to run only instances requiring Windows licensing.

1. Set the scheduler.enable_isolated_aggregate_filtering config option to true in nova.conf and restart the nova-scheduler service.

2. Add trait CUSTOM_LICENSED_WINDOWS to the resource providers for HOST1 and HOST2 in the Placement service.

   First create the CUSTOM_LICENSED_WINDOWS trait

   # openstack --os-placement-api-version 1.6 trait create CUSTOM_LICENSED_WINDOWS
   

   Discover the ID of HOST1, which is its resource provider UUID

   # host_id=$(openstack resource provider show <HOST1_UUID>)
   

   Start to build the command line by first collecting existing traits for HOST1

   # traits=$(openstack --os-placement-api-version 1.6 resource provider trait list -f value $server_id | sed 's/^/--trait /')
   

   Replace HOST1’s traits, adding CUSTOM_LICENSED_WINDOWS

   # openstack --os-placement-api-version 1.6 resource provider trait set $traits --trait CUSTOM_LICENSED_WINDOWS $server_id
   

   Repeat the above steps for HOST2.

3. Add the trait:CUSTOM_LICENSED_WINDOWS=required metadata property to aggregate ABC.

   openstack --os-compute-api-version 2.53 aggregate set --property trait:CUSTOM_LICENSED_WINDOWS=required ABC
   

As before, any instance spawned with a flavor or image containing trait:CUSTOM_LICENSED_WINDOWS=required will land on HOST1 or HOST2 because those hosts expose that trait.

However, now that the isolate_aggregates request filter is configured, any instance whose flavor or image does not contain trait:CUSTOM_LICENSED_WINDOWS=required will not land on HOST1 or HOST2 because aggregate ABC requires that trait.

The above example uses a CUSTOM_LICENSED_WINDOWS trait, but you can use any custom or standard trait in a similar fashion.

The filter supports the use of multiple traits across multiple aggregates. The combination of flavor and image metadata must require all of the traits configured on the aggregate in order to pass.