Cells Layout (v2)¶

Cross-cell instance migrations¶

Currently it is not possible to migrate an instance from a host in one cell to a host in another cell. This may be possible in the future, but it is currently unsupported. This impacts cold migration, resizes, live migrations, evacuate, and unshelve operations.

Quota-related quirks¶

Quotas are now calculated live at the point at which an operation would consume more resource, instead of being kept statically in the database. This means that a multi-cell environment may incorrectly calculate the usage of a tenant if one of the cells is unreachable, as those resources cannot be counted. In this case, the tenant may be able to consume more resource from one of the available cells, putting them far over quota when the unreachable cell returns. In the future, placement will provide us with a consistent way to calculate usage independent of the actual cell being reachable.

Performance of listing instances¶

With multiple cells, the instance list operation may not sort and paginate results properly when crossing multiple cell boundaries. Further, the performance of a sorted list operation will be considerably slower than with a single cell.

Notifications¶

With a multi-cell environment with multiple message queues, it is likely that operators will want to configure a separate connection to a unified queue for notifications. This can be done in the configuration file of all nodes. See the oslo.messaging configuration documentation for more details.

Neutron Metadata API proxy¶

The Neutron metadata API proxy should be global across all cells, and thus be configured as an API-level service with access to the [api_database]/connection information.

Consoleauth service and console proxies¶

The consoleauth service should be global across all cells and thus be configured as an API-level service with access to the [api_database]/connection information. The various console proxies should also be global across all cells but they don't need access to the API database.

Future work will deprecate the consoleauth service, store token authorizations in the cell databases, and require console proxies running per cell instead of globally.

Operations Requiring upcalls¶

If you deploy multiple cells with a superconductor as described above, computes and cell-based conductors will not have the ability to speak to the scheduler as they are not connected to the same MQ. This is by design for isolation, but currently the processes are not in place to implement some features without such connectivity. Thus, anything that requires a so-called "upcall" will not function. This impacts the following:

1. Instance reschedules during boot and resize

   Note

   This has been resolved in the Queens release [2].

2. Instance affinity reporting from the compute nodes to scheduler

3. The late anti-affinity check during server create and evacuate

4. Querying host aggregates from the cell

5. Attaching a volume and [cinder]/cross_az_attach=False

The first is simple: if you boot an instance, it gets scheduled to a compute node, fails, it would normally be re-scheduled to another node. That requires scheduler intervention and thus it will not work in Pike with a multi-cell layout. If you do not rely on reschedules for covering up transient compute-node failures, then this will not affect you. To ensure you do not make futile attempts at rescheduling, you should set [scheduler]/max_attempts=1 in nova.conf.

The second two are related. The summary is that some of the facilities that Nova has for ensuring that affinity/anti-affinity is preserved between instances does not function in Pike with a multi-cell layout. If you don't use affinity operations, then this will not affect you. To make sure you don't make futile attempts at the affinity check, you should set [workarounds]/disable_group_policy_check_upcall=True and [filter_scheduler]/track_instance_changes=False in nova.conf.

The fourth is currently only a problem when performing live migrations using the XenAPI driver and not specifying --block-migrate. The driver will attempt to figure out if block migration should be performed based on source and destination hosts being in the same aggregate. Since aggregates data has migrated to the API database, the cell conductor will not be able to access the aggregate information and will fail.

The fifth is a problem because when a volume is attached to an instance in the nova-compute service, and [cinder]/cross_az_attach=False in nova.conf, we attempt to look up the availability zone that the instance is in which includes getting any host aggregates that the instance.host is in. Since the aggregates are in the API database and the cell conductor cannot access that information, so this will fail. In the future this check could be moved to the nova-api service such that the availability zone between the instance and the volume is checked before we reach the cell, except in the case of boot from volume where the nova-compute service itself creates the volume and must tell Cinder in which availability zone to create the volume. Long-term, volume creation during boot from volume should be moved to the top-level superconductor which would eliminate this AZ up-call check problem.