Cells functionality allows you to scale an OpenStack Compute cloud in a more distributed fashion without having to use complicated technologies like database and message queue clustering. It is intended to support very large deployments.
When this functionality is enabled, the hosts in an OpenStack Compute cloud are
partitioned into groups called cells. Cells are configured as a tree. The top-level cell
should have a host that runs a nova-api
service,
but no nova-compute
services. Each child cell
should run all of the typical nova-*
services in a
regular Compute cloud except for nova-api
. You can
think of cells as a normal Compute deployment in that each cell has its own database server
and message queue broker.
The nova-cells
service handles communication
between cells and selects cells for new instances. This service is required for every cell.
Communication between cells is pluggable, and currently the only option is communication through
RPC.
Cells scheduling is separate from host scheduling. nova-cells
first picks a cell. Once a cell is selected and the new build
request reaches its nova-cells
service, it is sent
over to the host scheduler in that cell and the build proceeds
as it would have without cells.
Warning | |
---|---|
Cell functionality is currently considered experimental. |
Cells are disabled by default. All cell-related configuration options go under a
[cells]
section in nova.conf
. The following
cell-related options are currently supported:
enable
Set this is
True
to turn on cell functionality, which is off by default.name
Name of the current cell. This must be unique for each cell.
capabilities
List of arbitrary
pairs defining capabilities of the current cell. Values includekey
=value
hypervisor=xenserver;kvm,os=linux;windows
.call_timeout
How long in seconds to wait for replies from calls between cells.
- scheduler_filter_classes
Filter classes that the cells scheduler should use. By default, uses "
nova.cells.filters.all_filters
" to map to all cells filters included with Compute.- scheduler_weight_classes
Weight classes the cells scheduler should use. By default, uses "
nova.cells.weights.all_weighers
" to map to all cells weight algorithms (weighers) included with Compute.- ram_weight_multiplier
Multiplier used for weighing ram. Negative numbers mean you want Compute to stack VMs on one host instead of spreading out new VMs to more hosts in the cell. Default value is 10.0.
The compute API class must be changed in the API cell so that requests can be proxied
through nova-cells down to the correct cell properly. Add the following to
nova.conf
in the API
cell:
[DEFAULT] compute_api_class=nova.compute.cells_api.ComputeCellsAPI ... [cells] enable=True name=api
Add the following to nova.conf
in the child cells, replacing
cell1
with the name of each
cell:
[DEFAULT]
# Disable quota checking in child cells. Let API cell do it exclusively.
quota_driver=nova.quota.NoopQuotaDriver
[cells]
enable=True
name=cell1
Before bringing the services online, the database in each cell needs to be configured with information about related cells. In particular, the API cell needs to know about its immediate children, and the child cells must know about their immediate agents. The information needed is the RabbitMQ server credentials for the particular cell.
Use the nova-manage cell create command to add this information to the database in each cell:
# nova-manage cell create -h Options: -h, --help show this help message and exit --name=<name> Name for the new cell --cell_type=<parent|child> Whether the cell is a parent or child --username=<username> Username for the message broker in this cell --password=<password> Password for the message broker in this cell --hostname=<hostname> Address of the message broker in this cell --port=<number> Port number of the message broker in this cell --virtual_host=<virtual_host> The virtual host of the message broker in this cell --woffset=<float> (weight offset) It might be used by some cell scheduling code in the future --wscale=<float> (weight scale) It might be used by some cell scheduling code in the future
As an example, assume we have an API cell named api
and a child
cell named cell1
. Within the api cell, we have the following RabbitMQ
server
info:
rabbit_host=10.0.0.10 rabbit_port=5672 rabbit_username=api_user rabbit_password=api_passwd rabbit_virtual_host=api_vhost
And in the child cell named cell1
we have the following RabbitMQ
server
info:
rabbit_host=10.0.1.10 rabbit_port=5673 rabbit_username=cell1_user rabbit_password=cell1_passwd rabbit_virtual_host=cell1_vhost
We would run this in the API cell, as root.
# nova-manage cell create --name=cell1 --cell_type=child \ --username=cell1_user --password=cell1_passwd --hostname=10.0.1.10 \ --port=5673 --virtual_host=cell1_vhost --woffset=1.0 --wscale=1.0
Repeat the above for all child cells.
In the child cell, we would run the following, as root:
# nova-manage cell create --name=api --cell_type=parent \ --username=api_user --password=api_passwd --hostname=10.0.0.10 \ --port=5672 --virtual_host=api_vhost --woffset=1.0 --wscale=1.0
To customize the Compute cells, use the configuration option settings documented in Table 2.18, “Description of configuration options for cells”.
To determine the best cell for launching a new instance, Compute uses a set of
filters and weights configured in /etc/nova/nova.conf
. The
following options are available to prioritize cells for scheduling:
scheduler_filter_classes
- Specifies the list of filter classes. By defaultnova.cells.weights.all_filters
is specified, which maps to all cells filters included with Compute (see the section called “Filters”).scheduler_weight_classes
- Specifies the list of weight classes. By defaultnova.cells.weights.all_weighers
is specified, which maps to all cell weight algorithms (weighers) included with Compute. The following modules are available:mute_child
: Downgrades the likelihood of child cells being chosen for scheduling requests, which haven't sent capacity or capability updates in a while. Options includemute_weight_multiplier
(multiplier for mute children; value should be negative) andmute_weight_value
(assigned to mute children; should be a positive value).ram_by_instance_type
: Select cells with the most RAM capacity for the instance type being requested. Because higher weights win, Compute returns the number of available units for the instance type requested. Theram_weight_multiplier
option defaults to 10.0 that adds to the weight by a factor of 10. Use a negative number to stack VMs on one host instead of spreading out new VMs to more hosts in the cell.weight_offset
: Allows modifying the database to weight a particular cell. You can use this when you want to disable a cell (for example, '0'), or to set a default cell by making its weight_offset very high (for example, '999999999999999'). The highest weight will be the first cell to be scheduled for launching an instance.
Additionally, the following options are available for the cell scheduler:
scheduler_retries
- Specifies how many times the scheduler tries to launch a new instance when no cells are available (default=10).scheduler_retry_delay
- Specifies the delay (in seconds) between retries (default=2).
As an admin user, you can also add a filter that directs builds to
a particular cell. The policy.json
file must
have a line with "cells_scheduler_filter:TargetCellFilter"
: "is_admin:True"
to let an admin user specify a
scheduler hint to direct a build to a particular cell.
Cells currently keeps all inter-cell communication data, including
user names and passwords, in the database. This is undesirable and
unnecessary since cells data isn't updated very frequently. Instead,
create a JSON file to input cells data specified via a
[cells]cells_config
option. When specified, the
database is no longer consulted when reloading cells data. The file
will need the columns present in the Cell model (excluding common
database fields and the id
column). The queue
connection information must be specified through a
transport_url
field, instead of
username
, password
, and so on. The
transport_url
has the following form:
rabbit://USERNAME
:PASSWORD
@HOSTNAME
:PORT
/VIRTUAL_HOST
The scheme can be either qpid
or rabbit
, as shown previously. The following sample shows this optional configuration:
{ "parent": { "name": "parent", "api_url": "http://api.example.com:8774", "transport_url": "rabbit://rabbit.example.com", "weight_offset": 0.0, "weight_scale": 1.0, "is_parent": true }, "cell1": { "name": "cell1", "api_url": "http://api.example.com:8774", "transport_url": "rabbit://rabbit1.example.com", "weight_offset": 0.0, "weight_scale": 1.0, "is_parent": false }, "cell2": { "name": "cell2", "api_url": "http://api.example.com:8774", "transport_url": "rabbit://rabbit2.example.com", "weight_offset": 0.0, "weight_scale": 1.0, "is_parent": false } }