Cells¶

Status¶

Cells v1 is considered experimental and receives much less testing than the rest of Nova. For example, there is no job for testing cells v1 with Neutron.

The priority for the core team is implementation of and migration to cells v2. Because of this, there are a few restrictions placed on cells v1:

1. Cells v1 is in feature freeze. This means no new feature proposals for cells v1 will be accepted by the core team, which includes but is not limited to API parity, e.g. supporting virtual interface attach/detach with Neutron.
2. Latent bugs caused by the cells v1 design will not be fixed, e.g. bug 1489581. So if new tests are added to Tempest which trigger a latent bug in cells v1 it may not be fixed. However, regressions in working function should be tracked with bugs and fixed.

Suffice it to say, new deployments of cells v1 are not encouraged.

The restrictions above are basically meant to prioritize effort and focus on getting cells v2 completed, and feature requests and hard to fix latent bugs detract from that effort. Further discussion on this can be found in the 2015/11/12 Nova meeting minutes.

There are no plans to remove Cells V1 until V2 is usable by existing deployments and there is a migration path.

Manifesto¶

Database split¶

As mentioned above there is a split between global data and data that is local to a cell.

The following is a breakdown of what data can uncontroversially considered global versus local to a cell. Missing data will be filled in as consensus is reached on the data that is more difficult to cleanly place. The missing data is mostly concerned with scheduling and networking.

Overview¶

As more of the CellsV2 implementation is finished, all operators are required to make changes to their deployment. For all deployments (even those that only intend to have one cell), these changes are configuration-related, both in the main nova configuration file as well as some extra records in the databases.

All nova deployments must now have the following databases available and configured:

1. The “API” database
2. One special “cell” database called “cell0”
3. One (or eventually more) “cell” databases

Thus, a small nova deployment will have an API database, a cell0, and what we will call here a “cell1” database. High-level tracking information is kept in the API database. Instances that are never scheduled are relegated to the cell0 database, which is effectively a graveyard of instances that failed to start. All successful/running instances are stored in “cell1”.

First Time Setup¶

Since there is only one API database, the connection information for it is stored in the nova.conf file.

[api_database]
connection = mysql+pymysql://root:secretmysql@dbserver/nova_api?charset=utf8

Since there may be multiple “cell” databases (and in fact everyone will have cell0 and cell1 at a minimum), connection info for these is stored in the API database. Thus, you must have connection information in your config file for the API database before continuing to the steps below, so that nova-manage can find your other databases.

The following examples show the full expanded command line usage of the setup commands. This is to make it easier to visualize which of the various URLs are used by each of the commands. However, you should be able to put all of that in the config file and nova-manage will use those values. If need be, you can create separate config files and pass them as nova-manage –config-file foo.conf to control the behavior without specifying things on the command lines.

The commands below use the API database so remember to run nova-manage api_db sync first.

First we will create the necessary records for the cell0 database. To do that we use nova-manage like this:

nova-manage cell_v2 map_cell0 --database_connection \
  mysql+pymysql://root:secretmysql@dbserver/nova_cell0?charset=utf8

Since no hosts are ever in cell0, nothing further is required for its setup. Note that all deployments only ever have one cell0, as it is special, so once you have done this step you never need to do it again, even if you add more regular cells.

Now, we must create another cell which will be our first “regular” cell, which has actual compute hosts in it, and to which instances can actually be scheduled. First, we create the cell record like this:

nova-manage cell_v2 create_cell --verbose --name cell1 \
  --database_connection  mysql+pymysql://root:secretmysql@127.0.0.1/nova?charset=utf8
  --transport-url rabbit://stackrabbit:secretrabbit@mqserver:5672/

The nova-manage cell_v2 create_cell command will print the UUID of the newly-created cell if –verbose is passed, which is useful if you need to run commands like discover_hosts targeted at a specific cell.

Now we have a cell, but no hosts are in it which means the scheduler will never actually place instances there. The next step is to scan the database for compute node records and add them into the cell we just created. For this step, you must have had a compute node started such that it registers itself as a running service. Once that has happened, you can scan and add it to the cell:

nova-manage cell_v2 discover_hosts

This command will connect to any databases for which you have created cells (as above), look for hosts that have registered themselves there, and map those hosts in the API database so that they are visible to the scheduler as available targets for instances. Any time you add more compute hosts to a cell, you need to re-run this command to map them from the top-level so they can be utilized.

Template URLs in Cell Mappings¶

Starting in the Rocky release, the URLs provided in the cell mappings for --database_connection and --transport-url can contain variables which are evaluated each time they are loaded from the database, and the values of which are taken from the corresponding base options in the host’s configuration file. The base URL is parsed and the following elements may be substituted into the cell mapping URL (using rabbit://bob:s3kret@myhost:123/nova?sync=true#extra):

Variables are provided in curly brackets, like {username}. A simple template of rabbit://{username}:{password}@otherhost/{path} will generate a full URL of rabbit://bob:s3kret@otherhost/nova when used with the above example.

The templating of these URLs may be helpful in order to provide each service host with its own credentials for, say, the database. Without templating, all hosts will use the same URL (and thus credentials) for accessing services like the database and message queue. By using a URL with a template that results in the credentials being taken from the host-local configuration file, each host will use different values for those connections.

Assuming you have two service hosts that are normally configured with the cell0 database as their primary connection, their (abbreviated) configurations would look like this:

[database]
connection = mysql+pymysql://service1:foo@myapidbhost/nova_cell0

and:

[database]
connection = mysql+pymysql://service2:bar@myapidbhost/nova_cell0

Without cell mapping template URLs, they would still use the same credentials (as stored in the mapping) to connect to the cell databases. However, consider template URLs like the following:

mysql+pymysql://{username}:{password}@mycell1dbhost/nova

and:

mysql+pymysql://{username}:{password}@mycell2dbhost/nova

Using the first service and cell1 mapping, the calculated URL that will actually be used for connecting to that database will be:

mysql+pymysql://service1:foo@mycell1dbhost/nova

References¶

• nova-manage man page

Fresh Install¶

You are installing Nova for the first time and have no compute hosts in the database yet. This will set up a single cell Nova deployment.

1. Reminder: You should have already created and synced the Nova API database by creating a database, configuring its connection in the [api_database]/connection setting in the Nova configuration file, and running nova-manage api_db sync.

2. Create a database for cell0. If you are going to pass the database connection url on the command line in step 3, you can name the cell0 database whatever you want. If you are not going to pass the database url on the command line in step 3, you need to name the cell0 database based on the name of your existing Nova database: <Nova database name>_cell0. For example, if your Nova database is named nova, then your cell0 database should be named nova_cell0.

3. Run the map_cell0 command to create and map cell0:

   nova-manage cell_v2 map_cell0 \
     --database_connection <database connection url>
   

   The database connection url is generated based on the [database]/connection setting in the Nova configuration file if not specified on the command line.

4. Run nova-manage db sync to populate the cell0 database with a schema. The db sync command reads the database connection for cell0 that was created in step 3.

5. Run the create_cell command to create the single cell which will contain your compute hosts:

   nova-manage cell_v2 create_cell --name <name> \
     --transport-url <transport url for message queue> \
     --database_connection <database connection url>
   

   The transport url is taken from the [DEFAULT]/transport_url setting in the Nova configuration file if not specified on the command line. The database url is taken from the [database]/connection setting in the Nova configuration file if not specified on the command line.

6. Configure your compute host(s), making sure [DEFAULT]/transport_url matches the transport URL for the cell created in step 5, and start the nova-compute service. Before step 7, make sure you have compute hosts in the database by running:

   nova service-list --binary nova-compute
   

7. Run the discover_hosts command to map compute hosts to the single cell by running:

   nova-manage cell_v2 discover_hosts
   

   The command will search for compute hosts in the database of the cell you created in step 5 and map them to the cell. You can also configure a periodic task to have Nova discover new hosts automatically by setting the [scheduler]/discover_hosts_in_cells_interval to a time interval in seconds. The periodic task is run by the nova-scheduler service, so you must be sure to configure it on all of your nova-scheduler hosts.

Upgrade (minimal)¶

You are upgrading an existing Nova install and have compute hosts in the database. This will set up a single cell Nova deployment.

1. If you haven’t already created a cell0 database in a prior release, create a database for cell0 with a name based on the name of your Nova database: <Nova database name>_cell0. If your Nova database is named nova, then your cell0 database should be named nova_cell0.

2. Run the simple_cell_setup command to create and map cell0, create and map the single cell, and map existing compute hosts and instances to the single cell:

   nova-manage cell_v2 simple_cell_setup \
     --transport-url <transport url for message queue>
   

   The transport url is taken from the [DEFAULT]/transport_url setting in the Nova configuration file if not specified on the command line. The database connection url will be generated based on the [database]/connection setting in the Nova configuration file.

Upgrade with Cells V1¶

You are upgrading an existing Nova install that has Cells V1 enabled and have compute hosts in your databases. This will set up a multiple cell Nova deployment. At this time, it is recommended to keep Cells V1 enabled during and after the upgrade as multiple Cells V2 cell support is not fully finished and may not work properly in all scenarios. These upgrade steps will help ensure a simple cutover from Cells V1 to Cells V2 in the future.

1. If you haven’t already created a cell0 database in a prior release, create a database for cell0. If you are going to pass the database connection url on the command line in step 2, you can name the cell0 database whatever you want. If you are not going to pass the database url on the command line in step 2, you need to name the cell0 database based on the name of your existing Nova database: <Nova database name>_cell0. For example, if your Nova database is named nova, then your cell0 database should be named nova_cell0.

2. Run the map_cell0 command to create and map cell0:

   nova-manage cell_v2 map_cell0 \
     --database_connection <database connection url>
   

   The database connection url is generated based on the [database]/connection setting in the Nova configuration file if not specified on the command line.

3. Run nova-manage db sync to populate the cell0 database with a schema. The db sync command reads the database connection for cell0 that was created in step 2.

4. Run the create_cell command to create cells which will contain your compute hosts:

   nova-manage cell_v2 create_cell --name <cell name> \
     --transport-url <transport url for message queue> \
     --database_connection <database connection url>
   

   You will need to repeat this step for each cell in your deployment. Your existing cell database will be re-used – this simply informs the top-level API database about your existing cell databases.

   It is a good idea to specify a name for the new cell you create so you can easily look up cell uuids with the list_cells command later if needed.

   The transport url is taken from the [DEFAULT]/transport_url setting in the Nova configuration file if not specified on the command line. The database url is taken from the [database]/connection setting in the Nova configuration file if not specified on the command line. If you are not going to specify --database_connection and --transport-url on the command line, be sure to specify your existing cell Nova configuration file:

   nova-manage --config-file <cell nova.conf> cell_v2 create_cell \
     --name <cell name>
   

5. Run the discover_hosts command to map compute hosts to cells:

   nova-manage cell_v2 discover_hosts --cell_uuid <cell uuid>
   

   You will need to repeat this step for each cell in your deployment unless you omit the --cell_uuid option. If the cell uuid is not specified on the command line, discover_hosts will search for compute hosts in each cell database and map them to the corresponding cell. You can use the list_cells command to look up cell uuids if you are going to specify --cell_uuid.

   You can also configure a periodic task to have Nova discover new hosts automatically by setting the [scheduler]/discover_hosts_in_cells_interval to a time interval in seconds. The periodic task is run by the nova-scheduler service, so you must be sure to configure it on all of your nova-scheduler hosts.

6. Run the map_instances command to map instances to cells:

   nova-manage cell_v2 map_instances --cell_uuid <cell uuid> \
     --max-count <max count>
   

   You will need to repeat this step for each cell in your deployment. You can use the list_cells command to look up cell uuids.

   The --max-count option can be specified if you would like to limit the number of instances to map in a single run. If --max-count is not specified, all instances will be mapped. Repeated runs of the command will start from where the last run finished so it is not necessary to increase --max-count to finish. An exit code of 0 indicates that all instances have been mapped. An exit code of 1 indicates that there are remaining instances that need to be mapped.

Adding a new cell to an existing deployment¶

To expand your deployment with a new cell, first follow the usual steps for standing up a new Cells V1 cell. After that is finished, follow step 4 in Upgrade with Cells V1 to create a new Cells V2 cell for it. If you have added new compute hosts for the new cell, you will also need to run the discover_hosts command after starting them to map them to the new cell if you did not configure the automatic host discovery as described in step 5 in Upgrade with Cells V1.

References¶

• nova-manage man page