Placement API¶

References¶

The following specifications represent the stages of design and development of resource providers and the Placement service. Implementation details may have changed or be partially complete at this time.

• Generic Resource Pools
• Compute Node Inventory
• Resource Provider Allocations
• Resource Provider Base Models
• Nested Resource Providers
• Custom Resource Classes
• Scheduler Filters in DB
• Scheduler claiming resources to the Placement API
• The Traits API - Manage Traits with ResourceProvider
• Request Traits During Scheduling

Steps¶

1. Deploy the API service

At this time the placement API code is still in Nova alongside the compute REST API code (nova-api). So once you have upgraded nova-api to Newton you already have the placement API code, you just need to install the service. Nova provides a nova-placement-api WSGI script for running the service with Apache, nginx or other WSGI-capable web servers. Depending on what packaging solution is used to deploy OpenStack, the WSGI script may be in /usr/bin or /usr/local/bin.

nova-placement-api, as a standard WSGI script, provides a module level application attribute that most WSGI servers expect to find. This means it is possible to run it with lots of different servers, providing flexibility in the face of different deployment scenarios. Common scenarios include:

• apache2 with mod_wsgi
• apache2 with mod_proxy_uwsgi
• nginx with uwsgi
• nginx with gunicorn

In all of these scenarios the host, port and mounting path (or prefix) of the application is controlled in the web server's configuration, not in the configuration (nova.conf) of the placement application.

When placement was first added to DevStack it used the mod_wsgi style. Later it was updated to use mod_proxy_uwsgi. Looking at those changes can be useful for understanding the relevant options.

DevStack is configured to host placement at /placement on either the default port for http or for https (80 or 443) depending on whether TLS is being used. Using a default port is desirable.

By default, the placement application will get its configuration for settings such as the database connection URL from /etc/nova/nova.conf. The directory the configuration file will be found in can be changed by setting OS_PLACEMENT_CONFIG_DIR in the environment of the process that starts the application.

This document refrains from declaring a set of installation instructions for the placement service. This is because a major point of having a WSGI application is to make the deployment as flexible as possible. Because the placement API service is itself stateless (all state is in the database), it is possible to deploy as many servers as desired behind a load balancing solution for robust and simple scaling. If you familiarize yourself with installing generic WSGI applications (using the links in the common scenarios list, above), those techniques will be applicable here.

2. Synchronize the database

In the Newton release the Nova api database is the only deployment option for the placement API service and the resources it manages. After upgrading the nova-api service for Newton and running the nova-manage api_db sync command the placement tables will be created.

3. Create accounts and update the service catalog

Create a placement service user with an admin role in Keystone.

The placement API is a separate service and thus should be registered under a placement service type in the service catalog as that is what the resource tracker in the nova-compute node will use to look up the endpoint.

Devstack sets up the placement service on the default HTTP port (80) with a /placement prefix instead of using an independent port.

4. Configure and restart nova-compute services

The 14.0.0 Newton nova-compute service code will begin reporting resource provider inventory and usage information as soon as the placement API service is in place and can respond to requests via the endpoint registered in the service catalog.

nova.conf on the compute nodes must be updated in the [placement] group to contain credentials for making requests from nova-compute to the placement-api service.

Ocata (15.0.0)¶

• The nova-compute service will fail to start in Ocata unless the [placement] section of nova.conf on the compute is configured. As mentioned in the deployment steps above, the Placement service should be deployed by this point so the computes can register and start reporting inventory and allocation information. If the computes are deployed and configured before the Placement service, they will continue to try and reconnect in a loop so that you do not need to restart the nova-compute process to talk to the Placement service after the compute is properly configured.
• The nova.scheduler.filter_scheduler.FilterScheduler in Ocata will fallback to not using the Placement service as long as there are older nova-compute services running in the deployment. This allows for rolling upgrades of the computes to not affect scheduling for the FilterScheduler. However, the fallback mechanism will be removed in the 16.0.0 Pike release such that the scheduler will make decisions based on the Placement service and the resource providers (compute nodes) registered there. This means if the computes are not reporting into Placement by Pike, build requests will fail with NoValidHost errors.
• While the FilterScheduler technically depends on the Placement service in Ocata, if you deploy the Placement service after you upgrade the nova-scheduler service to Ocata and restart it, things will still work. The scheduler will gracefully handle the absence of the Placement service. However, once all computes are upgraded, the scheduler not being able to make requests to Placement will result in NoValidHost errors.
• It is currently possible to exclude the CoreFilter, RamFilter and DiskFilter from the list of enabled FilterScheduler filters such that scheduling decisions are not based on CPU, RAM or disk usage. Once all computes are reporting into the Placement service, however, and the FilterScheduler starts to use the Placement service for decisions, those excluded filters are ignored and the scheduler will make requests based on VCPU, MEMORY_MB and DISK_GB inventory. If you wish to effectively ignore that type of resource for placement decisions, you will need to adjust the corresponding cpu_allocation_ratio, ram_allocation_ratio, and/or disk_allocation_ratio configuration options to be very high values, e.g. 9999.0.
• Users of CellsV1 will need to deploy a placement per cell, matching the scope and cardinality of the regular nova-scheduler process.

Pike (16.0.0)¶

• The nova.scheduler.filter_scheduler.FilterScheduler in Pike will no longer fall back to not using the Placement Service, even if older computes are running in the deployment.

• The FilterScheduler now requests allocation candidates from the Placement service during scheduling. The allocation candidates information was introduced in the Placement API 1.10 microversion, so you should upgrade the placement service before the Nova scheduler service so that the scheduler can take advantage of the allocation candidate information.

  The scheduler gets the allocation candidates from the placement API and uses those to get the compute nodes, which come from the cell(s). The compute nodes are passed through the enabled scheduler filters and weighers. The scheduler then iterates over this filtered and weighed list of hosts and attempts to claim resources in the placement API for each instance in the request. Claiming resources involves finding an allocation candidate that contains an allocation against the selected host's UUID and asking the placement API to allocate the requested instance resources. We continue performing this claim request until success or we run out of allocation candidates, resulting in a NoValidHost error.

  For a move operation, such as migration, allocations are made in Placement against both the source and destination compute node. Once the move operation is complete, the resource tracker in the nova-compute service will adjust the allocations in Placement appropriately.

  For a resize to the same host, allocations are summed on the single compute node. This could pose a problem if the compute node has limited capacity. Since resizing to the same host is disabled by default, and generally only used in testing, this is mentioned for completeness but should not be a concern for production deployments.

Queens (17.0.0)¶

• The minimum Placement API microversion required by the nova-scheduler service is 1.17 in order to support Request Traits During Scheduling. This means you must upgrade the placement service before upgrading any nova-scheduler services to Queens.

Microversions¶

The placement API uses microversions for making incremental changes to the API which client requests must opt into.

It is especially important to keep in mind that nova-compute is a client of the placement REST API and based on how Nova supports rolling upgrades the nova-compute service could be Newton level code making requests to an Ocata placement API, and vice-versa, an Ocata compute service in a cells v2 cell could be making requests to a Newton placement API.

REST API Version History¶

This documents the changes made to the REST API with every microversion change. The description for each version should be a verbose one which has enough information to be suitable for use in user documentation.