Real Time¶
New in version 13.0.0: (Mitaka)
Nova supports configuring real-time policies for instances. This builds upon the improved performance offered by CPU pinning by providing stronger guarantees for worst case scheduler latency for vCPUs.
Enabling Real-Time¶
Currently the creation of real-time instances is only supported when using the
libvirt compute driver with a libvirt.virt_type
of
kvm
or qemu
. It requires extensive configuration of the host and this
document provides but a rough overview of the changes required. Configuration
will vary depending on your hardware, BIOS configuration, host and guest OS’
and application.
BIOS configuration¶
Configure your host BIOS as recommended in the rt-wiki page. The most important steps are:
Disable power management, including CPU sleep states
Disable SMT (hyper-threading) or any option related to logical processors
These are standard steps used in benchmarking as both sets of features can result in non-deterministic behavior.
OS configuration¶
This is inherently specific to the distro used, however, there are some common steps:
Install the real-time (preemptible) kernel (
PREEMPT_RT_FULL
) and real-time KVM modulesConfigure hugepages
Isolate host cores to be used for instances from the kernel
Disable features like CPU frequency scaling (e.g. P-States on Intel processors)
RHEL and RHEL-derived distros like CentOS provide packages in their
repositories to accomplish. The kernel-rt
and kernel-rt-kvm
packages will provide the real-time kernel and real-time KVM module,
respectively, while the tuned-profiles-realtime
package will provide
tuned profiles to configure the host for real-time workloads. You should
refer to your distro documentation for more information.
Configuring a flavor or image¶
Changed in version 22.0.0: (Victoria)
Previously, it was necessary to specify
hw:cpu_realtime_mask
when
hw:cpu_realtime
was set to true
.
Starting in Victoria, it is possible
to omit this when an emulator thread policy is configured using the
hw:emulator_threads_policy
extra spec, thus allowing all
guest cores to be be allocated as real-time cores.
Changed in version 22.0.0: (Victoria)
Previously, a leading caret was necessary when specifying the value for
hw:cpu_realtime_mask
and omitting it would be equivalent
to not setting the mask, resulting in a failure to spawn the instance.
Compared to configuring the host, configuring the guest is relatively trivial and merely requires a combination of flavor extra specs and image metadata properties, along with a suitable real-time guest OS.
Enable real-time by setting the hw:cpu_realtime
flavor extra
spec to true
. When this is configured, it is necessary to specify where
guest overhead processes should be scheduled to. This can be accomplished in
one of three ways. Firstly, the hw:cpu_realtime_mask
extra
spec or equivalent image metadata property can be used to indicate which guest
cores should be scheduled as real-time cores, leaving the remainder to be
scheduled as non-real-time cores and to handle overhead processes. For example,
to allocate the first two cores of an 8 core instance as the non-real-time
cores:
$ openstack flavor set $FLAVOR \
--property hw:cpu_realtime=yes \
--property hw:cpu_realtime_mask=2-7 # so 0,1 are non-real-time
In this configuration, any non-real-time cores configured will have an implicit
dedicated
CPU pinning policy applied. It is
possible to apply a shared
policy for these non-real-time cores by
specifying the mixed
CPU pinning policy via
the hw:cpu_policy
extra spec. This can be useful to increase
resource utilization of the host. For example:
$ openstack flavor set $FLAVOR \
--property hw:cpu_policy=mixed \
--property hw:cpu_realtime=yes \
--property hw:cpu_realtime_mask=2-7 # so 0,1 are non-real-time and unpinned
Finally, you can explicitly offload guest overhead processes to another
host core using the
hw:emulator_threads_policy
extra spec. For example:
$ openstack flavor set $FLAVOR \
--property hw:cpu_realtime=yes \
--property hw:emulator_thread_policy=share
Note
Emulator thread pinning requires additional host configuration. Refer to the documentation for more information.
In addition to configuring the instance CPUs, it is also likely that you will need to configure guest huge pages. For information on how to configure these, refer to the documentation