AMD SEV (Secure Encrypted Virtualization)¶
New in version 20.0.0: (Train)
Secure Encrypted Virtualization (SEV) is a technology from AMD which enables the memory for a VM to be encrypted with a key unique to the VM. SEV is particularly applicable to cloud computing since it can reduce the amount of trust VMs need to place in the hypervisor and administrator of their host system.
Enabling SEV¶
First the operator will need to ensure the following prerequisites are met:
Currently SEV is only supported when using the libvirt compute driver with a
libvirt.virt_type
ofkvm
orqemu
.At least one of the Nova compute hosts must be AMD hardware capable of supporting SEV. It is entirely possible for the compute plane to be a mix of hardware which can and cannot support SEV, although as per the section on Permanent limitations below, the maximum number of simultaneously running guests with SEV will be limited by the quantity and quality of SEV-capable hardware available.
In order for users to be able to use SEV, the operator will need to perform the following steps:
Ensure that sufficient memory is reserved on the SEV compute hosts for host-level services to function correctly at all times. This is particularly important when hosting SEV-enabled guests, since they pin pages in RAM, preventing any memory overcommit which may be in normal operation on other compute hosts.
It is recommended to achieve this by configuring an
rlimit
at the/machine.slice
top-levelcgroup
on the host, with all VMs placed inside that. (For extreme detail, see this discussion on the spec.)An alternative approach is to configure the
reserved_host_memory_mb
option in the[DEFAULT]
section ofnova.conf
, based on the expected maximum number of SEV guests simultaneously running on the host, and the details provided in an earlier version of the AMD SEV spec regarding memory region sizes, which cover how to calculate it correctly.See the Memory Locking and Accounting section of the AMD SEV spec and previous discussion for further details.
A cloud administrator will need to define one or more SEV-enabled flavors as described below, unless it is sufficient for users to define SEV-enabled images.
Additionally the cloud operator should consider the following optional steps:
Configure the
libvirt.num_memory_encrypted_guests
option innova.conf
to represent the number of guests an SEV compute node can host concurrently with memory encrypted at the hardware level. For example:[libvirt] num_memory_encrypted_guests = 15
This option exists because on AMD SEV-capable hardware, the memory controller has a fixed number of slots for holding encryption keys, one per guest. For example, at the time of writing, earlier generations of hardware only have 15 slots, thereby limiting the number of SEV guests which can be run concurrently to 15. Nova needs to track how many slots are available and used in order to avoid attempting to exceed that limit in the hardware.
At the time of writing (September 2019), work is in progress to allow QEMU and libvirt to expose the number of slots available on SEV hardware; however until this is finished and released, it will not be possible for Nova to programmatically detect the correct value.
So this configuration option serves as a stop-gap, allowing the cloud operator the option of providing this value manually. It may later be demoted to a fallback value for cases where the limit cannot be detected programmatically, or even removed altogether when Nova’s minimum QEMU version guarantees that it can always be detected.
Note
When deciding whether to use the default of
None
or manually impose a limit, operators should carefully weigh the benefits vs. the risk. The benefits of using the default are a) immediate convenience since nothing needs to be done now, and b) convenience later when upgrading compute hosts to future versions of Nova, since again nothing will need to be done for the correct limit to be automatically imposed. However the risk is that until auto-detection is implemented, users may be able to attempt to launch guests with encrypted memory on hosts which have already reached the maximum number of guests simultaneously running with encrypted memory. This risk may be mitigated by other limitations which operators can impose, for example if the smallest RAM footprint of any flavor imposes a maximum number of simultaneously running guests which is less than or equal to the SEV limit.Configure
ram_allocation_ratio
on all SEV-capable compute hosts to1.0
. Use of SEV requires locking guest memory, meaning it is not possible to overcommit host memory.Alternatively, you can explicitly configure small pages for instances using the
hw:mem_page_size
flavor extra spec and equivalent image metadata property. For more information, see Huge pages.Configure
libvirt.hw_machine_type
on all SEV-capable compute hosts to includex86_64=q35
, so that all x86_64 images use theq35
machine type by default. (Currently Nova defaults to thepc
machine type for thex86_64
architecture, although it is expected that this will change in the future.)Changing the default from
pc
toq35
makes the creation and configuration of images by users more convenient by removing the need for thehw_machine_type
property to be set toq35
on every image for which SEV booting is desired.Caution
Consider carefully whether to set this option. It is particularly important since a limitation of the implementation prevents the user from receiving an error message with a helpful explanation if they try to boot an SEV guest when neither this configuration option nor the image property are set to select a
q35
machine type.On the other hand, setting it to
q35
may have other undesirable side-effects on other images which were expecting to be booted withpc
, so it is suggested to set it on a single compute node or aggregate, and perform careful testing of typical images before rolling out the setting to all SEV-capable compute hosts.
Configuring a flavor or image¶
Once an operator has covered the above steps, users can launch SEV
instances either by requesting a flavor for which the operator set the
hw:mem_encryption
extra spec to True
, or by using an
image with the hw_mem_encryption
property set to True
. For example, to
enable SEV for a flavor:
$ openstack flavor set FLAVOR-NAME \
--property hw:mem_encryption=true
These do not inherently cause a preference for SEV-capable hardware,
but for now SEV is the only way of fulfilling the requirement for
memory encryption. However in the future, support for other
hardware-level guest memory encryption technology such as Intel MKTME
may be added. If a guest specifically needs to be booted using SEV
rather than any other memory encryption technology, it is possible to
ensure this by setting the trait{group}:HW_CPU_X86_AMD_SEV
extra spec or equivalent image metadata property to required
.
In all cases, SEV instances can only be booted from images which have
the hw_firmware_type
property set to uefi
, and only when the
machine type is set to q35
. This can be set per image by setting
the image property hw_machine_type=q35
, or per compute node by
the operator via libvirt.hw_machine_type
as
explained above.
Limitations¶
Impermanent limitations¶
The following limitations may be removed in the future as the hardware, firmware, and various layers of software receive new features:
SEV-encrypted VMs cannot yet be live-migrated or suspended, therefore they will need to be fully shut down before migrating off an SEV host, e.g. if maintenance is required on the host.
SEV-encrypted VMs cannot contain directly accessible host devices (PCI passthrough). So for example mdev vGPU support will not currently work. However technologies based on vhost-user should work fine.
The boot disk of SEV-encrypted VMs can only be
virtio
. (virtio-blk
is typically the default for libvirt disks on x86, but can also be explicitly set e.g. via the image propertyhw_disk_bus=virtio
). Valid alternatives for the disk include usinghw_disk_bus=scsi
withhw_scsi_model=virtio-scsi
, orhw_disk_bus=sata
.QEMU and libvirt cannot yet expose the number of slots available for encrypted guests in the memory controller on SEV hardware. Until this is implemented, it is not possible for Nova to programmatically detect the correct value. As a short-term workaround, operators can optionally manually specify the upper limit of SEV guests for each compute host, via the new
libvirt.num_memory_encrypted_guests
configuration option described above.
Permanent limitations¶
The following limitations are expected long-term:
The number of SEV guests allowed to run concurrently will always be limited. On the first generation of EPYC machines it will be limited to 15 guests; however this limit becomes much higher with the second generation (Rome).
The operating system running in an encrypted virtual machine must contain SEV support.
Non-limitations¶
For the sake of eliminating any doubt, the following actions are not expected to be limited when SEV encryption is used:
Cold migration or shelve, since they power off the VM before the operation at which point there is no encrypted memory (although this could change since there is work underway to add support for PMEM)
Snapshot, since it only snapshots the disk
nova evacuate
(despite the name, more akin to resurrection than evacuation), since this is only initiated when the VM is no longer runningAttaching any volumes, as long as they do not require attaching via an IDE bus
Use of spice / VNC / serial / RDP consoles