The encryption offered by nova’s
libvirt.live_migration_tunnelled
does not secure
all the different migration streams of a nova instance, namely: guest
RAM, device state, and disks (via NBD) when using non-shared storage.
Further, the “tunnelling via libvirtd” has inherent limitations: (a) it
cannot handle live migration of disks in a non-shared storage setup
(a.k.a. “block migration”); and (b) has a huge performance overhead and
latency, because it burns more CPU and memory bandwidth due to increased
number of data copies on both source and destination hosts.
To solve this existing limitation, QEMU and libvirt have gained (refer below for version details) support for “native TLS”, i.e. TLS built into QEMU. This will secure all data transports, including disks that are not on shared storage, without incurring the limitations of the “tunnelled via libvirtd” transport.
To take advantage of the “native TLS” support in QEMU and libvirt, nova
has introduced new configuration attribute
libvirt.live_migration_with_native_tls
.
Version requirement: This feature needs at least libvirt 4.4.0 and QEMU 2.11.
A pre-configured TLS environment—i.e. CA, server, and client certificates, their file permissions, et al—must be “correctly” configured (typically by an installer tool) on all relevant compute nodes. To simplify your PKI (Public Key Infrastructure) setup, use deployment tools that take care of handling all the certificate lifecycle management. For example, refer to the “TLS everywhere” guide from the TripleO project.
Password-less SSH setup for all relevant compute nodes.
On all relevant compute nodes, ensure the TLS-related config
attributes in /etc/libvirt/qemu.conf
are in place:
default_tls_x509_cert_dir = "/etc/pki/qemu"
default_tls_x509_verify = 1
If it is not already configured, modify /etc/sysconfig/libvirtd
on both (ComputeNode1 & ComputeNode2) to listen for TCP/IP
connections:
LIBVIRTD_ARGS="--listen"
Then, restart the libvirt daemon (also on both nodes):
$ systemctl restart libvirtd
Refer to the “Related information” section on a note about the
other TLS-related configuration attributes in
/etc/libvirt/qemu.conf
.
Assuming you have two compute hosts (ComputeNode1
, and
ComputeNode2
) run the virt-pki-validate tool (comes with
the libvirt-client
package on your Linux distribution) on both the
nodes to ensure all the necessary PKI files are configured are
configured:
[ComputeNode1]$ virt-pki-validate
Found /usr/bin/certtool
Found CA certificate /etc/pki/CA/cacert.pem for TLS Migration Test
Found client certificate /etc/pki/libvirt/clientcert.pem for ComputeNode1
Found client private key /etc/pki/libvirt/private/clientkey.pem
Found server certificate /etc/pki/libvirt/servercert.pem for ComputeNode1
Found server private key /etc/pki/libvirt/private/serverkey.pem
Make sure /etc/sysconfig/libvirtd is setup to listen to
TCP/IP connections and restart the libvirtd service
[ComputeNode2]$ virt-pki-validate
Found /usr/bin/certtool
Found CA certificate /etc/pki/CA/cacert.pem for TLS Migration Test
Found client certificate /etc/pki/libvirt/clientcert.pem for ComputeNode2
Found client private key /etc/pki/libvirt/private/clientkey.pem
Found server certificate /etc/pki/libvirt/servercert.pem for ComputeNode2
Found server private key /etc/pki/libvirt/private/serverkey.pem
Make sure /etc/sysconfig/libvirtd is setup to listen to
TCP/IP connections and restart the libvirtd service
On all relevant compute nodes, enable the
libvirt.live_migration_with_native_tls
configuration attribute and set the
libvirt.live_migration_scheme
configuration attribute to tls:
[libvirt]
live_migration_with_native_tls = true
live_migration_scheme = tls
Note
Setting both
libvirt.live_migration_with_native_tls
and
libvirt.live_migration_tunnelled
at the
same time is invalid (and disallowed).
Note
Not setting
libvirt.live_migration_scheme
to tls
will result in libvirt using the unencrypted TCP connection
without displaying any error or a warning in the logs.
And restart the nova-compute
service:
$ systemctl restart openstack-nova-compute
Now that all TLS-related configuration is in place, migrate guests
(with or without shared storage) from ComputeNode1
to
ComputeNode2
. Refer to the Live-migrate instances document
on details about live migration.
Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.