TLS¶
This guide describes how to configure Kolla Ansible to deploy OpenStack with TLS enabled. Enabling TLS on the provided internal and/or external VIP address allows OpenStack clients to authenticate and encrypt network communication with OpenStack services.
When an OpenStack service exposes an API endpoint, Kolla Ansible will configure HAProxy for that service to listen on the internal and/or external VIP address. The HAProxy container load-balances requests on the VIPs to the nodes running the service container.
There are two different layers of TLS configuration for OpenStack APIs:
Enabling TLS on the internal and/or external VIP, so communication between an OpenStack client and the HAProxy listening on the VIP is secure.
Enabling TLS on the backend network, so communication between HAProxy and the backend API services is secure.
Note
TLS authentication is based on certificates that have been signed by trusted Certificate Authorities. Examples of commercial CAs are Comodo, Symantec, GoDaddy, and GlobalSign. Letsencrypt.org is a CA that will provide trusted certificates at no charge. If using a trusted CA is not possible for your project, you can use a private CA, e.g. Hashicorp Vault, to create a certificate for your domain, or see Generating a Private Certificate Authority to use a Kolla Ansible generated private CA.
For details on ACME-enabled CAs, such as letsencrypt.org, please see ACME http-01 challenge support.
Quick Start¶
Note
The certificates generated by Kolla Ansible use a simple Certificate Authority setup and are not suitable for a production deployment. Only certificates signed by a trusted Certificate Authority should be used in a production deployment.
To deploy OpenStack with TLS enabled for the external, internal and backend
APIs, configure the following in globals.yml:
kolla_enable_tls_internal: "yes"
kolla_enable_tls_external: "yes"
kolla_enable_tls_backend: "yes"
kolla_copy_ca_into_containers: "yes"
If deploying on Debian or Ubuntu:
openstack_cacert: "/etc/ssl/certs/ca-certificates.crt"
If on CentOS or RHEL:
openstack_cacert: "/etc/pki/tls/certs/ca-bundle.crt"
The Kolla Ansible certificates command generates a private test Certificate
Authority, and then uses the CA to sign the generated certificates for the
enabled VIP(s) to test TLS in your OpenStack deployment. Assuming you are
using the multinode inventory:
kolla-ansible -i ~/multinode certificates
TLS Configuration for internal/external VIP¶
The configuration variables that control TLS for the internal and/or external VIP are:
kolla_enable_tls_externalkolla_enable_tls_internalkolla_internal_fqdn_certkolla_external_fqdn_cert
Note
If TLS is enabled only on the internal or external network, then
kolla_internal_vip_address and kolla_external_vip_address must
be different.
If there is only a single network configured in your topology (as opposed to separate internal and external networks), TLS can only be enabled using the internal network configuration variables.
The default state for TLS networking is disabled. To enable external TLS encryption:
kolla_enable_tls_external: "yes"
To enable internal TLS encryption:
kolla_enable_tls_internal: "yes"
Two certificate files are required to use TLS securely with authentication, which will be provided by your Certificate Authority:
server certificate with private key
CA certificate with any intermediate certificates
The combined server certificate and private key needs to be provided to Kolla
Ansible, with the path configured via kolla_external_fqdn_cert or
kolla_internal_fqdn_cert. These paths default to {{
kolla_certificates_dir }}/haproxy.pem and {{ kolla_certificates_dir
}}/haproxy-internal.pem respectively, where kolla_certificates_dir is
/etc/kolla/certificates by default.
If the server certificate provided is not already trusted by clients, then the CA certificate file will need to be distributed to the clients. This is discussed in more detail in Configuring the OpenStack Client for TLS and Adding CA Certificates to the Service Containers.
Configuring the OpenStack Client for TLS¶
The location for the CA certificate for the admin-openrc.sh file is
configured with the kolla_admin_openrc_cacert variable, which is not set by
default. This must be a valid path on all hosts where admin-openrc.sh is
used.
When TLS is enabled on a VIP, and kolla_admin_openrc_cacert is set to
/etc/pki/tls/certs/ca-bundle.crt, an OpenStack client will have settings
similar to this configured by admin-openrc.sh:
export OS_PROJECT_DOMAIN_NAME=Default
export OS_USER_DOMAIN_NAME=Default
export OS_PROJECT_NAME=admin
export OS_TENANT_NAME=admin
export OS_USERNAME=admin
export OS_PASSWORD=demoPassword
export OS_AUTH_URL=https://mykolla.example.net:5000
export OS_INTERFACE=internal
export OS_ENDPOINT_TYPE=internalURL
export OS_MISTRAL_ENDPOINT_TYPE=internalURL
export OS_IDENTITY_API_VERSION=3
export OS_REGION_NAME=RegionOne
export OS_AUTH_PLUGIN=password
# os_cacert is optional for trusted certificates
export OS_CACERT=/etc/pki/tls/certs/ca-bundle.crt
Adding CA Certificates to the Service Containers¶
To copy CA certificate files to the service containers:
kolla_copy_ca_into_containers: "yes"
When kolla_copy_ca_into_containers is configured to “yes”, the CA
certificate files in /etc/kolla/certificates/ca will be copied into
service containers to enable trust for those CA certificates. This is required
for any certificates that are either self-signed or signed by a private CA,
and are not already present in the service image trust store. Kolla will
install these certificates in the container system wide trust store when the
container starts.
All certificate file names will have the kolla-customca- prefix prepended
to them when they are copied into the containers. For example, if a certificate
file is named internal.crt, it will be named
kolla-customca-internal.crt in the containers.
For Debian and Ubuntu containers, the certificate files will be copied to the
/usr/local/share/ca-certificates/ directory.
For CentOS and RHEL containers, the certificate files will be copied to the
/etc/pki/ca-trust/source/anchors/ directory.
In both cases, valid certificates will be added to the system trust store -
/etc/ssl/certs/ca-certificates.crt on Debian and Ubuntu, and
/etc/pki/tls/certs/ca-bundle.crt on CentOS and RHEL.
Configuring a CA bundle¶
OpenStack services do not always trust CA certificates from the system trust
store by default. To resolve this, the openstack_cacert variable should be
configured with the path to the CA Certificate in the container.
To use the system trust store on Debian or Ubuntu:
openstack_cacert: /etc/ssl/certs/ca-certificates.crt
For CentOS or RHEL:
openstack_cacert: /etc/pki/tls/certs/ca-bundle.crt
Back-end TLS Configuration¶
Enabling TLS on the backend services secures communication between the HAProxy listing on the internal/external VIP and the OpenStack services. It also enables secure end-to-end communication between OpenStack services that support TLS termination. The OpenStack services that support backend TLS termination in Victoria are: Nova, Ironic, Neutron, Keystone, Glance, Heat, Placement, Horizon, Barbican, and Cinder.
The configuration variables that control back-end TLS for service endpoints are:
kolla_enable_tls_backendkolla_tls_backend_certkolla_tls_backend_keyhaproxy_backend_cacerthaproxy_backend_cacert_dir
The default state for back-end TLS is disabled. To enable TLS for the back-end communication:
kolla_enable_tls_backend: "yes"
It is also possible to enable back-end TLS on a per-service basis. For example,
to enable back-end TLS for Keystone, set keystone_enable_tls_backend to
yes.
The default values for haproxy_backend_cacert and
haproxy_backend_cacert_dir should suffice if the certificate is in the
system trust store. Otherwise, they should be configured to a location of the
CA certificate installed in the service containers.
Each backend service requires a certificate and private key. In many cases it is necessary to use a separate certificate and key for each host, or even per-service. The following precedence is used for the certificate:
{{ kolla_certificates_dir }}/{{ inventory_hostname }}/{{ project_name }}-cert.pem{{ kolla_certificates_dir }}/{{ inventory_hostname }}-cert.pem{{ kolla_certificates_dir }}/{{ project_name }}-cert.pem{{ kolla_tls_backend_cert }}
And for the private key:
{{ kolla_certificates_dir }}/{{ inventory_hostname }}/{{ project_name }}-key.pem{{ kolla_certificates_dir }}/{{ inventory_hostname }}-key.pem{{ kolla_certificates_dir }}/{{ project_name }}-key.pem{{ kolla_tls_backend_key }}
The default for kolla_certificates_dir is /etc/kolla/certificates.
kolla_tls_backend_cert and kolla_tls_backend_key, default to {{
kolla_certificates_dir }}/backend-cert.pem and {{ kolla_certificates_dir
}}/backend-key.pem respectively.
project_name is the name of the OpenStack service, e.g. keystone or
cinder.
Note
The back-end TLS cert/key can be the same certificate that is used for the VIP, as long as those certificates are configured to allow requests from both the VIP and internal networks.
By default, the TLS certificate will be verified as trustable by the OpenStack services. Although not recommended for production, it is possible to disable verification of the backend certificate:
kolla_verify_tls_backend: "no"
Generating a Private Certificate Authority¶
Note
The certificates generated by Kolla Ansible use a simple Certificate Authority setup and are not suitable for a production deployment. Only certificates signed by a trusted Certificate Authority should be used in a production deployment.
It’s not always practical to get a certificate signed by a trusted CA. In a development or internal test OpenStack deployment, it can be useful to generate certificates locally to enable TLS.
For convenience, the kolla-ansible command will generate the necessary
certificate files based on the information in the globals.yml
configuration file and the inventory file:
kolla-ansible -i multinode certificates
The certificates role performs the following actions:
Generates a test root Certificate Authority
Generates the internal/external certificates which are signed by the root CA.
If back-end TLS is enabled, generate the back-end certificate signed by the root CA.
The combined certificate and key file haproxy.pem (which is the default
value for kolla_external_fqdn_cert) will be generated and stored in the
/etc/kolla/certificates/ directory, and a copy of the CA certificate
(root.crt) will be stored in the /etc/kolla/certificates/ca/ directory.
Generating your certificates without kolla-ansible¶
If you want to manage your TLS certificates outside kolla-ansible directly on
your hosts, you can do it by setting kolla_externally_managed_cert to
true. This will make kolla-ansible ignore any copy of certificate from
the operator to kolla-ansible managed hosts and will keep other configuration
options for TLS as is.
If using this option, make sure that all certificates are present on the appropriate hosts in the appropriate location.
