This document contains several examples of using basic load balancing services as a tenant or “regular” cloud user.
For the purposes of this guide we assume that the neutron and barbican command-line interfaces are going to be used to configure all features of Neutron LBaaS with an Octavia back-end. In order to keep these examples short, we also assume that tasks not directly associated with deploying load balancing services have already been accomplished. This might include such things as deploying and configuring web servers, setting up Neutron networks, obtaining TLS certificates from a trusted provider, and so on. A description of the starting conditions is given in each example below.
Please also note that this guide assumes you are familiar with the specific load balancer terminology defined in the Octavia Glossary. For a description of load balancing itself and the Octavia project, please see: Introducing Octavia.
While this is technically the simplest complete load balancing solution that can be deployed, we recommend deploying HTTP load balancers with a health monitor to ensure back-end member availability. See Deploy a basic HTTP load balancer with a health monitor below.
Scenario description:
Solution:
CLI commands:
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --name listener1 --protocol HTTP --protocol-port 80 lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
This is the simplest recommended load balancing solution for HTTP applications. This solution is appropriate for operators with provider networks that are not compatible with Neutron floating-ip functionality (such as IPv6 networks). However, if you need to retain control of the external IP through which a load balancer is accessible, even if the load balancer needs to be destroyed or recreated, it may be more appropriate to deploy your basic load balancer using a floating IP. See Deploy a basic HTTP load balancer using a floating IP below.
Scenario description:
Solution:
CLI commands:
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --name listener1 --protocol HTTP --protocol-port 80 lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP
openstack loadbalancer healthmonitor create --delay 5 --max-retries 4 --timeout 10 --type HTTP --url-path /healthcheck pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
It can be beneficial to use a floating IP when setting up a load balancer’s VIP in order to ensure you retain control of the IP that gets assigned as the floating IP in case the load balancer needs to be destroyed, moved, or recreated.
Note that this is not possible to do with IPv6 load balancers as floating IPs do not work with IPv6. Further, there is currently a bug in Neutron Distributed Virtual Routing (DVR) which prevents floating IPs from working correctly when DVR is in use. See: https://bugs.launchpad.net/neutron/+bug/1583694
Scenario description:
Solution:
CLI commands:
openstack loadbalancer create --name lb1 --vip-subnet-id private-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --name listener1 --protocol HTTP --protocol-port 80 lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP
openstack loadbalancer healthmonitor create --delay 5 --max-retries 4 --timeout 10 --type HTTP --url-path /healthcheck pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
openstack floating ip create public
# The following IDs should be visible in the output of previous commands
openstack floating ip set --port <load_balancer_vip_port_id> <floating_ip_id>
Scenario description:
Solution:
CLI commands:
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --name listener1 --protocol HTTP --protocol-port 80 lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP --session-persistence type=APP_COOKIE,cookie_name=PHPSESSIONID
openstack loadbalancer healthmonitor create --delay 5 --max-retries 4 --timeout 10 --type HTTP --url-path /healthcheck pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
This is generally suitable when load balancing a non-HTTP TCP-based service.
Scenario description:
Solution:
CLI commands:
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --name listener1 --protocol TCP --protocol-port 23456 lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol TCP
openstack loadbalancer healthmonitor create --delay 5 --max-retries 4 --timeout 10 --type TCP pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
This solution limits the bandwidth available through the Load Balancer’s VIP by applying a Neutron Quality of Service(QoS) policy to the VIP, so Load Balancer can accept the QoS Policy from Neutron; Then limits the vip of Load Balancer incoming or outgoing traffic.
Note
Before using this feature, please make sure the Neutron QoS externsion(qos) is enabled on runing OpenStack environment by command
openstack extension list
Scenario description:
Solution:
CLI commands:
openstack network qos policy create qos-policy-bandwidth
openstack network qos rule create --type bandwidth_limit --max-kbps 1024 --max-burst-kbits 1024 qos-policy-bandwidth
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet --vip-qos-policy-id qos-policy-bandwidth
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --name listener1 lb1 --protocol HTTP --protocol-port 80
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP
openstack loadbalancer member create --subnet-id <private_subnet_id> --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id <private_subnet_id> --address 192.0.2.11 --protocol-port 80 pool1
A non-terminated HTTPS load balancer acts effectively like a generic TCP load balancer: The load balancer will forward the raw TCP traffic from the web client to the back-end servers without decrypting it. This means that the back-end servers themselves must be configured to terminate the HTTPS connection with the web clients, and in turn, the load balancer cannot insert headers into the HTTP session indicating the client IP address. (That is, to the back-end server, all web requests will appear to originate from the load balancer.) Also, advanced load balancer features (like Layer 7 functionality) cannot be used with non-terminated HTTPS.
Scenario description:
Solution:
CLI commands:
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --name listener1 --protocol HTTPS --protocol-port 443 lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTPS
openstack loadbalancer healthmonitor create --delay 5 --max-retries 4 --timeout 10 --type HTTPS --url-path /healthcheck pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 443 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 443 pool1
With a TLS-terminated HTTPS load balancer, web clients communicate with the load balancer over TLS protocols. The load balancer terminates the TLS session and forwards the decrypted requests to the back-end servers. By terminating the TLS session on the load balancer, we offload the CPU-intensive encryption work to the load balancer, and enable the possibility of using advanced load balancer features, like Layer 7 features and header manipulation.
Scenario description:
Solution:
CLI commands:
openssl pkcs12 -export -inkey server.key -in server.crt -certfile ca-chain.crt -passout pass: -out server.p12
openstack secret store --name='tls_secret1' -t 'application/octet-stream' -e 'base64' --payload="$(base64 < server.p12)"
openstack acl user add -u admin_id $(openstack secret list | awk '/ tls_secret1 / {print $2}')
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --protocol-port 443 --protocol TERMINATED_HTTPS --name listener1 --default-tls-container=$(openstack secret list | awk '/ tls_secret1 / {print $2}' lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
This example is exactly like Deploy a TLS-terminated HTTPS load balancer, except that we have multiple TLS certificates that we would like to use on the same listener using Server Name Indication (SNI) technology.
Scenario description:
Solution:
CLI commands:
openssl pkcs12 -export -inkey server.key -in server.crt -certfile ca-chain.crt -passout pass: -out server.p12
openssl pkcs12 -export -inkey server2.key -in server2.crt -certfile ca-chain2.crt -passout pass: -out server2.p12
openstack secret store --name='tls_secret1' -t 'application/octet-stream' -e 'base64' --payload="$(base64 < server.p12)"
openstack secret store --name='tls_secret2' -t 'application/octet-stream' -e 'base64' --payload="$(base64 < server2.p12)"
openstack acl user add -u admin_id $(openstack secret list | awk '/ tls_secret1 / {print $2}')
openstack acl user add -u admin_id $(openstack secret list | awk '/ tls_secret2 / {print $2}')
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --protocol-port 443 --protocol TERMINATED_HTTPS --name listener1 --default-tls-container=$(openstack secret list | awk '/ tls_secret1 / {print $2}' --sni-container_refs $(openstack secret list | awk '/ tls_secret1 / {print $2}') $(openstack secret list | awk '/ tls_secret2 / {print $2}') lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
This example is exactly like Deploy a TLS-terminated HTTPS load balancer, except that we would like to have both an HTTP and TERMINATED_HTTPS listener that use the same back-end pool (and therefore, probably respond with the exact same content regardless of whether the web client uses the HTTP or HTTPS protocol to connect).
Please note that if you wish all HTTP requests to be redirected to HTTPS (so that requests are only served via HTTPS, and attempts to access content over HTTP just get redirected to the HTTPS listener), then please see the example in the Layer 7 Cookbook.
Scenario description:
Solution:
CLI commands:
openssl pkcs12 -export -inkey server.key -in server.crt -certfile ca-chain.crt -passout pass: -out server.p12
openstack secret store --name='tls_secret1' -t 'application/octet-stream' -e 'base64' --payload="$(base64 < server.p12)"
openstack acl user add -u admin_id $(openstack secret list | awk '/ tls_secret1 / {print $2}')
openstack loadbalancer create --name lb1 --vip-subnet-id public-subnet
# Re-run the following until lb1 shows ACTIVE and ONLINE statuses:
openstack loadbalancer show lb1
openstack loadbalancer listener create --protocol-port 443 --protocol TERMINATED_HTTPS --name listener1 --default-tls-container=$(openstack secret list | awk '/ tls_secret1 / {print $2}' lb1
openstack loadbalancer pool create --name pool1 --lb-algorithm ROUND_ROBIN --listener listener1 --protocol HTTP
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.10 --protocol-port 80 pool1
openstack loadbalancer member create --subnet-id private-subnet --address 192.0.2.11 --protocol-port 80 pool1
openstack secret store --name='tls_secret1' --payload-content-type='text/plain' --payload="$(cat server.crt)"
openstack loadbalancer listener create --protocol-port 80 --protocol HTTP --name listener2 --default-pool pool1 lb1
While it is possible to set up a listener without a health monitor, if a back-end pool member goes down, Octavia will not remove the failed server from the pool until a considerable time has passed. This can lead to service disruption for web clients. Because of this, we recommend always configuring production load balancers to use a health monitor.
The health monitor itself is a process that does periodic health checks on each back-end server to pre-emptively detect failed servers and temporarily pull them out of the pool. Since effective health monitors depend as much on back-end application server configuration as proper load balancer configuration, some additional discussion of best practices is warranted here.
See also: Octavia API Reference
All of the health monitors Octavia supports have the following configurable options:
delay
: Number of seconds to wait between health checks.timeout
: Number of seconds to wait for any given health check to
complete. timeout
should always be smaller than delay
.max-retries
: Number of subsequent health checks a given back-end
server must fail before it is considered down, or that a failed back-end
server must pass to be considered up again.In general, the application-side component of HTTP health checks are a part of the web application being load balanced. By default, Octavia will probe the “/” path on the application server. However, in many applications this is not appropriate because the “/” path ends up being a cached page, or causes the application server to do more work than is necessary for a basic health check.
In addition to the above options, HTTP health monitors also have the following options:
url_path
: Path part of the URL that should be retrieved from the back-end
server. By default this is “/”.http_method
: HTTP method that should be used to retrieve the
url_path
. By default this is “GET”.expected_codes
: List of HTTP status codes that indicate an OK health
check. By default this is just “200”.Please keep the following best practices in mind when writing the code that generates the health check in your web application:
url_path
should not require authentication to load.url_path
should return a HTTP 200 OK status
code to indicate a healthy server unless you specify alternate
expected_codes
.url_path
would incorporate the results of this cron job in the tests it
performs.Other health monitor types include PING
, TCP
, HTTPS
, and
TLS-HELLO
.
PING
health monitors send periodic ICMP PING requests to the back-end
servers. Obviously, your back-end servers must be configured to allow PINGs in
order for these health checks to pass.
TCP
health monitors open a TCP connection to the back-end server’s protocol
port. Your custom TCP application should be written to respond OK to the load
balancer connecting, opening a TCP connection, and closing it again after the
TCP handshake without sending any data.
HTTPS
health monitors operate exactly like HTTP health monitors, but with
ssl back-end servers. Unfortunately, this causes problems if the servers are
performing client certificate validation, as HAProxy won’t have a valid cert.
In this case, using TLS-HELLO
type monitoring is an alternative.
TLS-HELLO
health monitors simply ensure the back-end server responds to
SSLv3 client hello messages. It will not check any other health metrics, like
status code or body contents.
Some TLS certificates require you to install an intermediate certificate chain in order for web client browsers to trust the certificate. This chain can take several forms, and is a file provided by the organization from whom you obtained your TLS certificate.
The simplest form of the intermediate chain is a PEM-encoded text file that
either contains a sequence of individually-encoded PEM certificates, or a PEM
encoded PKCS7 block(s). If this is the type of intermediate chain you have been
provided, the file will contain either -----BEGIN PKCS7-----
or
-----BEGIN CERTIFICATE-----
near the top of the file, and one or more
blocks of 64-character lines of ASCII text (that will look like gobbedlygook to
a human). These files are also typically named with a .crt
or .pem
extension.
If the intermediates chain provided to you is a file that contains what appears
to be random binary data, it is likely that it is a PKCS7 chain in DER format.
These files also may be named with a .p7b
extension.
You may use the binary DER file as-is when building your PKCS12 bundle:
openssl pkcs12 -export -inkey server.key -in server.crt -certfile ca-chain.p7b -passout pass: -out server.p12
… or you can convert it to a series of PEM-encoded certificates:
openssl pkcs7 -in intermediates-chain.p7b -inform DER -print_certs -out intermediates-chain.crt
… or you can convert it to a PEM-encoded PKCS7 bundle:
openssl pkcs7 -in intermediates-chain.p7b -inform DER -outform PEM -out intermediates-chain.crt
If the file is not a PKCS7 DER bundle, either of the two openssl pkcs7
commands will fail.
For examples of using Layer 7 features for more advanced load balancing, please see: Layer 7 Cookbook
Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.