Octavia HAProxy Amphora API¶
Introduction¶
This document describes the API interface between the reference haproxy driver and its corresponding haproxy-based amphorae.
Octavia reference haproxy amphorae use a web service API for configuration and control. This API should be secured through the use of TLS encryption as well as bi-directional verification of client- and server-side certificates. (The exact process for generating and distributing these certificates should be covered in another document.)
In addition to the web service configuration and control interface, the amphorae may use an HMAC-signed UDP protocol for communicating regular, less- vital information to the controller (ex. statistics updates and health checks). Information on this will also be covered in another document.
If a given loadbalancer is being serviced by multiple haproxy amphorae at the same time, configuration and control actions should be made on all these amphorae at approximately the same time. (Amphorae do not communicate directly with each other, except in an active-standby topology, and then this communication is limited to fail-over protocols.)
Versioning¶
All Octavia APIs (including internal APIs like this one) are versioned. For the purposes of this document, the initial version of this API shall be 1.0.
Response codes¶
Typical response codes are:
200 OK - Operation was completed as requested.
201 Created - Operation successfully resulted in the creation / processing of a file.
202 Accepted - Command was accepted but is not completed. (Note that this is used for asynchronous processing.)
400 Bad Request - API handler was unable to complete request.
401 Unauthorized - Authentication of the client certificate failed.
404 Not Found - The requested file was not found.
500 Internal Server Error - Usually indicates a permissions problem
503 Service Unavailable - Usually indicates a change to a listener was attempted during a transition of amphora topology.
A note about storing state¶
In the below API, it will become apparent that at times the amphora will need to be aware of the state of things (topology-wise, or simply in terms running processes on the amphora). When it comes to storing or gathering this data, we should generally prefer to try to resolve these concerns in the following order. Note also that not every kind of state data will use all of the steps in this list:
Get state information by querying running processes (ex. parsing haproxy status page or querying iptables counters, etc.)
Get state by consulting on-disk cache generated by querying running processes. (In the case where state information is relatively expensive to collect– eg. package version listings.)
Get state by consulting stored configuration data as sent by the controller. (ex. amphora topology, haproxy configuration or TLS certificate data)
Get state by querying a controller API (not described here).
In no case should the amphora assume it ever has direct access to the Octavia database. Also, sensitive data (like TLS certificates) should be stored in a secure way (ex. memory filesystem).
API¶
Get amphora info¶
URL: /info
Method: GET
URL params: none
Data params: none
Success Response:
Code: 200
Content: JSON formatted listing of several basic amphora data.
Error Response:
none
JSON Response attributes:
hostname - amphora hostname
uuid - amphora UUID
haproxy_version - Version of the haproxy installed
api_version - Version of haproxy amphora API in use
Notes: The data in this request is used by the controller for determining the amphora and API version numbers.
It’s also worth noting that this is the only API command that doesn’t have a version string prepended to it.
Examples:
Success code 200:
{
'hostname': 'octavia-haproxy-img-00328.local',
'uuid': '6e2bc8a0-2548-4fb7-a5f0-fb1ef4a696ce',
'haproxy_version': '1.5.11',
'api_version': '0.1',
}
Get amphora details¶
URL: /1.0/details
Method: GET
URL params: none
Data params: none
Success Response:
Code: 200
Content: JSON formatted listing of various amphora statistics.
Error Response:
none
JSON Response attributes:
hostname - amphora hostname
uuid - amphora UUID
haproxy_version - Version of the haproxy installed
api_version - Version of haproxy amphora API/agent in use
network_tx - Current total outbound bandwidth in bytes/sec (30-second snapshot)
network_rx - Current total inbound bandwidth in bytes/sec (30-second snapshot)
active - Boolean (is amphora in an “active” role?)
haproxy_count - Number of running haproxy processes
cpu - list of percent CPU usage broken down into:
total
user
system
soft_irq
memory - memory usage in kilobytes broken down into:
total
free
available
buffers
cached
swap_used
shared
slab
committed_as
disk - disk usage in kilobytes for root filesystem, listed as:
used
available
load - System load (list)
topology - One of SINGLE, ACTIVE-STANDBY, ACTIVE-ACTIVE
topology_status - One of OK, TOPOLOGY-CHANGE
listeners - list of listener UUIDs being serviced by this amphora
packages - list of load-balancing related packages installed with versions (eg. OpenSSL, haproxy, nginx, etc.)
Notes: The data in this request is meant to provide intelligence for an auto-scaling orchestration controller (heat) in order to determine whether additional (or fewer) virtual amphorae are necessary to handle load. As such, we may add additional parameters to the JSON listing above if they prove to be useful for making these decisions.
The data in this request is also used by the controller for determining overall health of the amphora, currently-configured topology and role, etc.
Examples
Success code 200:
{
'hostname': 'octavia-haproxy-img-00328.local',
'uuid': '6e2bc8a0-2548-4fb7-a5f0-fb1ef4a696ce',
'haproxy_version': '1.5.11',
'api_version': '0.1',
'networks': {
'eth0': {
'network_tx': 3300138,
'network_rx': 982001, }}
'active': 'TRUE',
'haproxy_count': 3,
'cpu':{
'total': 0.43,
'user': 0.30,
'system': 0.05,
'soft_irq': 0.08,
},
'memory':{
'total': 4087402,
'free': 760656,
'available': 2655901,
'buffers': 90980,
'cached': 1830143,
'swap_used': 943,
'shared': 105792,
'slab': 158819,
'committed_as': 2643480,
},
'disk':{
'used': 1234567,
'available': 5242880,
},
'load': [0.50, 0.45, 0.47],
'tolopogy': 'SINGLE',
'topology_status': 'OK',
'listeners':[
'02d0da8d-fc65-4bc4-bc46-95cadb2315d2',
'98e706a7-d22c-422f-9632-499fd83e12c0',
],
'packages':[
{'haproxy': '1.5.1'},
{'bash': '4.3.23'},
{'lighttpd': '1.4.33-1'},
{'openssl': '1.0.1f'},
<cut for brevity>
],
}
Get interface¶
URL: /1.0/interface/:ip
Method: GET
URL params:
:ip = the ip address to find the interface name
Data params: none
Success Response:
Code: 200
Content: OK
Content: JSON formatted interface
Error Response:
Code: 400
Content: Bad IP address version
Code: 404
Content: Error interface not found for IP address
Response:
Examples:
Success code 200:
GET URL:
https://octavia-haproxy-img-00328.local/1.0/interface/10.0.0.1
JSON Response:
{
'message': 'OK',
'interface': 'eth1'
}
Error code 404:
GET URL:
https://octavia-haproxy-img-00328.local/1.0/interface/10.5.0.1
JSON Response:
{
'message': 'Error interface not found for IP address',
}
Error code 404:
GET URL:
https://octavia-haproxy-img-00328.local/1.0/interface/10.6.0.1.1
JSON Response:
{
'message': 'Bad IP address version',
}
Get all listeners’ statuses¶
URL: /1.0/listeners
Method: GET
URL params: none
Data params: none
Success Response:
Code: 200
Content: JSON-formatted listing of each listener’s status
Error Response:
none
JSON Response attributes:
Note that the command will return an array of all listeners’ statuses. Each listener status contains the following attributes:
status - One of the operational status: ACTIVE, STOPPED, ERROR - future versions might support provisioning status: PENDING_CREATE, PENDING_UPDATE, PENDING_DELETE, DELETED
uuid - Listener UUID
type - One of: TCP, HTTP, TERMINATED_HTTPS
Notes: Note that this returns a status if: the pid file exists, the stats socket exists, or an haproxy configuration is present (not just if there is a valid haproxy configuration).
Examples
Success code 200:
[{
'status': 'ACTIVE',
'uuid': 'e2dfddc0-5b9e-11e4-8ed6-0800200c9a66',
'type': 'HTTP',
},
{
'status': 'STOPPED',
'uuid': '19d45130-5b9f-11e4-8ed6-0800200c9a66',
'type': 'TERMINATED_HTTPS',
}]
Start or Stop a load balancer¶
URL: /1.0/loadbalancer/:object_id/:action
Method: PUT
URL params:
:object_id = Object UUID
:action = One of: start, stop, reload
Data params: none
Success Response:
Code: 202
Content: OK
(Also contains preliminary results of attempt to start / stop / soft restart (reload) the haproxy daemon)
Error Response:
Code: 400
Content: Invalid request
Code: 404
Content: Listener Not Found
Code: 500
Content: Error starting / stopping / reload_config haproxy
(Also contains error output from attempt to start / stop / soft restart (reload) haproxy)
Code: 503
Content: Topology transition in progress
Response:
Examples:
Success code 201:
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/start
JSON Response:
{
'message': 'OK',
'details': 'Configuration file is valid\nhaproxy daemon for 85e2111b-29c4-44be-94f3-e72045805801 started',
}
Error code 400:
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/BAD_TEST_DATA
JSON Response:
{
'message': 'Invalid Request',
'details': 'Unknown action: BAD_TEST_DATA',
}
Error code 404:
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/04bff5c3-5862-4a13-b9e3-9b440d0ed50a/stop
JSON Response:
{
'message': 'Listener Not Found',
'details': 'No loadbalancer with UUID: 04bff5c3-5862-4a13-b9e3-9b440d0ed50a',
}
Error code 500:
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/stop
Response:
{
'message': 'Error stopping haproxy',
'details': 'haproxy process with PID 3352 not found',
}
Error code 503:
Response:
{
'message': 'Topology transition in progress',
}
Delete a listener¶
URL: /1.0/listeners/:listener
Method: DELETE
URL params:
:listener = Listener UUID
Data params: none
Success Response:
Code: 200
Content: OK
Error Response:
Code: 404
Content: Not Found
Code: 503
Content: Topology transition in progress
Response:
Implied actions:
Stop listener
Delete IPs, iptables accounting rules, etc. from this amphora if they’re no longer in use.
Clean up listener configuration directory.
Delete listener’s SSL certificates
Clean up logs (ship final logs to logging destination if configured)
Clean up stats socket.
Examples
Success code 200:
DELETE URL:
https://octavia-haproxy-img-00328.local/1.0/listeners/04bff5c3-5862-4a13-b9e3-9b440d0ed50a
JSON Response:
{
'message': 'OK'
}
Error code 404:
DELETE URL:
https://octavia-haproxy-img-00328.local/1.0/listeners/04bff5c3-5862-4a13-b9e3-9b440d0ed50a
JSON Response:
{
'message': 'Listener Not Found',
'details': 'No listener with UUID: 04bff5c3-5862-4a13-b9e3-9b440d0ed50a',
}
Error code 503:
Response:
{
'message': 'Topology transition in progress',
}
Upload SSL certificate PEM file¶
URL: /1.0/loadbalancer/:loadbalancer_id/certificates/:filename.pem
Method: PUT
URL params:
:loadbalancer_id = Load balancer UUID
:filename = PEM filename (see notes below for naming convention)
Data params: Certificate data. (PEM file should be a concatenation of unencrypted RSA key, certificate and chain, in that order)
Success Response:
Code: 201
Content: OK
Error Response:
Code: 400
Content: No certificate found
Code: 400
Content: No RSA key found
Code: 400
Content: Certificate and key do not match
Code: 404
Content: Not Found
Code: 503
Content: Topology transition in progress
Response:
Notes: * filename.pem should match the primary CN for which the certificate is valid. All-caps WILDCARD should be used to replace an asterisk in a wildcard certificate (eg. a CN of ‘*.example.com’ should have a filename of ‘WILDCARD.example.com.pem’). Filenames must also have the .pem extension. * In order for the new certificate to become effective the haproxy needs to be explicitly restarted
Examples:
Success code 201:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/certificates/www.example.com.pem
(Put data should contain the certificate information, concatenated as
described above)
JSON Response:
{
'message': 'OK'
}
Error code 400:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/certificates/www.example.com.pem
(If PUT data does not contain a certificate)
JSON Response:
{
'message': 'No certificate found'
}
Error code 400:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/certificates/www.example.com.pem
(If PUT data does not contain an RSA key)
JSON Response:
{
'message': 'No RSA key found'
}
Error code 400:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/certificates/www.example.com.pem
(If the first certificate and the RSA key do not have the same modulus.)
JSON Response:
{
'message': 'Certificate and key do not match'
}
Error code 404:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/certificates/www.example.com.pem
JSON Response:
{
'message': 'Listener Not Found',
'details': 'No loadbalancer with UUID: 04bff5c3-5862-4a13-b9e3-9b440d0ed50a',
}
Error code 503:
Response:
{
'message': 'Topology transition in progress',
}
Get SSL certificate md5sum¶
URL: /1.0/loadbalancer/:loadbalancer_id/certificates/:filename.pem
Method: GET
URL params:
:loadbalancer_id = Load balancer UUID
:filename = PEM filename (see notes below for naming convention)
Data params: none
Success Response:
Code: 200
Content: PEM file md5sum
Error Response:
Code: 404
Content: Not Found
Response:
Implied actions: none
Notes: The md5sum is the sum from the raw certificate data as stored on the amphora (which will usually include the RSA key, certificate and chain concatenated together). Note that we don’t return any actual raw certificate data as the controller should already know this information, and unnecessarily disclosing it over the wire from the amphora is a security risk.
Examples:
Success code 200:
JSON response:
{
'md5sum': 'd8f6629d5e3c6852fa764fb3f04f2ffd',
}
Error code 404:
JSON Response:
{
'message': 'Listener Not Found',
'details': 'No loadbalancer with UUID: 04bff5c3-5862-4a13-b9e3-9b440d0ed50a',
}
Error code 404:
JSON Response:
{
'message': 'Certificate Not Found',
'details': 'No certificate with file name: www.example.com.pem',
}
Delete SSL certificate PEM file¶
URL: /1.0/loadbalancer/:loadbalancer_id/certificates/:filename.pem
Method: DELETE
URL params:
:loadbalancer_id = Load balancer UUID
:filename = PEM filename (see notes below for naming convention)
Data params: none
Success Response:
Code: 200
Content: OK
Error Response:
Code: 404
Content: Not found
Code: 503
Content: Topology transition in progress
Implied actions:
Clean up listener configuration directory if it’s now empty.
Examples:
Success code 200:
DELETE URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/certificates/www.example.com.pem
JSON Response:
{
'message': 'OK'
}
Error code 404:
DELETE URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/certificates/www.example.com.pem
JSON Response:
{
'message': 'Certificate Not Found',
'details': 'No certificate with file name: www.example.com.pem',
}
Error code 503:
Response:
{
'message': 'Topology transition in progress',
}
Upload load balancer haproxy configuration¶
URL: /1.0/loadbalancer/:amphora_id/:loadbalancer_id/haproxy
Method: PUT
URL params:
:loadbalancer_id = Load Balancer UUID
:amphora_id = Amphora UUID
Data params: haproxy configuration file for the listener
Success Response:
Code: 201
Content: OK
Error Response:
Code: 400
Content: Invalid configuration
(Also includes error output from configuration check command)
Code: 503
Content: Topology transition in progress
Response:
Implied actions:
Do a syntax check on haproxy configuration file prior to an attempt to run it.
Add resources needed for stats, logs, and connectivity
Notes: The uploaded configuration file should be a complete and syntactically-correct haproxy config. The amphora does not have intelligence to generate these itself and has only rudimentary ability to parse certain features out of the configuration file (like bind addresses and ports for purposes of setting up stats, and specially formatted comments meant to indicate pools and members that will be parsed out of the haproxy daemon status interface for tracking health and stats).
Examples:
Success code 201:
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/d459b1c8-54b0-4030-9bec-4f449e73b1ef/85e2111b-29c4-44be-94f3-e72045805801/haproxy
(Upload PUT data should be a raw haproxy.conf file.)
JSON Response:
{
'message': 'OK'
}
Error code 400:
JSON Response:
{
'message': 'Invalid request',
'details': '[ALERT] 300/013045 (28236) : parsing [haproxy.cfg:4]: unknown keyword 'BAD_LINE' out of section.\n[ALERT] 300/013045 (28236) : Error(s) found in configuration file : haproxy.cfg\n[ALERT] 300/013045 (28236) : Fatal errors found in configuration.',
}
Error code 503:
Response:
{
'message': 'Topology transition in progress',
}
Get loadbalancer haproxy configuration¶
URL: /1.0/loadbalancer/:loadbalancer_id/haproxy
Method: GET
URL params:
:loadbalancer_id = Load balancer UUID
Data params: none
Success Response:
Code: 200
Content: haproxy configuration file for the listener
Error Response:
Code: 404
Content: Not found
Response:
Implied actions: none
Examples:
Success code 200:
GET URL:
https://octavia-haproxy-img-00328.local/1.0/loadbalancer/85e2111b-29c4-44be-94f3-e72045805801/haproxy
Response is the raw haproxy.cfg:
# Config file for 85e2111b-29c4-44be-94f3-e72045805801
(cut for brevity)
Error code 404:
JSON Response:
{
'message': 'Loadbalancer Not Found',
'details': 'No loadbalancer with UUID: 04bff5c3-5862-4a13-b9e3-9b440d0ed50a',
}
Plug VIP¶
URL: /1.0/plug/vip/:ip
Method: Post
URL params:
:ip = the vip’s ip address
Data params:
subnet_cidr: The vip subnet in cidr notation
gateway: The vip subnet gateway address
mac_address: The mac address of the interface to plug
Success Response:
Code: 202
Content: OK
Error Response: * Code: 400
Content: Invalid IP
Content: Invalid subnet information
Code: 404
Content: No suitable network interface found
Code: 500
Content: Error plugging VIP
(Also contains error output from the ip up command)
Code: 503
Content: Topology transition in progress
Response:
Implied actions:
Look for an interface marked as down (recently added port)
Assign VIP
Bring that interface up
Examples:
Success code 202:
POST URL:
https://octavia-haproxy-img-00328.local/1.0/plug/vip/203.0.113.2
JSON POST parameters:
{
'subnet_cidr': '203.0.113.0/24',
'gateway': '203.0.113.1',
'mac_address': '78:31:c1:ce:0b:3c'
}
JSON Response:
{
'message': 'OK',
'details': 'VIP 203.0.113.2 plugged on interface eth1'
}
Error code 400:
JSON Response:
{
'message': 'Invalid VIP',
}
Error code 404:
JSON Response:
{
'message': 'No suitable network interface found',
}
Plug Network¶
URL: /1.0/plug/network/
Method: POST
URL params: none
Data params:
mac_address: The mac address of the interface to plug
Success Response:
Code: 202
Content: OK
Error Response:
Code: 404
Content: No suitable network interface found
Code: 500
Content: Error plugging Port
(Also contains error output from the ip up command)
Code: 503
Content: Topology transition in progress
Response:
Examples:
Success code 202:
POST URL:
https://octavia-haproxy-img-00328.local/1.0/plug/network/
JSON POST parameters:
{
'mac_address': '78:31:c1:ce:0b:3c'
}
JSON Response:
{
'message': 'OK',
'details': 'Plugged interface eth1'
}
Error code 404:
JSON Response:
{
'message': 'No suitable network interface found',
}
Upload SSL server certificate PEM file for Controller Communication¶
URL: /1.0/certificate
Method: PUT
Data params: Certificate data. (PEM file should be a concatenation of unencrypted RSA key, certificate and chain, in that order)
Success Response:
Code: 202
Content: OK
Error Response:
Code: 400
Content: No certificate found
Code: 400
Content: No RSA key found
Code: 400
Content: Certificate and key do not match
Response:
Notes: Since certificates might be valid for a time smaller than the amphora is in existence this add a way to rotate them. Once the certificate is uploaded the agent is being recycled so depending on the implementation the service might not be available for some time.
Examples:
Success code 202:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/certificate
(Put data should contain the certificate information, concatenated as
described above)
JSON Response:
{
'message': 'OK'
}
Error code 400:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/certificates
(If PUT data does not contain a certificate)
JSON Response:
{
'message': 'No certificate found'
}
Error code 400:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/certificate
(If PUT data does not contain an RSA key)
JSON Response:
{
'message': 'No RSA key found'
}
Error code 400:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/certificate
(If the first certificate and the RSA key do not have the same modulus.)
JSON Response:
{
'message': 'Certificate and key do not match'
}
Upload keepalived configuration¶
URL: /1.0/vrrp/upload
Method: PUT
URL params: none
Data params: none
Success Response:
Code: 200
Content: OK
Error Response:
Code: 500
Content: Failed to upload keepalived configuration.
Response:
OK
Examples:
Success code 200:
PUT URI:
https://octavia-haproxy-img-00328.local/1.0/vrrp/upload
JSON Response:
{
'message': 'OK'
}
Start, Stop, or Reload keepalived¶
URL: /1.0/vrrp/:action
Method: PUT
URL params:
:action = One of: start, stop, reload
Data params: none
Success Response:
Code: 202
Content: OK
Error Response:
Code: 400
Content: Invalid Request
Code: 500
Content: Failed to start / stop / reload keepalived service:
(Also contains error output from attempt to start / stop / reload keepalived)
Response:
Examples:
Success code 202:
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/vrrp/start
JSON Response:
{
'message': 'OK',
'details': 'keepalived started',
}
Error code: 400
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/vrrp/BAD_TEST_DATA
JSON Response:
{
'message': 'Invalid Request',
'details': 'Unknown action: BAD_TEST_DATA',
}
Error code: 500
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/vrrp/stop
JSON Response:
{
'message': 'Failed to stop keepalived service: keeepalived process with PID 3352 not found',
'details': 'keeepalived process with PID 3352 not found',
}
Update the amphora agent configuration¶
URL: /1.0/config
Method: PUT
Data params: A amphora-agent configuration file
Success Response:
Code: 202
Content: OK
Error Response:
Code: 500
message: Unable to update amphora-agent configuration.
details: (The exception details)
Response:
Implied actions:
The running amphora-agent configuration file is mutated.
Notes: Only options that are marked mutable in the oslo configuration will be updated.
Examples:
Success code 202:
PUT URL:
https://octavia-haproxy-img-00328.local/1.0/config
(Upload PUT data should be a raw amphora-agent.conf file.)
JSON Response:
{
'message': 'OK'
}
Error code 500:
JSON Response:
{
'message': 'Unable to update amphora-agent configuration.',
'details': *(The exception output)*,
}