Appendix R: Swift usage¶
Overview¶
There are two fundamental ways to deploy a Swift cluster with charms. Each method differs in how storage nodes get assigned to storage zones.
As of the 20.02 charm release, with OpenStack Newton or later, support for a multi-region (global) cluster, which is an extension of the single-region scenario, is available. See the upstream documentation on Global clusters for background information.
Warning
Charmed Swift global cluster functionality is in a preview state and is ready for testing. It is not production-ready.
Any Swift deployment relies upon the swift-proxy and swift-storage charms. Refer to those charms to learn about the various configuration options used throughout this guide.
Single-region cluster¶
Manual zone assignment¶
The ‘manual’ method (the default) allows the cluster to be designed by explicitly assigning a storage zone to a storage node. This zone gets associated with the swift-storage application. This means that this method involves multiple uniquely-named swift-storage applications.
Let file swift.yaml
contain the configuration:
swift-proxy:
zone-assignment: manual
replicas: 3
swift-storage-zone1:
zone: 1
block-device: /dev/sdb
swift-storage-zone2:
zone: 2
block-device: /dev/sdb
swift-storage-zone3:
zone: 3
block-device: /dev/sdb
Deploy the proxy and storage nodes:
juju deploy --config swift.yaml swift-proxy
juju deploy --config swift.yaml swift-storage swift-storage-zone1
juju deploy --config swift.yaml swift-storage swift-storage-zone2
juju deploy --config swift.yaml swift-storage swift-storage-zone3
Add relations between the proxy node and all storage nodes:
juju add-relation swift-proxy:swift-storage swift-storage-zone1:swift-storage
juju add-relation swift-proxy:swift-storage swift-storage-zone2:swift-storage
juju add-relation swift-proxy:swift-storage swift-storage-zone3:swift-storage
This will result in a three-zone cluster, with each zone consisting of a single storage node, thereby satisfying the replica requirement of three.
Storage capacity is increased by adding swift-storage units to a zone. For example, to add two storage nodes to zone ‘3’:
juju add-unit -n 2 swift-storage-zone3
Note
When scaling out ensure the candidate machines are equipped with the block devices currently configured for the associated application.
This charm will not balance the storage ring until there are enough storage zones to meet its minimum replica requirement, in this case three.
Auto zone assignment¶
The ‘auto’ method automatically assigns storage zones to storage nodes. There is only one swift-storage application and only one relation between it and the swift-proxy application. The relation sets up the initial storage node in zone ‘1’ (by default). Newly-added nodes get assigned to zone ‘2’, ‘3’, and so on, until the number of stated replicas is reached. Zone numbering then falls back to ‘1’ and the zone-assignment cycle repeats. In this way, storage nodes get distributed evenly across zones.
Let file swift.yaml
contain the configuration:
swift-proxy:
zone-assignment: auto
replicas: 3
swift-storage:
block-device: /dev/sdb
Deploy the proxy node and the storage application:
juju deploy --config swift.yaml swift-proxy
juju deploy --config swift.yaml swift-storage
The first storage node gets assigned to zone ‘1’ when the initial relation is added:
juju add-relation swift-proxy:swift-storage swift-storage:swift-storage
The second and third units get assigned to zones ‘2’ and ‘3’, respectively, during scale-out operations:
juju add-unit -n 2 swift-storage
Note
When scaling out ensure the candidate machines are equipped with the block devices currently configured for the associated application.
At this point the replica requirement is satisfied and the ring is balanced. The ring is extended by continuing to add more units to the single application.
Multi-region cluster¶
The previous configurations provided a single-region cluster. Generally a region is composed of a group of nodes with high-bandwidth, low-latency links between them. This almost always translates to the same geographical location.
Multiple such clusters can be meshed together to create a multi-region (global) cluster. The goal is to achieve greater data resiliency by spanning zones across geographically dispersed regions.
This section includes two configurations for implementing a Swift global cluster: minimal and comprehensive.
A global cluster is an extension of the single cluster scenario. Refer to the Single-region cluster section for information on essential options.
Minimal configuration¶
The proxy and storage nodes for a global cluster require extra configuration:
On the proxy node,
option
enable-multi-region
is set to ‘true’option
region
is definedoption
swift-hash
is defined (same value for all regions)
On the storage nodes,
option
storage-region
is set
The below example has two storage regions, a single zone, one storage node per storage region, and a replica requirement of two. Manual zone assignment will be used.
Let file swift.yaml
contain the configuration:
swift-proxy-region1:
region: RegionOne
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-000000111111"
swift-proxy-region2:
region: RegionTwo
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-000000111111"
swift-storage-region1:
storage-region: 1
zone: 1
block-device: /dev/sdb
swift-storage-region2:
storage-region: 2
zone: 1
block-device: /dev/sdb
The value of swift-hash
is arbitrary. It is provided here in the form of a
UUID.
Important
The name of a storage region must be an integer. Here, OpenStack region ‘RegionOne’ corresponds to storage region ‘1’, and OpenStack region ‘RegionTwo’ corresponds to storage region ‘2’.
Deploy in RegionOne:
juju deploy --config swift.yaml swift-proxy swift-proxy-region1
juju deploy --config swift.yaml swift-storage swift-storage-region1
Deploy in RegionTwo:
juju deploy --config swift.yaml swift-proxy swift-proxy-region2
juju deploy --config swift.yaml swift-storage swift-storage-region2
Add relations between swift-proxy in RegionOne and swift-storage in both RegionOne and RegionTwo:
juju add-relation swift-proxy-region1:swift-storage swift-storage-region1:swift-storage
juju add-relation swift-proxy-region1:swift-storage swift-storage-region2:swift-storage
Add relations between swift-proxy in RegionTwo and swift-storage in both RegionOne and RegionTwo:
juju add-relation swift-proxy-region2:swift-storage swift-storage-region1:swift-storage
juju add-relation swift-proxy-region2:swift-storage swift-storage-region2:swift-storage
Add a relation between swift-proxy in RegionOne and swift-proxy in RegionTwo:
juju add-relation swift-proxy-region1:rings-distributor swift-proxy-region2:rings-consumer
More than one proxy can be deployed per OpenStack region, and each must have a relation to every proxy in all other OpenStack regions. Only one proxy can act as a “rings-distributor” at any one time; the proxy in RegionOne was arbitrarily chosen.
Comprehensive configuration¶
A global cluster is primarily useful when there are groups of storage nodes and proxy nodes in different physical regions, creating a geographically-distributed cluster. These regions typically reside in distinct Juju models, making Cross model relations a necessity. In addition, there are configuration options available for tuning read and write behaviour. The next example demonstrates how to implement these features and options in a realistic scenario.
Refer to the Minimal configuration section for basic settings.
Tuning configuration¶
The read-affinity
option is used to control what order the regions and
zones are examined when searching for an object. A common approach would be to
put the local region first on the search path for a proxy. For instance, in the
deployment example below the Swift proxy in New York is configured to read from
the New York storage nodes first. Similarly the San Francisco proxy prefers
storage nodes in San Francisco.
The write-affinity
option allows nodes to be stored locally before being
eventually distributed globally. This write_affinity setting is useful only
when you do not read objects immediately after writing them.
The write-affinity-node-count
option is used to further configure
write-affinity
. This option dictates how many local storage servers will be
tried before falling back to remote ones.
Storage regions are referred to by prepending an ‘r’ to their names. Hence ‘r1’ refers to storage region ‘1’. Similarly for zones. Zone ‘1’ is referred to by ‘z1’.
For more details on these options see the upstream Global clusters document.
Deployment¶
This example assumes there are two data centres, one in San Francisco (SF) and one in New York (NY). These contain Juju models ‘swift-sf’ and ‘swift-ny’ respectively. Model ‘swift-ny’ contains OpenStack region ‘RegionOne’ and storage region ‘1’. Model ‘swift-sf’ contains OpenStack region ‘RegionTwo’ and storage region ‘2’.
Bundle overlays are needed for encapsulating cross-model relations. So the deployment in each OpenStack region consists of both a bundle and an overlay.
This is the contents of bundle swift-ny.yaml
:
series: bionic
applications:
swift-proxy-region1:
charm: cs:swift-proxy
num_units: 1
options:
region: RegionOne
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-000000111111"
read-affinity: "r1=100, r2=200"
write-affinity: "r1, r2"
write-affinity-node-count: '1'
openstack-origin: cloud:bionic-train
swift-storage-region1-zone1:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 1
zone: 1
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region1-zone2:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 1
zone: 2
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region1-zone3:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 1
zone: 3
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
percona-cluster:
charm: cs:~openstack-charmers-next/percona-cluster
num_units: 1
options:
dataset-size: 25%
max-connections: 1000
source: cloud:bionic-train
keystone:
expose: True
charm: cs:~openstack-charmers-next/keystone
num_units: 1
options:
openstack-origin: cloud:bionic-train
glance:
expose: True
charm: cs:~openstack-charmers-next/glance
num_units: 1
options:
openstack-origin: cloud:bionic-train
relations:
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone1:swift-storage
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone2:swift-storage
- - swift-proxy-region1:swift-storage
- swift-storage-region1-zone3:swift-storage
- - keystone:shared-db
- percona-cluster:shared-db
- - glance:shared-db
- percona-cluster:shared-db
- - glance:identity-service
- keystone:identity-service
- - swift-proxy-region1:identity-service
- keystone:identity-service
- - glance:object-store
- swift-proxy-region1:object-store
This is the contents of overlay bundle swift-ny-offers.yaml
:
applications:
keystone:
offers:
keystone-offer:
endpoints:
- identity-service
swift-proxy-region1:
offers:
swift-proxy-region1-offer:
endpoints:
- swift-storage
- rings-distributor
swift-storage-region1-zone1:
offers:
swift-storage-region1-zone1-offer:
endpoints:
- swift-storage
swift-storage-region1-zone2:
offers:
swift-storage-region1-zone2-offer:
endpoints:
- swift-storage
swift-storage-region1-zone3:
offers:
swift-storage-region1-zone3-offer:
endpoints:
- swift-storage
This is the contents of bundle swift-sf.yaml
:
series: bionic
applications:
swift-proxy-region2:
charm: cs:swift-proxy
num_units: 1
options:
region: RegionTwo
zone-assignment: manual
replicas: 2
enable-multi-region: true
swift-hash: "efcf2102-b9e9-4d71-afe6-000000111111"
read-affinity: "r1=100, r2=200"
write-affinity: "r1, r2"
write-affinity-node-count: '1'
openstack-origin: cloud:bionic-train
swift-storage-region2-zone1:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 2
zone: 1
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region2-zone2:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 2
zone: 2
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
swift-storage-region2-zone3:
charm: cs:swift-storage
num_units: 1
options:
storage-region: 2
zone: 3
block-device: /etc/swift/storage.img|2G
openstack-origin: cloud:bionic-train
relations:
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone1:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone2:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region2-zone3:swift-storage
This is the contents of overlay bundle swift-sf-consumer.yaml
:
relations:
- - swift-proxy-region2:identity-service
- keystone:identity-service
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone1:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone2:swift-storage
- - swift-proxy-region2:swift-storage
- swift-storage-region1-zone3:swift-storage
- - swift-storage-region2-zone1:swift-storage
- swift-proxy-region1:swift-storage
- - swift-storage-region2-zone2:swift-storage
- swift-proxy-region1:swift-storage
- - swift-storage-region2-zone3:swift-storage
- swift-proxy-region1:swift-storage
- - swift-proxy-region2:rings-consumer
- swift-proxy-region1:rings-distributor
saas:
keystone:
url: admin/swift-ny.keystone-offer
swift-proxy-region1:
url: admin/swift-ny.swift-proxy-region1-offer
swift-storage-region1-zone1:
url: admin/swift-ny.swift-storage-region1-zone1-offer
swift-storage-region1-zone2:
url: admin/swift-ny.swift-storage-region1-zone2-offer
swift-storage-region1-zone3:
url: admin/swift-ny.swift-storage-region1-zone3-offer
With the current configuration, swift-ny.yaml
must be deployed first as it
contains the Juju “offers” that swift-sf.yaml
will consume:
juju deploy -m swift-ny ./swift-ny.yaml --overlay ./swift-ny-offers.yaml
juju deploy -m swift-sf ./swift-sf.yaml --overlay ./swift-sf-consumer.yaml