Autoscaling with Heat¶
Goal¶
There are Senlin resource types in Heat which make deployment of a full-featured auto-scaling solution easily attainable. This document is to provide a tutorial for users who want to use heat to create a senlin cluster.
It is often required by real deployment practices to make the cluster load-balanced and auto-scaled. We also want the scaling action triggered based on business data instead of infrastructure metrics. When existing cluster is not enough to afford the throughput/workload, the cluster will be scaled-out; when low throughput or workload, the cluster will be scaled-in.
Moreover, custom is easy to do when auto-scaling. Receivers can be created to generate webhooks from scale_out and scale_in actions. Moreover, placement_zone.yaml and placement_region.yaml can be attached to cluster and guide which zone/region to place new nodes when scale_out; deletion_policy can be attached to the cluster and guide the choice of candidates to delete when scale_in.
Sample template¶
There have a sample template in heat-template project under directory of senlin for creation of Senlin elastic load-balanced cluster by Heat. Here we choose some important parts of the sample to explain one by one.
The resource below defines a security_group for connection to created load-balanced cluster:
security_group:
type: OS::Neutron::SecurityGroup
properties:
rules:
- protocol: icmp
- protocol: tcp
port_range_min: 22
port_range_max: 22
- protocol: tcp
port_range_min: 80
port_range_max: 80
The resource below defines the profile used to create the targeted cluster:
profile:
type: OS::Senlin::Profile
properties:
type: os.nova.server-1.0
properties:
flavor: {get_param: flavor}
image: {get_param: image}
key_name: {get_param: key_name}
networks:
- network: {get_param: network}
security_groups:
- {get_resource: security_group}
The resource below defines to create a Senlin cluster with two nodes at least:
cluster:
type: OS::Senlin::Cluster
properties:
desired_capacity: 2
min_size: 2
profile: {get_resource: profile}
The two resources below define scale_in_policy and scale_out_policy attached to the created cluster. Where, the property of event is used to define the objective action the policy works. When type of the property of adjustment is set as CHANGE_IN_CAPACITY, the cluster will increase the number of nodes when scale_out or decrease the number of nodes when scale_in:
scale_in_policy:
type: OS::Senlin::Policy
properties:
type: senlin.policy.scaling-1.0
bindings:
- cluster: {get_resource: cluster}
properties:
event: CLUSTER_SCALE_IN
adjustment:
type: CHANGE_IN_CAPACITY
number: 1
scale_out_policy:
type: OS::Senlin::Policy
properties:
type: senlin.policy.scaling-1.0
bindings:
- cluster: {get_resource: cluster}
properties:
event: CLUSTER_SCALE_OUT
adjustment:
type: CHANGE_IN_CAPACITY
number: 1
The resource below defines a lb_policy to be attached to the target cluster. Once the policy is attached to the cluster, Senlin will automatically create loadbalancer, pool, and health_monitor by invoking neutron LBaas V2 APIs for load-balancing purpose:
lb_policy:
type: OS::Senlin::Policy
properties:
type: senlin.policy.loadbalance-1.0
bindings:
- cluster: {get_resource: cluster}
properties:
pool:
protocol: HTTP
protocol_port: 80
subnet: {get_param: pool_subnet}
lb_method: ROUND_ROBIN
vip:
subnet: {get_param: vip_subnet}
protocol: HTTP
protocol_port: 80
health_monitor:
type: HTTP
delay: 10
timeout: 5
max_retries: 4
The two resources below define the receivers to be triggered when a certain alarm or event occurs:
receiver_scale_out:
type: OS::Senlin::Receiver
properties:
cluster: {get_resource: cluster}
action: CLUSTER_SCALE_OUT
type: webhook
receiver_scale_in:
type: OS::Senlin::Receiver
properties:
cluster: {get_resource: cluster}
action: CLUSTER_SCALE_IN
type: webhook
The resource below define the policy for selecting candidate nodes for deletion when the cluster is to be shrank:
deletion_policy:
type: OS::Senlin::Policy
properties:
type: senlin.policy.deletion-1.0
bindings:
- cluster: {get_resource: cluster}
properties:
criteria: YOUNGEST_FIRST
destroy_after_deletion: True
grace_period: 20
reduce_desired_capacity: False
The two resources below define the alarms to trigger the above two receivers respectively. We use the average rate of incoming bytes at LoadBalancer as the metrics to trigger the scaling operations:
scale_in_alarm:
type: OS::Ceilometer::Alarm
properties:
description: trigger when bandwidth overflow
meter_name: network.services.lb.incoming.bytes.rate
statistic: avg
period: 180
evaluation_periods: 1
threshold: 12000
repeat_actions: True
alarm_actions:
- {get_attr: [receiver_scale_in, channel, alarm_url]}
comparison_operator: le
query:
metadata.user_metadata.cluster_id: {get_resource: cluster}
scale_out_alarm:
type: OS::Ceilometer::Alarm
properties:
description: trigger when bandwidth insufficient
meter_name: network.services.lb.incoming.bytes.rate
statistic: avg
period: 60
evaluation_periods: 1
threshold: 28000
repeat_actions: True
alarm_actions:
- {get_attr: [receiver_scale_out, channel, alarm_url]}
comparison_operator: ge
query:
metadata.user_metadata.cluster_id: {get_resource: cluster}
Deployment Steps¶
Before the deployment, please ensure that neutron LBaas v2 and ceilometer/Aodh has been installed and configured in your environment.
Step one is to generate key-pair using the followed command:
$ openstack keypair create heat_key
Step two is to create a heat template as by downloading the template file from heat template.
Step three is to create a heat stack using the followed command:
$ openstack stack create test -t ./ex_aslb.yaml --parameter "key_name=heat_key"
The steps and samples introduced in this tutorial can also work well together with composition of ceilometer, Aodh, and Gnocchi without any change.
