Hooks. Hook trigger plugins¶
Why Hooks?¶
All Rally workloads repeat their actions as many times as it is configured by runner. Once run, there is no way to interrupt the runner to evaluate any change or restart event on the stability of the cloud under test. For example we would like to test how configuration change or cloud component restart would affect performance and stability.
Task hooks were added to fill this gap and allow to use Rally for reliability and high availability testing. Generally, hooks allow to perform any actions on specified iteration or specified time since the workload has been started.
Also, task html-report provides results of hook execution. They can contain graphical or textual information with timing and statistics.
Hooks & Triggers Overview¶
Architecture¶
Rally uses runners to specify how many times the workload should be executed. Hooks do not use runners, instead they rely on trigger plugins to specify when and how many times hook should be called. Therefore hooks are isolated from workload runners and do not affect them because each hook is executed in separate thread.
Sample of usage¶
Hooks can be added to the task configuration. Lets take a look at hook configuration:
{
"name": "sys_call",
"args": "/bin/echo 123",
"trigger": {
"name": "event",
"args": {
"unit": "iteration",
"at": [5, 50, 200, 1000]
}
}
}
It specifies hook plugin with name "sys_call". "args" field contains string that will be used by sys_call plugin, but in case of any other hook plugin it can contain any other Python object, that is assumed to be passed to the hook. "trigger" field specifies which trigger plugin should be used to run this hook. "trigger" contains similar fields "name" and "args" which represent trigger plugin name and arguments for trigger plugin. In this example "event" trigger is specified and configured to run the hook at 5th, 50th, 200th and 1000th iterations.
Here is a full task config that contains previous hook configuraiton:
{
"Dummy.dummy": [
{
"args": {
"sleep": 0.01
},
"runner": {
"type": "constant",
"times": 1500,
"concurrency": 1
},
"hooks": [
{
"name": "sys_call",
"args": "/bin/echo 123",
"trigger": {
"name": "event",
"args": {
"unit": "iteration",
"at": [5, 50, 200, 1000]
}
}
}
]
}
]
}
Note
In this example, runner is configured to run workload 1500 times. So there is a limit for iterations and hook will be triggered only if certain iteration is started by runner. In other words, if trigger specifies iteration out of runner iterations scope then such trigger will not be called.
Task report for this example will contain minimal information about hook execution: duration of each hook call and its status(success of failure).
Let's take a look at more complicated config that can produce graphical and textual information.
---
Dummy.dummy:
-
args:
sleep: 0.75
runner:
type: "constant"
times: 20
concurrency: 2
hooks:
- name: sys_call
description: Run script
args: sh rally/rally-jobs/extra/hook_example_script.sh
trigger:
name: event
args:
unit: iteration
at: [2, 5, 8, 13, 17]
- name: sys_call
description: Show time
args: date +%Y-%m-%dT%H:%M:%S
trigger:
name: event
args:
unit: time
at: [0, 2, 5, 6, 9]
- name: sys_call
description: Show system name
args: uname -a
trigger:
name: event
args:
unit: iteration
at: [2, 3, 4, 5, 6, 8, 10, 12, 13, 15, 17, 18]
sla:
failure_rate:
max: 0
hook_example_script.sh generates dummy output in JSON format. Grafical information format is the same as for workloads and the same types of charts are supported for the hooks.
Here is a report that shows aggregated table and chart with hook results:
Here is report that shows lines chart and pie chart for first hook on the second iteration:
Writing your own Hook plugin¶
Problem description¶
Hook plugin should implement custom action that can be done one or multiple times during the workload. Examples of such actions might be the following:
- Destructive action inside cloud (Fault Injection)
- Getting information about current state of cloud (load/health)
- Upgrading/downgrading a component of cloud
- Changing configuration of cloud
- etc.
Plugin code¶
The following example shows simple hook code that performs system call.
It is inherited from the base Hook class and contains implemented run()
method:
import shlex
import subprocess
from rally import consts
from rally.task import hook
@hook.configure(name="simple_sys_call")
class SimpleSysCallHook(hook.Hook):
"""Performs system call."""
CONFIG_SCHEMA = {
"$schema": consts.JSON_SCHEMA,
"type": "string",
}
def run(self):
proc = subprocess.Popen(shlex.split(self.config),
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
proc.wait()
if proc.returncode:
self.set_error(
exception_name="n/a", # no exception class
description="Subprocess returned {}".format(proc.returncode),
details=proc.stdout.read(),
)
Any exceptions risen during execution of run method will be caught by Hook
base class and saved as a result. Although hook should manually call
Hook.set_error() to indicate logical error in case if there is no exception
raised.
Also there is a method for saving charts data: Hook.add_output().
Sample of task that uses Hook¶
{
"Dummy.dummy": [
{
"args": {
"sleep": 0.01
},
"runner": {
"type": "constant",
"times": 10,
"concurrency": 1
},
"hooks": [
{
"name": "simple_sys_call",
"args": "/bin/echo 123",
"trigger": {
"name": "event",
"args": {
"unit": "iteration",
"at": [3, 6]
}
}
}
]
}
]
}
Writing your own Trigger plugin¶
Problem description¶
Trigger plugin should implement an event processor that decides whether to start hook or not. Rally has two basic triggers that should cover most cases:
Plugin code¶
This example shows the code of the existing Event trigger:
from rally import consts
from rally.task import trigger
@trigger.configure(name="event")
class EventTrigger(trigger.Trigger):
"""Triggers hook on specified event and list of values."""
CONFIG_SCHEMA = {
"type": "object",
"$schema": consts.JSON_SCHEMA,
"oneOf": [
{
"properties": {
"unit": {"enum": ["time"]},
"at": {
"type": "array",
"minItems": 1,
"uniqueItems": True,
"items": {
"type": "integer",
"minimum": 0,
}
},
},
"required": ["unit", "at"],
"additionalProperties": False,
},
{
"properties": {
"unit": {"enum": ["iteration"]},
"at": {
"type": "array",
"minItems": 1,
"uniqueItems": True,
"items": {
"type": "integer",
"minimum": 1,
}
},
},
"required": ["unit", "at"],
"additionalProperties": False,
},
]
}
def get_listening_event(self):
return self.config["unit"]
def on_event(self, event_type, value=None):
if not (event_type == self.get_listening_event()
and value in self.config["at"]):
# do nothing
return
super(EventTrigger, self).on_event(event_type, value)
Trigger plugins must override two methods:
get_listening_event- this method should return currently configured event name. (So far Rally supports only "time" and "iteration")on_event- this method is called each time certain events occur. It calls base method when the hook is triggered on specified event.
Plugin Placement¶
All trigger plugins should be placed in Trigger folder.