Arguments and results¶

Arguments specification¶

There are different ways to specify the task argument requires set.

>>> class MyTask(task.Task):
...     def execute(self, spam, eggs, bacon=None):
...         return spam + eggs
...
>>> sorted(MyTask().requires)
['eggs', 'spam']
>>> sorted(MyTask().optional)
['bacon']

>>> class UniTask(task.Task):
...     def execute(self, *args, **kwargs):
...         pass
...
>>> sorted(UniTask().requires)
[]

class SpawnVMTask(task.Task):

    def execute(self, vm_name, vm_image_id, **kwargs):
        pass  # TODO(imelnikov): use parameters to spawn vm

SpawnVMTask(rebind={'vm_name': 'name'})

SpawnVMTask(rebind_args=('name', 'vm_image_id'))

SpawnVMTask(rebind=dict(vm_name='name',
                        vm_image_id='vm_image_id'))

SpawnVMTask(rebind=('name', 'vm_image_id', 'admin_key_name'))

>>> class Cat(task.Task):
...     def __init__(self, **kwargs):
...         if 'requires' not in kwargs:
...             kwargs['requires'] = ("food", "milk")
...         super(Cat, self).__init__(**kwargs)
...     def execute(self, food, **kwargs):
...         pass
...
>>> cat = Cat()
>>> sorted(cat.requires)
['food', 'milk']

>>> class Dog(task.Task):
...     def execute(self, food, **kwargs):
...         pass
>>> dog = Dog(requires=("water", "grass"))
>>> sorted(dog.requires)
['food', 'grass', 'water']

>>> class Bird(task.Task):
...     def execute(self, food, **kwargs):
...         pass
>>> bird = Bird(requires=("food", "water", "grass"), auto_extract=False)
>>> sorted(bird.requires)
['food', 'grass', 'water']

Results specification¶

In python, function results are not named, so we can not infer what a task/retry returns. This is important since the complete result (what the task execute() or retry execute() method returns) is saved in (potentially persistent) storage, and it is typically (but not always) desirable to make those results accessible to others. To accomplish this the task/retry specifies names of those values via its provides constructor parameter or by its default provides attribute.

>>> class TheAnswerReturningTask(task.Task):
...    def execute(self):
...        return 42
...
>>> sorted(TheAnswerReturningTask(provides='the_answer').provides)
['the_answer']

class BitsAndPiecesTask(task.Task):
    def execute(self):
        return 'BITs', 'PIECEs'

BitsAndPiecesTask(provides=('bits', 'pieces'))

>>> storage.fetch('bits')
'BITs'
>>> storage.fetch('pieces')
'PIECEs'

class BitsAndPiecesTask(task.Task):

    def execute(self):
        return {
            'bits': 'BITs',
            'pieces': 'PIECEs'
        }

BitsAndPiecesTask(provides=set(['bits', 'pieces']))

>>> storage.fetch('bits')
'BITs'
>>> storage.fetch('pieces')
'PIECEs'

class BitsAndPiecesTask(task.Task):
    default_provides = ('bits', 'pieces')
    def execute(self):
        return 'BITs', 'PIECEs'

BitsAndPiecesTask(provides=('b', 'p'))

BitsAndPiecesTask(provides='bnp')

BitsAndPiecesTask(provides=())

Revert arguments¶

To revert a task the engine calls the tasks revert() method. This method should accept the same arguments as the execute() method of the task and one more special keyword argument, named result.

For result value, two cases are possible:

• If the task is being reverted because it failed (an exception was raised from its execute() method), the result value is an instance of a Failure object that holds the exception information.
• If the task is being reverted because some other task failed, and this task finished successfully, result value is the result fetched from storage: ie, what the execute() method returned.

All other arguments are fetched from storage in the same way it is done for execute() method.

To determine if a task failed you can check whether result is instance of Failure:

from taskflow.types import failure

class RevertingTask(task.Task):

    def execute(self, spam, eggs):
        return do_something(spam, eggs)

    def revert(self, result, spam, eggs):
        if isinstance(result, failure.Failure):
            print("This task failed, exception: %s"
                  % result.exception_str)
        else:
            print("do_something returned %r" % result)

If this task failed (ie do_something raised an exception) it will print "This task failed, exception:" and a exception message on revert. If this task finished successfully, it will print "do_something returned" and a representation of the do_something result.

Retry arguments¶

A Retry controller works with arguments in the same way as a Task. But it has an additional parameter 'history' that is itself a History object that contains what failed over all the engines attempts (aka the outcomes). The history object can be viewed as a tuple that contains a result of the previous retries run and a table/dict where each key is a failed atoms name and each value is a Failure object.

Consider the following implementation:

class MyRetry(retry.Retry):

    default_provides = 'value'

    def on_failure(self, history, *args, **kwargs):
        print(list(history))
        return RETRY

    def execute(self, history, *args, **kwargs):
        print(list(history))
        return 5

    def revert(self, history, *args, **kwargs):
        print(list(history))

Imagine the above retry had returned a value '5' and then some task 'A' failed with some exception. In this case on_failure method will receive the following history (printed as a list):

[('5', {'A': failure.Failure()})]

At this point (since the implementation returned RETRY) the execute() method will be called again and it will receive the same history and it can then return a value that subsequent tasks can use to alter their behavior.

If instead the execute() method itself raises an exception, the revert() method of the implementation will be called and a Failure object will be present in the history object instead of the typical result.