Patterns

class taskflow.flow.Flow(name, retry=None)

Bases: object

The base abstract class of all flow implementations.

A flow is a structure that defines relationships between tasks. You can add tasks and other flows (as subflows) to the flow, and the flow provides a way to implicitly or explicitly define how they are interdependent. Exact structure of the relationships is defined by concrete implementation, while this class defines common interface and adds human-readable (not necessary unique) name.

NOTE(harlowja): if a flow is placed in another flow as a subflow, a desired way to compose flows together, then it is valid and permissible that during compilation the subflow & parent flow may be flattened into a new flow.

name

A non-unique name for this flow (human readable).

retry

The associated flow retry controller.

This retry controller object will affect & control how (and if) this flow and its contained components retry when execution is underway and a failure occurs.

add(*items)

Adds a given item/items to this flow.

iter_links()

Iterates over dependency links between children of the flow.

Iterates over 3-tuples (A, B, meta), where

• A is a child (atom or subflow) link starts from;
• B is a child (atom or subflow) link points to; it is said that B depends on A or B requires A;
• meta is link metadata, a dictionary.

iter_nodes()

Iterate over nodes of the flow.

Iterates over 2-tuples (A, meta), where

• A is a child (atom or subflow) of current flow;
• meta is link metadata, a dictionary.

provides

Set of symbol names provided by the flow.

requires

Set of unsatisfied symbol names required by the flow.

Linear flow

class taskflow.patterns.linear_flow.Flow(name, retry=None)

Bases: taskflow.flow.Flow

Linear flow pattern.

A linear (potentially nested) flow of tasks/flows that can be applied in order as one unit and rolled back as one unit using the reverse order that the tasks/flows have been applied in.

add(*items)

Adds a given task/tasks/flow/flows to this flow.

Unordered flow

class taskflow.patterns.unordered_flow.Flow(name, retry=None)

Bases: taskflow.flow.Flow

Unordered flow pattern.

A unordered (potentially nested) flow of tasks/flows that can be executed in any order as one unit and rolled back as one unit.

add(*items)

Adds a given task/tasks/flow/flows to this flow.

Graph flow

class taskflow.patterns.graph_flow.Flow(name, retry=None)

Bases: taskflow.flow.Flow

Graph flow pattern.

Contained flows/tasks will be executed according to their dependencies which will be resolved by using the flows/tasks provides and requires mappings or by following manually created dependency links.

From dependencies a directed graph is built. If it has edge A -> B, this means B depends on A (and that the execution of B must wait until A has finished executing, on reverting this means that the reverting of A must wait until B has finished reverting).

Note: cyclic dependencies are not allowed.

link(u, v, decider=None, decider_depth=None)

Link existing node u as a runtime dependency of existing node v.

Note that if the addition of these edges creates a cyclic graph then a DependencyFailure will be raised and the provided changes will be discarded. If the nodes that are being requested to link do not exist in this graph than a ValueError will be raised.

Parameters:

• u – task or flow to create a link from (must exist already)
• v – task or flow to create a link to (must exist already)
• decider – A callback function that will be expected to decide at runtime whether v should be allowed to execute (or whether the execution of v should be ignored, and therefore not executed). It is expected to take as single keyword argument history which will be the execution results of all u decidable links that have v as a target. It is expected to return a single boolean (True to allow v execution or False to not).
• decider_depth – One of the Depth enumerations (or a string version of) that will be used to influence what atoms are ignored when the decider provided results false. If not provided (and a valid decider is provided then this defaults to ALL).

add(*nodes, **kwargs)

Adds a given task/tasks/flow/flows to this flow.

Note that if the addition of these nodes (and any edges) creates a cyclic graph then a DependencyFailure will be raised and the applied changes will be discarded.

Parameters:

• nodes – node(s) to add to the flow
• kwargs –

  keyword arguments, the two keyword arguments currently processed are:

  • resolve_requires a boolean that when true (the default) implies that when node(s) are added their symbol requirements will be matched to existing node(s) and links will be automatically made to those providers. If multiple possible providers exist then a AmbiguousDependency exception will be raised and the provided additions will be discarded.
  • resolve_existing, a boolean that when true (the default) implies that on addition of a new node that existing node(s) will have their requirements scanned for symbols that this newly added node can provide. If a match is found a link is automatically created from the newly added node to the requiree.

class taskflow.patterns.graph_flow.TargetedFlow(*args, **kwargs)

Bases: taskflow.patterns.graph_flow.Flow

Graph flow with a target.

Adds possibility to execute a flow up to certain graph node (task or subflow).

set_target(target_node)

Set target for the flow.

Any node(s) (tasks or subflows) not needed for the target node will not be executed.

reset_target()

Reset target for the flow.

All node(s) of the flow will be executed.

add(*args, **kwargs)

Adds a given task/tasks/flow/flows to this flow.

Note that if the addition of these nodes (and any edges) creates a cyclic graph then a DependencyFailure will be raised and the applied changes will be discarded.

Parameters:

• nodes – node(s) to add to the flow
• kwargs –

  keyword arguments, the two keyword arguments currently processed are:

  • resolve_requires a boolean that when true (the default) implies that when node(s) are added their symbol requirements will be matched to existing node(s) and links will be automatically made to those providers. If multiple possible providers exist then a AmbiguousDependency exception will be raised and the provided additions will be discarded.
  • resolve_existing, a boolean that when true (the default) implies that on addition of a new node that existing node(s) will have their requirements scanned for symbols that this newly added node can provide. If a match is found a link is automatically created from the newly added node to the requiree.

link(*args, **kwargs)

Link existing node u as a runtime dependency of existing node v.

Note that if the addition of these edges creates a cyclic graph then a DependencyFailure will be raised and the provided changes will be discarded. If the nodes that are being requested to link do not exist in this graph than a ValueError will be raised.

Parameters:

• u – task or flow to create a link from (must exist already)
• v – task or flow to create a link to (must exist already)
• decider – A callback function that will be expected to decide at runtime whether v should be allowed to execute (or whether the execution of v should be ignored, and therefore not executed). It is expected to take as single keyword argument history which will be the execution results of all u decidable links that have v as a target. It is expected to return a single boolean (True to allow v execution or False to not).
• decider_depth – One of the Depth enumerations (or a string version of) that will be used to influence what atoms are ignored when the decider provided results false. If not provided (and a valid decider is provided then this defaults to ALL).

class taskflow.deciders.Depth

Bases: taskflow.utils.misc.StrEnum

Enumeration of decider(s) area of influence.

ALL = ‘ALL’

Default decider depth that affects all successor atoms (including ones that are in successor nested flows).

FLOW = ‘FLOW’

Decider depth that affects all successor tasks in the same flow (it will not affect tasks/retries that are in successor nested flows).

Warning

While using this kind we are allowed to execute successors of things that have been ignored (for example nested flows and the tasks they contain), this may result in symbol lookup errors during running, user beware.

NEIGHBORS = ‘NEIGHBORS’

Decider depth that affects only next successor tasks (and does not traverse past one level of successor tasks).

Warning

While using this kind we are allowed to execute successors of things that have been ignored (for example nested flows and the tasks they contain), this may result in symbol lookup errors during running, user beware.

ATOM = ‘ATOM’

Decider depth that affects only targeted atom (and does not traverse into any level of successor atoms).

Warning

While using this kind we are allowed to execute successors of things that have been ignored (for example nested flows and the tasks they contain), this may result in symbol lookup errors during running, user beware.

classmethod translate(desired_depth)

Translates a string into a depth enumeration.

taskflow.deciders.pick_widest(depths)

Pick from many depths which has the widest area of influence.

Hierarchy