Loading the Plugins

There are several different enabling and invocation patterns for consumers of plugins, depending on your needs.

Loading Drivers

The most common way plugins are used is as individual drivers. In this case, there may be many plugin options to choose from, but only one needs to be loaded and called. The DriverManager class supports this pattern.

This example program uses a DriverManager to load a formatter defined in the examples for stevedore. It then uses the formatter to convert a data structure to a text format, which it can print.

# stevedore/example/load_as_driver.py
from __future__ import print_function

import argparse

from stevedore import driver


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument(
        'format',
        nargs='?',
        default='simple',
        help='the output format',
    )
    parser.add_argument(
        '--width',
        default=60,
        type=int,
        help='maximum output width for text',
    )
    parsed_args = parser.parse_args()

    data = {
        'a': 'A',
        'b': 'B',
        'long': 'word ' * 80,
    }

    mgr = driver.DriverManager(
        namespace='stevedore.example.formatter',
        name=parsed_args.format,
        invoke_on_load=True,
        invoke_args=(parsed_args.width,),
    )
    for chunk in mgr.driver.format(data):
        print(chunk, end='')

The manager takes the plugin namespace and name as arguments, and uses them to find the plugin. Then, because invoke_on_load is true, it calls the object loaded. In this case that object is the plugin class registered as a formatter. The invoke_args are positional arguments passed to the class constructor, and are used to set the maximum width parameter.

    mgr = driver.DriverManager(
        namespace='stevedore.example.formatter',
        name=parsed_args.format,
        invoke_on_load=True,
        invoke_args=(parsed_args.width,),
    )

After the manager is created, it holds a reference to a single object returned by calling the code registered for the plugin. That object is the actual driver, in this case an instance of the formatter class from the plugin. The single driver can be accessed via the driver property of the manager, and then its methods can be called directly.

    for chunk in mgr.driver.format(data):
        print(chunk, end='')

Running the example program produces this output:

$ python -m stevedore.example.load_as_driver a = A
b = B
long = word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word

$ python -m stevedore.example.load_as_driver field
: a : A
: b : B
: long : word word word word word word word word word word
    word word word word word word word word word word word
    word word word word word word word word word word word
    word word word word word word word word word word word
    word word word word word word word word word word word
    word word word word word word word word word word word
    word word word word word word word word word word word
    word word word word

$ python -m stevedore.example.load_as_driver field --width 30
: a : A
: b : B
: long : word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word

Loading Extensions

Another common use case is to load several extensions at one time, and do something with all of them. Several of the other manager classes support this invocation pattern, including ExtensionManager, NamedExtensionManager, and EnabledExtensionManager.

# stevedore/example/load_as_extension.py
from __future__ import print_function

import argparse

from stevedore import extension


if __name__ == '__main__':
    parser = argparse.ArgumentParser()
    parser.add_argument(
        '--width',
        default=60,
        type=int,
        help='maximum output width for text',
    )
    parsed_args = parser.parse_args()

    data = {
        'a': 'A',
        'b': 'B',
        'long': 'word ' * 80,
    }

    mgr = extension.ExtensionManager(
        namespace='stevedore.example.formatter',
        invoke_on_load=True,
        invoke_args=(parsed_args.width,),
    )

    def format_data(ext, data):
        return (ext.name, ext.obj.format(data))

    results = mgr.map(format_data, data)

    for name, result in results:
        print('Formatter: {0}'.format(name))
        for chunk in result:
            print(chunk, end='')
        print('')

The ExtensionManager is created slightly differently from the DriverManager because it does not need to know in advance which plugin to load. It loads all of the plugins it finds.

    mgr = extension.ExtensionManager(
        namespace='stevedore.example.formatter',
        invoke_on_load=True,
        invoke_args=(parsed_args.width,),
    )

To call the plugins, use the map() method, passing a callable to be invoked for each extension. The format_data() function used with map() in this example takes two arguments, the Extension and the data argument given to map().

    def format_data(ext, data):
        return (ext.name, ext.obj.format(data))

    results = mgr.map(format_data, data)

The Extension passed format_data() is a class defined by stevedore that wraps the plugin. It includes the name of the plugin, the EntryPoint returned by pkg_resources, and the plugin itself (the named object referenced by the plugin definition). When invoke_on_load is true, the Extension will also have an obj attribute containing the value returned when the plugin was invoked.

map() returns a sequence of the values returned by the callback function. In this case, format_data() returns a tuple containing the extension name and the iterable that produces the text to print. As the results are processed, the name of each plugin is printed and then the formatted data.

    for name, result in results:
        print('Formatter: {0}'.format(name))
        for chunk in result:
            print(chunk, end='')
        print('')

The order the plugins are loaded is undefined, and depends on the order packages are found on the import path as well as the way the metadata files are read. If the order extensions are used matters, try the NamedExtensionManager.

$ python -m stevedore.example.load_as_extension --width 30
Formatter: simple
a = A
b = B
long = word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word

Formatter: field
: a : A
: b : B
: long : word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word word word word word
    word

Formatter: plain
a = A
b = B
long = word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word word

Why Not Call Plugins Directly?

Using a separate callable argument to map(), rather than just invoking the plugin directly introduces a separation between your application code and the plugins. The benefits of this separation manifest in the application code design and in the plugin API design.

If map() called the plugin directly, each plugin would have to be a callable. That would mean a separate namespace for what is really just a method of the plugin. By using a separate callable argument, the plugin API does not need to match exactly any particular use case in the application. This frees you to create a finer-grained API, with more individual methods that can be called in different ways to achieve different goals.

See also