Watcher Decision Engine has an external scoring engine plugin interface which gives anyone the ability to integrate an external scoring engine in order to make use of it in a strategy.
This section gives some guidelines on how to implement and integrate custom scoring engines with Watcher. If you wish to create a third-party package for your plugin, you can refer to our documentation for third-party package creation.
Because scoring engines execute a purely mathematical tasks, they typically do not have any additional dependencies. Additional requirements might be defined by specific scoring engine implementations. For example, some scoring engines might require to prepare learning data, which has to be loaded during the scoring engine startup. Some other might require some external services to be available (e.g. if the scoring infrastructure is running in the cloud).
In order to create a new scoring engine you have to:
ScoringEngine
classget_name()
method to return the
unique ID of the new scoring engine you want to create. This unique ID
should be the same as the name of the entry point we will declare later
on.get_description()
method to return the
user-friendly description of the implemented scoring engine. It might contain
information about algorithm used, learning data etc.get_metainfo()
method to return the
machine-friendly metadata about this scoring engine. For example, it could be
a JSON formatted text with information about the data model used, its input
and output data format, column names, etc.calculate_score()
method to return the
result calculated by this scoring engine.Here is an example showing how you can write a plugin called NewScorer
:
# filepath: thirdparty/new.py
# import path: thirdparty.new
from watcher.decision_engine.scoring import base
class NewScorer(base.ScoringEngine):
def get_name(self):
return 'new_scorer'
def get_description(self):
return ''
def get_metainfo(self):
return """{
"feature_columns": [
"column1",
"column2",
"column3"],
"result_columns": [
"value",
"probability"]
}"""
def calculate_score(self, features):
return '[12, 0.83]'
As you can see in the above example, the
calculate_score()
method returns a string. Both this
class and the client (caller) should perform all the necessary serialization
or deserialization.
Optionally, it’s possible to implement a container plugin, which can return a
list of scoring engines. This list can be re-evaluated multiple times during
the lifecycle of Watcher Decision Engine and synchronized with Watcher
Database using the watcher-sync
command line
tool.
Below is an example of a container using some scoring engine implementation that is simply made of a client responsible for communicating with a real scoring engine deployed as a web service on external servers:
class NewScoringContainer(base.ScoringEngineContainer):
@classmethod
def get_scoring_engine_list(self):
return [
RemoteScoringEngine(
name='scoring_engine1',
description='Some remote Scoring Engine 1',
remote_url='http://engine1.example.com/score'),
RemoteScoringEngine(
name='scoring_engine2',
description='Some remote Scoring Engine 2',
remote_url='http://engine2.example.com/score'),
]
Here below is the abstract ScoringEngine
class:
watcher.decision_engine.scoring.base.
ScoringEngine
(config)[source]A base class for all the Scoring Engines.
A Scoring Engine is an instance of a data model, to which the learning data was applied.
Please note that this class contains non-static and non-class methods by design, so that it’s easy to create multiple Scoring Engine instances using a single class (possibly configured differently).
calculate_score
(features)[source]Calculates a score value based on arguments passed.
Scoring Engines might be very different to each other. They might solve different problems or use different algorithms or frameworks internally. To enable this kind of flexibility, the method takes only one argument (string) and produces the results in the same format (string). The consumer of the Scoring Engine is ultimately responsible for providing the right arguments and parsing the result.
Parameters: | features (str) – Input data for Scoring Engine |
---|---|
Returns: | A score result |
Return type: | str |
get_config_opts
()[source]Defines the configuration options to be associated to this loadable
Returns: | A list of configuration options relative to this Loadable |
---|---|
Return type: | list of oslo_config.cfg.Opt instances |
get_description
()[source]Returns the description of the Scoring Engine.
The description might contain any human readable information, which might be useful for Strategy developers planning to use this Scoring Engine. It will be also visible in the Watcher API and CLI.
Returns: | A Scoring Engine description |
---|---|
Return type: | str |
get_metainfo
()[source]Returns the metadata information about Scoring Engine.
The metadata might contain a machine-friendly (e.g. in JSON format) information needed to use this Scoring Engine. For example, some Scoring Engines require to pass the array of features in particular order to be able to calculate the score value. This order can be defined in metadata and used in Strategy.
Returns: | A Scoring Engine metadata |
---|---|
Return type: | str |
get_name
()[source]Returns the name of the Scoring Engine.
The name should be unique across all Scoring Engines.
Returns: | A Scoring Engine name |
---|---|
Return type: | str |
Here below is the abstract ScoringContainer
class:
watcher.decision_engine.scoring.base.
ScoringEngineContainer
(config)[source]A base class for all the Scoring Engines Containers.
A Scoring Engine Container is an abstraction which allows to plugin multiple Scoring Engines as a single Stevedore plugin. This enables some more advanced scenarios like dynamic reloading of Scoring Engine implementations without having to restart any Watcher services.
get_config_opts
()[source]Defines the configuration options to be associated to this loadable
Returns: | A list of configuration options relative to this Loadable |
---|---|
Return type: | list of oslo_config.cfg.Opt instances |
get_scoring_engine_list
()[source]Returns a list of Scoring Engine instances.
Returns: | A list of Scoring Engine instances | ||
---|---|---|---|
Return type: |
|
In order for the Watcher Decision Engine to load your new scoring engine, it
must be registered as a named entry point under the watcher_scoring_engines
entry point of your setup.py
file. If you are using pbr, this entry point
should be placed in your setup.cfg
file.
The name you give to your entry point has to be unique and should be the same
as the value returned by the get_name()
method of your
strategy.
Here below is how you would proceed to register NewScorer
using pbr:
[entry_points]
watcher_scoring_engines =
new_scorer = thirdparty.new:NewScorer
To get a better understanding on how to implement a more advanced scoring
engine, have a look at the DummyScorer
class. This implementation
is not really using machine learning, but other than that it contains all the
pieces which the “real” implementation would have.
In addition, for some use cases there is a need to register a list (possibly
dynamic, depending on the implementation and configuration) of scoring engines
in a single plugin, so there is no need to restart Watcher Decision
Engine every time such list changes. For
these cases, an additional watcher_scoring_engine_containers
entry point
can be used.
For the example how to use scoring engine containers, please have a look at
the DummyScoringContainer
and the way it is configured in
setup.cfg
. For new containers it could be done like this:
[entry_points]
watcher_scoring_engine_containers =
new_scoring_container = thirdparty.new:NewContainer
The Watcher Decision Engine service will automatically discover any installed plugins when it is restarted. If a Python package containing a custom plugin is installed within the same environment as Watcher, Watcher will automatically make that plugin available for use.
At this point, Watcher will scan and register inside the Watcher Database all the scoring engines you implemented upon restarting the Watcher Decision Engine.
In addition, watcher-sync
tool can be used to trigger Watcher
Database synchronization. This might be used for
“dynamic” scoring containers, which can return different scoring engines based
on some external configuration (if they support that).
Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.