The watcher.decision_engine.model.collector.base Module¶

A Cluster Data Model (or CDM) is a logical representation of the current state and topology of the Cluster Managed resources.

It is represented as a set of Managed resources (which may be a simple tree or a flat list of key-value pairs) which enables Watcher Strategies to know the current relationships between the different resources) of the Cluster during an Audit and enables the Strategy to request information such as:

• What compute nodes are in a given Audit Scope?
• What Instances are hosted on a given compute node?
• What is the current load of a compute node?
• What is the current free memory of a compute node?
• What is the network link between two compute nodes?
• What is the available bandwidth on a given network link?
• What is the current space available on a given virtual disk of a given Instance ?
• What is the current state of a given Instance?
• …

In a word, this data model enables the Strategy to know:

• the current topology of the Cluster
• the current capacity for each Managed resource
• the current amount of used/free space for each Managed resource
• the current state of each Managed resources

In the Watcher project, we aim at providing a some generic and basic Cluster Data Model for each Goal, usable in the associated Strategies through a plugin-based mechanism which are called cluster data model collectors (or CDMCs). These CDMCs are responsible for loading and keeping up-to-date their associated CDM by listening to events and also periodically rebuilding themselves from the ground up. They are also directly accessible from the strategies classes. These CDMs are used to:

• simplify the development of a new Strategy for a given Goal when there already are some existing Strategies associated to the same Goal
• avoid duplicating the same code in several Strategies associated to the same Goal
• have a better consistency between the different Strategies for a given Goal
• avoid any strong coupling with any external Cluster Data Model (the proposed data model acts as a pivot data model)

There may be various generic and basic Cluster Data Models proposed in Watcher helpers, each of them being adapted to achieving a given Goal:

• For example, for a Goal which aims at optimizing the network resources the Strategy may need to know which resources are communicating together.
• Whereas for a Goal which aims at optimizing thermal and power conditions, the Strategy may need to know the location of each compute node in the racks and the location of each rack in the room.

Note however that a developer can use his/her own Cluster Data Model if the proposed data model does not fit his/her needs as long as the Strategy is able to produce a Solution for the requested Goal. For example, a developer could rely on the Nova Data Model to optimize some compute resources.

The Cluster Data Model may be persisted in any appropriate storage system (SQL database, NoSQL database, JSON file, XML File, In Memory Database, …). As of now, an in-memory model is built and maintained in the background in order to accelerate the execution of strategies.

class watcher.decision_engine.model.collector.base.BaseClusterDataModelCollector(config, osc=None)[source]¶

Bases: watcher.common.loader.loadable.LoadableSingleton

execute()[source]¶

Build a cluster data model

classmethod get_config_opts()[source]¶

get_latest_cluster_data_model()[source]¶

set_cluster_data_model_as_stale()[source]¶

synchronize()[source]¶

Synchronize the cluster data model

Whenever called this synchronization will perform a drop-in replacement with the existing cluster data model