Scheduler¶
The manila.scheduler.manager
Module¶
Scheduler Service
- class SchedulerManager(scheduler_driver=None, service_name=None, *args, **kwargs)
Bases:
Manager
Chooses a host to create shares.
- RPC_API_VERSION = '1.11'
- create_share_group(context, share_group_id, request_spec=None, filter_properties=None)
- create_share_instance(context, request_spec=None, filter_properties=None)
- create_share_replica(context, request_spec=None, filter_properties=None)
- extend_share(context, share_id, new_size, reservations, request_spec=None, filter_properties=None)
- get_host_list(context)
Get a list of hosts from the HostManager.
- get_pools(context, filters=None, cached=False)
Get active pools from the scheduler’s cache.
- get_service_capabilities(context)
Get the normalized set of capabilities for this zone.
- init_host_with_rpc(service_id=None)
A hook for service to do jobs after RPC is ready.
Like init_host(), this method is a hook where services get a chance to execute tasks that need RPC. Child classes should override this method.
- Parameters:
service_id – ID of the service where the manager is running.
- manage_share(context, share_id, driver_options, request_spec, filter_properties=None)
Ensure that the host exists and can accept the share.
- migrate_share_to_host(context, share_id, host, force_host_assisted_migration, preserve_metadata, writable, nondisruptive, preserve_snapshots, new_share_network_id, new_share_type_id, request_spec, filter_properties=None)
Ensure that the host exists and can accept the share.
- request_service_capabilities(context)
- update_service_capabilities(context, service_name=None, host=None, capabilities=None, timestamp=None, **kwargs)
Process a capability update from a service node.
The manila.scheduler.base_handler
Module¶
A common base for handling extension classes.
Used by BaseFilterHandler and BaseWeightHandler
- class BaseHandler(modifier_class_type, modifier_namespace)
Bases:
object
Base class to handle loading filter and weight classes.
- get_all_classes()
The manila.scheduler.host_manager
Module¶
Manage hosts in the current zone.
- class HostManager
Bases:
object
Base HostManager class.
- get_all_host_states_share(context, consider_disabled=False)
Returns a dict of all the hosts the HostManager knows about.
Each of the consumable resources in HostState are populated with capabilities scheduler received from RPC.
- For example:
{‘192.168.1.100’: HostState(), …}
- get_filtered_hosts(hosts, filter_properties, filter_class_names=None)
Filter hosts and return only ones passing all filters.
- get_pools(context, filters=None, cached=False)
Returns a dict of all pools on all hosts HostManager knows about.
- get_weighed_hosts(hosts, weight_properties, weigher_class_names=None)
Weigh the hosts.
- host_state_cls
alias of
HostState
- update_service_capabilities(service_name, host, capabilities, timestamp)
Update the per-service capabilities based on this notification.
- class HostState(host, capabilities=None, service=None)
Bases:
object
Mutable and immutable information tracked for a host.
- consume_from_share(share)
Incrementally update host state from an share.
- update_backend(capability)
- update_capabilities(capabilities=None, service=None)
- update_from_share_capability(capability, service=None, context=None)
Update information about a host from its share_node info.
‘capability’ is the status info reported by share backend, a typical capability looks like this:
capability = { 'share_backend_name': 'Local NFS', # 'vendor_name': 'OpenStack', # backend level 'driver_version': '1.0', # mandatory/fixed 'storage_protocol': 'NFS', #/ stats&capabilities 'active_shares': 10, # 'IOPS_provisioned': 30000, # optional custom 'fancy_capability_1': 'eat', # stats & capabilities 'fancy_capability_2': 'drink', #/ 'pools':[ { 'pool_name': '1st pool', # 'total_capacity_gb': 500, # mandatory stats 'free_capacity_gb': 230, # for pools 'allocated_capacity_gb': 270, # | 'qos': 'False', # | 'reserved_percentage': 0, # | 'reserved_snapshot_percentage': 0, # | 'reserved_share_extend_percentage': 0, #/ 'dying_disks': 100, # 'super_hero_1': 'spider-man', # optional custom 'super_hero_2': 'flash', # stats & 'super_hero_3': 'neoncat', # capabilities 'super_hero_4': 'green lantern', #/ }, { 'pool_name': '2nd pool', 'total_capacity_gb': 1024, 'free_capacity_gb': 1024, 'allocated_capacity_gb': 0, 'qos': 'False', 'reserved_percentage': 0, 'reserved_snapshot_percentage': 0, 'reserved_share_extend_percentage': 0, 'dying_disks': 200, 'super_hero_1': 'superman', 'super_hero_2': 'Hulk', }] }
- update_pools(capability, service, context=None)
Update storage pools information from backend reported info.
- class PoolState(host, capabilities, pool_name)
Bases:
HostState
- update_from_share_capability(capability, service=None, context=None)
Update information about a pool from its share_node info.
- update_pools(capability)
Update storage pools information from backend reported info.
- class ReadOnlyDict(source=None)
Bases:
UserDict
A read-only dict.
- clear() None. Remove all items from D.
- pop(k[, d]) v, remove specified key and return the corresponding value.
If key is not found, d is returned if given, otherwise KeyError is raised.
- popitem() (k, v), remove and return some (key, value) pair
as a 2-tuple; but raise KeyError if D is empty.
- update([E, ]**F) None. Update D from mapping/iterable E and F.
If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v
The manila.scheduler.rpcapi
Module¶
Client side of the scheduler manager RPC API.
- class SchedulerAPI
Bases:
object
Client side of the scheduler rpc API.
API version history:
1.0 - Initial version. 1.1 - Add get_pools method 1.2 - Introduce Share Instances. Replace
create_share()
withcreate_share_instance()
1.3 - Add create_consistency_group method (renamed in 1.7) 1.4 - Add migrate_share_to_host method 1.5 - Add create_share_replica 1.6 - Add manage_share 1.7 - Updated migrate_share_to_host method with new parameters 1.8 - Rename create_consistency_group -> create_share_group method 1.9 - Add cached parameter to get_pools method 1.10 - Add timestamp to update_service_capabilities 1.11 - Add extend_share- RPC_API_VERSION = '1.11'
- create_share_group(context, share_group_id, request_spec=None, filter_properties=None)
Casts an rpc to the scheduler to create a share group.
Example of ‘request_spec’ argument value:
{ 'share_group_type_id': 'fake_share_group_type_id', 'share_group_id': 'some_fake_uuid', 'availability_zone_id': 'some_fake_az_uuid', 'share_types': [models.ShareType], 'resource_type': models.ShareGroup, }
- create_share_instance(context, request_spec=None, filter_properties=None)
- create_share_replica(context, request_spec=None, filter_properties=None)
- extend_share(context, share_id, new_size, reservations, request_spec, filter_properties=None)
- get_pools(context, filters=None, cached=False)
- manage_share(context, share_id, driver_options, request_spec=None, filter_properties=None)
- migrate_share_to_host(context, share_id, host, force_host_assisted_migration, preserve_metadata, writable, nondisruptive, preserve_snapshots, new_share_network_id, new_share_type_id, request_spec=None, filter_properties=None)
- update_service_capabilities(context, service_name, host, capabilities)
The manila.scheduler.scheduler_options
Module¶
SchedulerOptions monitors a local .json file for changes and loads it if needed. This file is converted to a data structure and passed into the filtering and weighing functions which can use it for dynamic configuration.
- class SchedulerOptions
Bases:
object
Monitor and load local .json file for filtering and weighing.
SchedulerOptions monitors a local .json file for changes and loads it if needed. This file is converted to a data structure and passed into the filtering and weighing functions which can use it for dynamic configuration.
- get_configuration(filename=None)
Check the json file for changes and load it if needed.
The manila.scheduler.drivers.filter
Module¶
The FilterScheduler is for scheduling of share and share group creation. You can customize this scheduler by specifying your own share/share group filters and weighing functions.
- class FilterScheduler(*args, **kwargs)
Bases:
Scheduler
Scheduler that can be used for filtering and weighing.
- get_pools(context, filters, cached)
Must override schedule method for scheduler to work.
- host_passes_filters(context, host, request_spec, filter_properties)
Must override schedule method for migration to work.
- populate_filter_properties_scheduler_hints(context, request_spec, filter_properties)
- populate_filter_properties_share(context, request_spec, filter_properties)
Stuff things into filter_properties.
Can be overridden in a subclass to add more data.
- schedule_create_replica(context, request_spec, filter_properties)
Must override schedule method for create replica to work.
- schedule_create_share(context, request_spec, filter_properties)
Must override schedule method for scheduler to work.
- schedule_create_share_group(context, share_group_id, request_spec, filter_properties)
Must override schedule method for scheduler to work.
The manila.scheduler.drivers.base
Module¶
Scheduler base class that all Schedulers should inherit from
- class Scheduler
Bases:
object
The base class that all Scheduler classes should inherit from.
- get_host_list()
Get a list of hosts from the HostManager.
- get_pools(context, filters)
Must override schedule method for scheduler to work.
- get_service_capabilities()
Get the normalized set of capabilities for the services.
- host_passes_filters(context, host, request_spec, filter_properties)
Must override schedule method for migration to work.
- hosts_up(context, topic)
Return the list of hosts that have a running service for topic.
- schedule(context, topic, method, *_args, **_kwargs)
Must override schedule method for scheduler to work.
- schedule_create_replica(context, request_spec, filter_properties)
Must override schedule method for create replica to work.
- schedule_create_share(context, request_spec, filter_properties)
Must override schedule method for scheduler to work.
- schedule_create_share_group(context, share_group_id, request_spec, filter_properties)
Must override schedule method for scheduler to work.
- update_service_capabilities(service_name, host, capabilities, timestamp)
Process a capability update from a service node.
- share_group_update_db(context, share_group_id, host)
Set the host and set the updated_at field of a share group.
- Returns:
A share group with the updated fields set properly.
- share_replica_update_db(context, share_replica_id, host)
Set the host and the scheduled_at field of a share replica.
- Returns:
A Share Replica with the updated fields set.
- share_update_db(context, share_id, host)
Set the host and set the scheduled_at field of a share.
- Returns:
A Share with the updated fields set properly.
The manila.scheduler.drivers.chance
Module¶
Chance (Random) Scheduler implementation
- class ChanceScheduler
Bases:
Scheduler
Implements Scheduler as a random node selector.
- schedule_create_share(context, request_spec, filter_properties)
Picks a host that is up at random.
The manila.scheduler.drivers.simple
Module¶
Simple Scheduler
- class SimpleScheduler
Bases:
ChanceScheduler
Implements Naive Scheduler that tries to find least loaded host.
- schedule_create_share(context, request_spec, filter_properties)
Picks a host that is up and has the fewest shares.
Scheduler Filters¶
The manila.scheduler.filters.availability_zone
Filter¶
- class AvailabilityZoneFilter
Bases:
BaseHostFilter
Filters Hosts by availability zone.
- host_passes(host_state, filter_properties)
Return True if the HostState passes the filter, otherwise False.
Override this in a subclass.
- run_filter_once_per_request = True
The manila.scheduler.filters.base
Filter¶
Filter support
- class BaseFilter
Bases:
object
Base class for all filter classes.
- filter_all(filter_obj_list, filter_properties)
Yield objects that pass the filter.
Can be overridden in a subclass, if you need to base filtering decisions on all objects. Otherwise, one can just override _filter_one() to filter a single object.
- run_filter_for_index(index)
Check if filter needs to be run for the “index-th” instance.
Return True if the filter needs to be run for the “index-th” instance in a request. Only need to override this if a filter needs anything other than “first only” or “all” behaviour.
- run_filter_once_per_request = False
- class BaseFilterHandler(modifier_class_type, modifier_namespace)
Bases:
BaseHandler
Base class to handle loading filter classes.
This class should be subclassed where one needs to use filters.
- get_filtered_objects(filter_classes, objs, filter_properties, index=0)
Get objects after filter
- Parameters:
filter_classes – filters that will be used to filter the objects
objs – objects that will be filtered
filter_properties – client filter properties
index – This value needs to be increased in the caller function of get_filtered_objects when handling each resource.
The manila.scheduler.filters.base_host
Filter¶
Scheduler host filters
- class BaseHostFilter
Bases:
BaseFilter
Base class for host filters.
- host_passes(host_state, filter_properties)
Return True if the HostState passes the filter, otherwise False.
Override this in a subclass.
- class HostFilterHandler(namespace)
Bases:
BaseFilterHandler
The manila.scheduler.filters.capabilities
Filter¶
- class CapabilitiesFilter
Bases:
BaseHostFilter
HostFilter to work with resource (instance & volume) type records.
- host_passes(host_state, filter_properties)
Return a list of hosts that can create resource_type.
The manila.scheduler.filters.capacity
Filter¶
- class CapacityFilter
Bases:
BaseHostFilter
CapacityFilter filters based on share host’s capacity utilization.
- host_passes(host_state, filter_properties)
Return True if host has sufficient capacity.
The manila.scheduler.filters.extra_specs_ops
Filter¶
- match(value, req)
The manila.scheduler.filters.ignore_attempted_hosts
Filter¶
- class IgnoreAttemptedHostsFilter
Bases:
BaseHostFilter
Filter out previously attempted hosts
A host passes this filter if it has not already been attempted for scheduling. The scheduler needs to add previously attempted hosts to the ‘retry’ key of filter_properties in order for this to work correctly. For example:
{ 'retry': { 'hosts': ['host1', 'host2'], 'num_attempts': 3, } }
- host_passes(host_state, filter_properties)
Skip nodes that have already been attempted.
The manila.scheduler.filters.json
Filter¶
- class JsonFilter
Bases:
BaseHostFilter
Host Filter to allow simple JSON-based grammar for selecting hosts.
- commands = {'<': <function JsonFilter._less_than>, '<=': <function JsonFilter._less_than_equal>, '=': <function JsonFilter._equals>, '>': <function JsonFilter._greater_than>, '>=': <function JsonFilter._greater_than_equal>, 'and': <function JsonFilter._and>, 'in': <function JsonFilter._in>, 'not': <function JsonFilter._not>, 'or': <function JsonFilter._or>}
- host_passes(host_state, filter_properties)
Filters hosts.
Return a list of hosts that can fulfill the requirements specified in the query.
The manila.scheduler.filters.retry
Filter¶
- class RetryFilter
Bases:
BaseHostFilter
Filter out already tried nodes for scheduling purposes.
- host_passes(host_state, filter_properties)
Skip nodes that have already been attempted.
Scheduler Weighers¶
The manila.scheduler.weighers.base
Weigher¶
Pluggable Weighing support
- class BaseWeigher
Bases:
object
Base class for pluggable weighers.
The attributes maxval and minval can be specified to set up the maximum and minimum values for the weighed objects. These values will then be taken into account in the normalization step, instead of taking the values from the calculated weighers.
- maxval = None
- minval = None
- weigh_objects(weighed_obj_list, weight_properties)
Weigh multiple objects.
Override in a subclass if you need access to all objects in order to calculate weighers. Do not modify the weight of an object here, just return a list of weighers.
- weight_multiplier()
How weighted this weigher should be.
Override this method in a subclass, so that the returned value is read from a configuration option to permit operators specify a multiplier for the weigher.
- class BaseWeightHandler(modifier_class_type, modifier_namespace)
Bases:
BaseHandler
- get_weighed_objects(weigher_classes, obj_list, weighing_properties)
Return a sorted (descending), normalized list of WeighedObjects.
- object_class
alias of
WeighedObject
- class WeighedObject(obj, weight)
Bases:
object
Object with weight information.
- normalize(weight_list, minval=None, maxval=None)
Normalize the values in a list between 0 and 1.0.
The normalization is made regarding the lower and upper values present in weight_list. If the minval and/or maxval parameters are set, these values will be used instead of the minimum and maximum from the list.
If all the values are equal, they are normalized to 0.
The manila.scheduler.weighers.base_host
Weigher¶
Scheduler host weighers
- class BaseHostWeigher
Bases:
BaseWeigher
Base class for host weighers.
- class HostWeightHandler(namespace)
Bases:
BaseWeightHandler
- object_class
alias of
WeighedHost
- class WeighedHost(obj, weight)
Bases:
WeighedObject
- to_dict()
The manila.scheduler.weighers.capacity
Weigher¶
Capacity Weigher. Weigh hosts by their virtual or actual free capacity.
For thin provisioning, weigh hosts by their virtual free capacity calculated by the total capacity multiplied by the max over subscription ratio and subtracting the provisioned capacity; Otherwise, weigh hosts by their actual free capacity, taking into account the reserved space.
The default is to spread shares across all hosts evenly. If you prefer stacking, you can set the ‘capacity_weight_multiplier’ option to a negative number and the weighing has the opposite effect of the default.
- class CapacityWeigher
Bases:
BaseHostWeigher
- weigh_objects(weighed_obj_list, weight_properties)
Weigh multiple objects.
Override in a subclass if you need access to all objects in order to calculate weighers. Do not modify the weight of an object here, just return a list of weighers.
- weight_multiplier()
Override the weight multiplier.
The manila.scheduler.weighers.pool
Weigher¶
- class PoolWeigher
Bases:
BaseHostWeigher
- weight_multiplier()
Override the weight multiplier.