Ocata Series Release Notes

3.0.1

New Features

  • A new recovery action “REBOOT” has been added to the health policy.

  • Added support to listen to heat event notifications for stack failure detection.

  • The numeric properties in the spec for a scaling policy now have stricter validations.

Bug Fixes

  • The bug where the availability zone info from a nova server deployment was not available has been fixed.

  • The parameter checking for the cluster update operation may incorrectly parse the provided value(s). This bug has been fixed.

3.0.0

New Features

  • A new API “cluster-op” is introduced to trigger a profile type specific operation on all nodes in a cluster. This API is available since API micro-version 1.4.

  • Docker container profile now supports operations like restart, pause and unpause.

  • A new, optional parameter “destroy_after_deletion” is added to the cluster-del-nodes request since API micro-version 1.4.

  • The health manager is improved to use dynamic timers instead of fix interval timers when polling cluster’s status.

  • Error messages returned from API requests are now unified. All parameter validation failures of the same reason returns a similar message.

  • A configuration option “exclude_derived_actions” is introduced into the “dispatchers” group for controlling whether derived actions should lead into event notifications and/or DB records.

  • A event_purge subcommand is added to senlin-manage tool for purging events generated in a specific project.

  • Health policy recovery actions now contains a list of dictionaries instead of a list of simple names. This is to make room for workflow invocations.

  • Many new operations are added to os.nova.server profile type. These operations can be shown using the “profile-type-ops” API.

  • Added new node-operation API for performing profile type supported operations on a node.

  • New API “node-op” is introduced for triggering profile type specific operations on a node. This is available since API micro-version 1.4.

  • Event notifications (versioned) are added to enable senlin-engine to send out messaging events when configured. The old event repo is adapted to follow the same design.

  • Versioned request support in API, RPC and engine layers.

  • Basic support for event/notification.

  • Enables osprofiler support.

  • Rally plugin for cluster scaling in.

  • Batch policy support for cluster actions.

  • Integration test for message receiver.

  • A new API “profile-type-ops” is introduced to expose the profile type specific operations’ schema to end users.

  • Integrated OSProfiler into Senlin, support using OSProfiler to measure performance of Senlin.

  • Profile type list and policy type list now returns the support status for each type since API micro-version 1.5.

  • RPC requests from the API service to the engine service are fully managed using versioned objects now. This will enable a smooth upgrade for the service in future.

Upgrade Notes

  • For resources which has a user, a project and a domain property, the lengths of these columns are increased from 32 chars to 64 chars for a better conformance with Keystone.

  • New setup configuration items are provided to enable the “message” and/or “database” event generation.

Critical Issues

  • The problem of having clusters or nodes still locked by actions executed by a dead engine is fixed.

Security Issues

  • Multi-tenancy is enhanced so that an admin role user has to respect project isolation unless explicitly asking for an exception.

Bug Fixes

  • Fixed the problem that health manager related configuration options were not properly exposed.

  • Removed LB_STATUS_POLLING from health policy since LBaaS still cannot provide reliable node status update.

  • The health policy recovery actions is designed to be a list but the current implementation can only handle one action. This is now explicitly checked.

  • Fixed the problem that the “updated_at” timestamp of a node was not correctly updated.

  • The notifications of profile type specific operations were not properly reporting the operation’s name. This has been fixed.

  • Fixed the notification logic so that it uses the proper transport obtained from oslo.messaging.

  • Fixed bug related to cluster-collect API where the path parameter is None.

  • When attaching a policy (especially a health policy) to a cluster, users may choose to keep the policy disabled. This has to be considered in the health manager and other places. This issue is fixed.

  • A nova server, if booted from volume, will not return a valid image ID. This situation is now taken care of.

Other Notes

  • The retrieval of some resources such as actions and policies are optimized to avoid object instantiation.

2.0.0

New Features

  • Improved the action scheduler so that it can decide how many node actions will be fired in each batch. Batch control is a throttling measure to avoid raising too many requests in a short interval to the backend services.

  • A new cluster_collect API is added.

  • The senlin-engine now supports fencing a corrupted VM instance by deleting it forcibly.

  • A new profile type ‘container.dockerinc.docker-1.0’ is added to support creation and management of docker clusters. This is still an experimental feature. Please use with caution.

  • The deletion policy is enhanced to handle ‘NODE_DELETE’ actions which derives from a standalone ‘node_delete’ request.

  • The cluster health manager has gained a new feature where nova server instance failures can be detected and handled, with and without a health policy attached to a cluster.

  • The health policy was improved so that it will suspend itself when a node deletion comes from senlin-engine or client request. The policy will only effect when node failure is ‘unexpected’.

  • The load-balancing policy is improved to handle ‘NODE_CREATE’ and ‘NODE_DELETE’ actions that derive from ‘node_create’ or ‘node_delete’ RPC requests directly.

  • A new “lb_status_timeout” option is added to the LB policy to cope with load-balancers that are not so responsive.

  • Added a new type of receiver (i.e. message) which is based on Zaqar message queue.

  • The region placement policy and the zone placement policy have been augmented with spec validation support.

  • The affinity policy is improved to handle NODE_CREATE actions which are derived from ‘node_create’ RPC requests.

  • The availability-zone placement policy is improved to handle NODE_CREATE actions which are derived from ‘node_create’ RPC requests.

  • The region placement policy is improved to handle the NODE_CREATE action which derives from a ‘node_create’ RPC request.

  • With the new ‘profile-validate’ API, the Nova server profile now supports the validation of its ‘flavour’, ‘image’ (if provided), ‘availability_zone’ and block device driver properties.

  • Added support to oslo.versionedobject so that DB interactions are abstracted. It is possible to do live upgrade for senlin service now.

  • A new policy-validate API has been added to validate the spec of a policy without actually creating an instance of it.

  • The affinity policy, loadbalancing policy now support spec validation. Invalid properties can be detected using policy-validate API.

  • A new profile-validate API has been added to validate the spec of a profile without actually creating an instance of it.

  • Engine scheduler was redesigned to work in “tickless” way.

  • Tempest API test for all Senlin API interfaces for both positive and negative cases.

  • Reimplement functional test using tempest.

  • Added ‘template_url’ support to heat stack profile.

  • Zaqar resources including “queue”, “message”, “subscription” and “claim” are now supported in Senlin driver.

Upgrade Notes

  • The cluster delete API calls may return a 409 status code if there are policies and/or receivers associated with it. Previously, we return a 400 status code.

  • DB columns obj_id, obj_type and obj_name in the event table are now renamed to oid, otype and oname correspondingly.

  • The ‘details/addresses’ property of a node output for a nova server used to contain only some trimed information. This has been changed to a faithful dumping of the ‘addresses’ property.

  • Several configuration options are consolidated into the ‘senlin_api’ group in ‘senlin.conf’ file (‘api_paste_config’, ‘wsgi_keep_alive’, ‘client_socket_timeout’, ‘max_json_body_size’).

  • With the newly added ‘message’ type of receivers, the ‘cluster’ and the ‘action’ property are not always required when creating a receiver. They are still required if the receiver type is ‘webhook’ (the default).

Security Issues

  • The configuration option ‘service_password’ is marked as secret so that its value won’t get leaked into log files.

Bug Fixes

  • Fixed a bug in affinity policy where the calls to nova driver was wrong.

  • The new API documentation include fixes to the header like ‘location’, ‘OpenStack-Request-Id’ and responses during version negotiation.

  • Fixed bug related to the desired_capacity calculation. The base number used now is the current capacity of the cluster instead of previous ‘desired’ capacity. This include all actions that change cluster capacity and all related policies.

  • The ‘desired_capacity’ reflects the expectation from a requester’s view point. The engine now changes the ‘desired_capacity’ after the request is validated/sanitized, before the action is actually implemented. This means the ‘desired_capacity’ will change event if an action fails.

  • Fixed cluster status update logic so that cluster status is solely determined by the status of its member nodes. The status is updated each time a cluster operation has completed.

  • Fix cluster next_index update when adding nodes to cluster.

  • Fixed DB layer dead lock issue that surfaced recently during concurrent DB operations.

  • Fixed resource delete operations which should return 204 status code with body length of zero.

  • Fixed error handling when network is not found in nova server creation.

  • Fixed node recover operation behavior so that unsupported operations can be detected and handled.

  • A cluster in the middle of an on-going action should not be deletable. The engine service has been improved to detect this situation.

  • The unimplemented properties for health policy are masked out.

  • The node action execution logic is fixed so that it will skip cluster checking for orphan nodes and policy checking will be skipped for derived node actions.

  • Fixed bug introduced by openstacksdk when updating nova server metadata.

  • Fixed dead service clean-up logic so that the clean-up operation can be retried.

  • The ‘senlin-manage’ command has been fixed so that it will report the senlin service status correctly.

  • The ‘tools/setup-service’ script has been fixed so that it works under keystone v3.

Other Notes

  • Senlin API/Engine configuration options are now documented and published online.