Newton Serie Releasenotes

3.0.0-15

Bug Fixes

• Previously Cinder Volumes created in MuranoPL were not released correctly on object destruction. The issue is now fixed.

3.0.0

New Features

• Added a new manifest format 1.4.0. Introduced the ‚Scope‘ keyword for class methods to declare a method’s accessibility from outside through the API call.

• Implemented the capability for API endpoint /catalog/packages to filter ‚id‘, ‚category‘, ‚tag‘ properties using the ‚in‘ operator. An example of using the ‚in‘ operator for ‚id‘ is ‚id=in:id1,id2,id3‘. This filter is added using syntax that conforms to the latest guidelines from the OpenStack API-WG.

• Added the timeout parameter to runCommand and putFile methods of the io.murano.configuration.Linux class.

• Added the driver configuration option to the networking group. It allows to explicitly select the networking driver. It supports ‚neutron‘ and ‚nova‘ options. If set to None (default), murano attempts to use ‚neutron‘ if available, ‚nova‘ otherwise. The change is backward compatible.

• Added the description_text field to environment and environment templates database tables and respective API objects.

• Introduced a new MuranoPL class io.murano.system.GC Now MuranoPL garbage collector can be used to set up destruction dependencies between murano objects. If object Foo is subscribed to object Bar’s destruction, it will be notified through a specific handler. If both Foo and Bar are going to be destroyed during one execution session, Foo will be destroyed after Bar. You can omit the handler, in this case destruction order will also be preserved. Handler can be a static or a usual function.

• New on-request garbage collector for MuranoPL objects were implemented. Garbage collection is triggered by io.murano.system.GC.collect() static method. Garbage collector destroys all object that are not reachable anymore. GC can handle objects with cross-references and isolated object graphs. When portion of object model becomes not reachable it destroyed in predictable order such that child objects get destroyed before their parents and, when possible, before objects that are subscribed to their destruction notifications.

• Internally, both pre-deployment garbage collection (that was done by comparision of Objects and ObjectsCopy) and post-deployment orphan object collection are now done through the new GC.

• io.murano.system.GC.isDoomed() static method was added. It can be used within the .destroy method to test if other object is also going to be destroyed.

• io.murano.system.GC.isDestroyed() static method was added. It checks if the object is destroyed and thus no methods can be invoked on it.

• Implemented the capability for the helper methods of Linux class to run concurrently if executed for different VM agents.

• Added the following meta-classes to the core library - Title Description HelpText Hidden Section Position ModelBuilder. These classes will later be used to implement dynamic object model generation.

• Added an overload of the new function - new($model, $owner). It loads complete object graph in a single call. Objects in the model can have cross references. In that case, this is the only way to instantiate the graph. Objects might be specified either in object model format (with ‚?‘ attribute or in MuranoPL format (used for Meta definitions).

• The contract class() now uses the same approach to load classes from dictionaries. Thus the same two syntaxes apply there as well.

• Added Support for application deployment across OpenStack regions. Now, all OpenStack resource classes inherit from io.murano.CloudResource that provides .getRegion() method and regionName property. This allows to assign resources to different regions. .getRegion() returns io.murano.CloudRegion instance that resource or its parent belongs to. CloudRegion has interface similar to Environment class and is the correct way to get HeatStack instance associated with the region, default network configuration, security group manager and agent listener instances. Environment now acts as default region so backward compatibility is not broken. However new applications should not use environment to set security group rules but rather a region(s) of their instance(s) in order to work correctly when their instances were configured to use region other than the default.

• Added the api_workers option to murano config group. It controls the number of API workers launched by murano. If not set, it would default to the number of CPUs available.

• Added a new engine RPC call to generate json-schema from MuranoPL class. The schema may be generated either from the entire class or for specific model builders - static actions that can be used to generate object model from their input. Class schema is built by inspecting class properties and method schema using the same algorithm but applied to its arguments.

• Implemented a new framework for MuranoPL contracts. Now, instead of several independent implementations of the same yaql methods (string(), class() etc.) all implementations of the same method are combined into single class. Therefore, we now have a class per contract method. This also simplifies development of new contracts. Each such class can provide methods for data transformation (default contract usage), validation that is used to decide if the method can be considered an extension method for the value, and json schema generation method that was moved from the schema generator script.

• Previously, when a class overrode a property from its parent class the value was stored separately for both of them, transformed by each of the contracts. Thus each class saw the value of its contract. In absolute majority of the cases, the observed value was the same. However, if the contracts were compatible on the provided value (say int() and string() contracts on the value „123“) they were different. This is considered to be a bad pattern. Now, the value is stored only once per object and transformed by the contract defined in the actual object type. All base contracts are used to validate the transformed object thus this pattern will not work anymore.

• The value that is stored in the object’s properties is obtained by executing special „finalize“ contract implementation which by default returns the input value unmodified. Because validation happens on the transformed value before it gets finalized it is possible for transformation to return a value that will pass the validation though the final value won’t. This is used to relax the template() contract limitation that prevented child class from excluding additional properties from the template.

• The string() contract no longer converts everything to string values. Now it only converts scalar values to strings. Previous behavior allowed string() property to accept lists and convert them to their Python string representation which is clearly not what developers expected.

• Due to refactoring, contracts work a little bit faster because there is no more need to generate yaql function definition for each contract method on each call.

• Changed the type representation in object model. Previous format was to have three attributes in „?“ section of the object - type, classVersion and package where only the „type“ is mandatory. Now they are merged into single attribute „type“ that has a format typeName/version@package. Version and package parts are still optional.

• Previously, when pre-deployment garbage collection occurred it executed .destroy method for objects that were present in the ObjectsCopy section of the object model (which is the the snapshot of the model after last deployment) and not present in the current model anymore (because they were deleted through the API between deployments). If the destroyed objects were to access another object that was not deleted it was accessing its copy from the ObjectsCopy. Thus any changes to the internal state made by that object were lost after the garbage collection finished (that is, before the .deploy method call) and could not affect the deployment. Now, if the object is present in both Objects and ObjectsCopy, a single instance (the one from Objects) is used for both garbage collection and deployment. As a consequence, instances (in their .destroy method) now may observe changes made to other objects they refer if they were not deleted, but modified through the API. In some rare cases, it may break existing applications.

• Separated murano service broker from murano-api into a murano-cfapi service. Created a separate database and paste.ini for service broker.

• Added a new API endpoint v1/actions to call static public methods. It accepts class name, method name, method arguments, and optionally package name and class version in the request body. This call does not create an environment, object instances or database records.

• Implemented a new contract function template(). template() works similar to the class() in regards to the data validation but does not instantiate objects. Instead, the data is left in the object model in dictionary format so that it could be instantiated later with the new() function. Additionally, the function allows excluding specified properties from validation and from the resulting template so that they could be provided later. Objects that are assigned to the property or argument with template() contract will be automatically converted to their object model representation.

• Split Instance’s .deploy() method into two phases - beginDeploy() and endDeploy(). This allows the application developer to provision multiple instances at once without the need to push the stack for each instance.

• Added API endpoint /templates/{env_template_id}/services/{path:.*?} for environment template application update operation.

• Added the capability to declare MuranoPL YAML methods with variable length positional and keyword arguments. This is done using argument Usage attribute. Regular arguments have Standard usage which is the default. Variable length args (args in Python) should have „Usage: VarArgs“ and keyword args (kwargs) are declared with „Usage: KwArgs“. Inside the method they are seen as a list and a dictionary correspondingly. For such arguments contracts are written for individual argument values thus no need to write them as lists/dicts.

Upgrade Notes

• New database migration 015 has to be applied.

Deprecation Notes

• Deprecated the ‚Usage Action‘ keyword. For format versions >= 1.4.0, use ‚Scope Public‘ instead.

• Renamed the workers option from the engine group to engine_workers to reduce ambiguity with the api_workers option.

Security Issues

• cve-2016-4972 has been addressed. In ceveral places Murano used loaders inherited directly from yaml.Loader when parsing MuranoPL and UI files from packages. This is unsafe, because this loader is capable of creating custom python objects from specifically constructed yaml files. With this change all yaml loading operations are done using safe loaders instead.

Bug Fixes

• Core Library’s init scripts used to have various problems detecting pre-installed (by DIB) murano-agent on non-ubuntu images. Agent setup script now checks wider list of directories before attempting to install murano-agnet and service script now does not impose strict script location.

• API call for deleting a service from environment template did not return result of its operation. The issue is fixed.

• Fixed a bug when the UI dialog was not displayed in Murano Dashboard for applications which don’t have UI definitions bundled in the package but generate them based on the package contents instead. This usually affected HOT-based packages and other non-muranopl-based applications.

• Removed the need for Keystone v2 options (admin_user, admin_password, admin_tenant_name) when Keystone v3 is in use.

• Previously murano assumed that the service user and service project are in the ‚Default‘ domain. These values can now be set in keystone_authtoken config group.

• Equality check (assertEqual) in test-runner can now properly compare two MuranoPl objects.

• Prevented the resource leak for objects created during deployment with new() function call.

• Murano is now able to deploy applications in the environments with disabled Neutron Security Groups. Detection is based on the presence of ‚security-group‘ Neutron extension.

• Whenever murano-engine accesses script files, text script files are opened in ‚rU‘ mode which recognizes all types of newlines, and binary files are opened in ‚rb‘ mode to prevent their corruption.

• It is now possible to use version specifications like ‚=0.0.0‘ when semantic_version library version ‚2.3.1‘ is installed. Previously such specifications caused an error and ‚==0.0.0‘ had to be used.

• Murano engine no longer logs methods string(), json(), and yaml() of the ‚io.murano.system.Resources‘ class. This is done to prevent UnicodeDecodeError’s when transferring binary files to murano agent.

• The test-runner now outputs the tests it runs and their results to stdout directly, instead of the logging system.

• The test-runner now does not output logs to stderr by default unless a ‚use_stderr‘ parameter is specified in the configuration file.

• Fixed the issue that prevented the test-runner from properly invoking setUp and tearDown methods of fixtures in some cases.