Queens Series Release Notes

0.11.3

Bug Fixes

• Fixed an issue where some valid query parameters were not listed causing errors due to the new behavior of throwing errors when an invalid filter condition is specified. Specifically, tenant_id as a filter for Neutron networks and ip_version for Neutron netork IP availability.

0.11.0

New Features

• Added a new method openstack.connection.Connection.add_service which allows the registration of Proxy/Resource classes defined externally.

• availability_zone_hints now accepted for create_network() when network_availability_zone extension is enabled on target cloud.

• availability_zone_hints now accepted for create_router() when router_availability_zone extension is enabled on target cloud.

Upgrade Notes

• Removed the metric service. It is not an OpenStack service and does not have an entry in service-types-authority.

• Meter and Alarm services have been removed. The Ceilometer REST API has been deprecated for quite some time and is no longer supported.

• The Resource2 and Proxy2 migration has been completed. The original Resource and Proxy clases have been removed and replaced with Resource2 and Proxy2.

Deprecation Notes

• openstack.profile.Profile has been deprecated and will be removed in the 1.0 release. Users should use the functions in openstack.config instead.

• The shade functionality that has been merged in to openstacksdk is found in openstack.cloud currently. None of these interfaces should be relied upon as the merge has not yet completed.

Other Notes

• Add betacloud region for Germany

0.10.0

Prelude

The shade and os-client-config libraries have been merged in to openstacksdk. As a result, their functionality is being integrated into the sdk functionality, and in some cases is replacing exisiting things. The openstack.profile.Profile and openstack.auth.base.BaseAuthPlugin classes are no more. Profile has been replace by openstack.config.cloud_region.CloudRegion from os-client-config openstack.auth.base.BaseAuthPlugin has been replaced with the Auth plugins from keystoneauth. Service proxy names on the openstack.connection.Connection are all based on the official names from the OpenStack Service Types Authority. openstack.proxy.Proxy is now a subclass of keystoneauth1.adapter.Adapter. Removed local logic that duplicates keystoneauth logic. This means every proxy also has direct REST primitives available.

connection = connection.Connection()
servers = connection.compute.servers()
server_response = connection.compute.get('/servers')

New Features

• Added a new property, ‘current_user_id’ which contains the id of the currently authenticated user from the token.

• All get and search functions can now take a jmespath expression in their filters parameter.

• Add a list_flavor_access method to list all the projects/tenants allowed to access a given flavor.

• Added get_server_console method to fetch the console log from a Server. On clouds that do not expose this feature, a debug line will be logged and an empty string will be returned.

• Added flag “show_all” to list_images. The behavior of Glance v2 to only show shared images if they have been accepted by the user can be confusing, and the only way to change it is to use search_images(filters=dict(member_status=’all’)) which isn’t terribly obvious. “show_all=True” will set that flag, as well as disabling the filtering of images in “deleted” state.

• Add description parameter to create_user, available on Keystone v3

• Add support for Designate recordsets resources, with the usual methods (search/list/get/create/update/delete).

• Add support for Designate zones resources, with the usual methods (search/list/get/create/update/delete).

• Add tags support when creating a stack, as specified by the openstack orchestration api at [1]

  [1]https://developer.openstack.org/api-ref/orchestration/v1/#create-stack

• Add support for host aggregates and host aggregate membership.

• Add support for Magnum baymodels, with the usual methods (search/list/get/create/update/delete). Due to upcoming rename in Magnum from baymodel to cluster_template, the shade functionality uses the term cluster_template. However, baymodel aliases are provided for each api call.

• Add support for listing Magnum services.

• Adds support to create and delete server groups.

• Add update_server method to update name or description of a server.

• Add the ability to update a keystone service information. This feature is not available on keystone v2.0. The new function, update_service(), allows the user to update description, name of service, service type, and enabled status.

• Added methods for making new cloud connections based on the current OpenStackCloud. This should enable working more easily across projects or user accounts.

• Added group parameter to create_server to allow booting a server into a specific server group.

• Swiftclient instantiation now provides authentication information so that long lived swiftclient objects can reauthenticate if necessary.

• Add support for explicit v2password auth type.

• Add SSL support to VEXXHOST vendor profile.

• Add zetta.io cloud vendor profile.

• Add support for Cinder volume backup resources, with the usual methods (search/list/get/create/delete).

• If shade has to create objects in swift to upload an image, it will now delete those objects upon successful image creation as they are no longer needed. They will also be deleted on fatal import errors.

• Add a field to vendor cloud profiles to indicate active, deprecated and shutdown status. A message to the user is triggered when attempting to use cloud with either deprecated or shutdown status.

• Add new APIs, OperatorCloud.get_compute_quotas(), OperatorCloud.set_compute_quotas() and OperatorCloud.delete_compute_quotas() to manage nova quotas for projects and users

• get_compute_usage now has a default value for the start parameter of 2010-07-06. That was the date the OpenStack project started. It’s completely impossible for someone to have Nova usage data that goes back further in time. Also, both the start and end date parameters now also accept strings which will be parsed and timezones will be properly converted to UTC which is what Nova expects.

• Adds ability to add a config setting to clouds.yaml to disable fetching extra_specs from flavors.

• Explicit data model contracts are now defined for Flavors, Images, Security Groups, Security Group Rules, and Servers.

• Resources with data model contracts are now being returned with ‘location’ attribute. The location carries cloud name, region name and information about the project that owns the resource.

• Added new method, delete_autocreated_image_objects that can be used to delete any leaked objects shade may have created on behalf of the user.

• Added name_or_id parameter to domain operations, allowing an admin to update/delete/get by domain name.

• Added support for dual stack networks where the IPv4 subnet and the IPv6 subnet have opposite public/private qualities. It is now possible to add configuration to clouds.yaml that will indicate that a network is public for v6 and private for v4, which is otherwise very difficult to correctly infer while setting server attributes like private_v4, public_v4 and public_v6.

• Add new method, ‘endpoint_for’ which will return the raw endpoint for a given service from the current catalog.

• Add new APIs, OpenStackCloud.set_server_metadata() and OpenStackCloud.delete_server_metadata() to manage metadata of existing nova compute instances

• Flavors will always contain an ‘extra_specs’ attribute. Client cruft, such as ‘links’, ‘HUMAN_ID’, etc. has been removed.

• name_or_id parameters to search/get methods now support filename-like globbing. This means search_servers(‘nb0*’) will return all servers whose names start with ‘nb0’.

• Allow to retrieve the limits of a specific project

• Allow to retrieve the usage of a specific project

• Added a new API call, OpenStackCloud.get_object(), to download objects from swift.

• add granting and revoking of roles from groups and users

• The image and flavor parameters for create_server now accept name in addition to id and dict. If given as a name or id, shade will do a get_image or a get_flavor to find the matching image or flavor. If you have an id already and are not using any caching and the extra lookup is annoying, passing the id in as “dict(id=’my-id’)” will avoid the lookup.

• Added ability to create an image from a volume.

• If a cloud does not have a neutron service, it is now assumed that Nova will be the source of security groups. To handle clouds that have nova-network and do not have the security group extension, setting secgroup_source to None will prevent attempting to use them at all. If the cloud has neutron but it is not a functional source of security groups, set secgroup_source to nova.

• Add support for passing Ironic microversion to the ironicclient constructor in get_legacy_client.

• Added list_availability_zone_names API call.

• Implement list_role_assignments for keystone v2, using roles_for_user.

• Add ‘all_projects’ parameter to list_servers and search_servers which will tell Nova to return servers for all projects rather than just for the current project. This is only available to cloud admins.

• Added a flag, ‘load_yaml_config’ that defaults to True. If set to false, no clouds.yaml files will be loaded. This is beneficial if os-client-config wants to be used inside of a service where end-user clouds.yaml files would make things more confusing.

• Add version argument to make_rest_client and plumb version discovery through get_session_client so that versioned endpoints are properly found if unversioned are in the catalog.

• create_object() now has a “metadata” parameter that can be used to create an object with metadata of each key and value pair in that dictionary

• Add an update_object() function that updates the metadata of a swift object

• Added a parameter to create_image ‘meta’ which allows for providing parameters to the API that will not have any type conversions performed. For the simple case, the existing kwargs approach to image metadata is still the best bet.

• Add min_version and max_version to get_legacy_client and to get_session_endpoint. At the moment this is only really fully plumbed through for cinder, which has extra special fun around volume, volumev2 and volumev3. Min and max versions to both methods will look through the options available in the service catalog and try to return the latest one available from the span of requested versions. This means a user can say volume_api_version=None, min_version=2, max_version=3 will get an endpoint from get_session_endpoint or a Client from cinderclient that will be either v2 or v3 but not v1. In the future, min and max version for get_session_endpoint should be able to sort out appropriate endpoints via version discovery, but that does not currently exist.

• Removed unneeded calls that were made when deleting servers with floating ips.

• Added pagination support for volume listing.

• Added nat_source flag for networks. In some more complex clouds there can not only be more than one valid network on a server that NAT can attach to, there can also be more than one valid network from which to get a NAT address. Allow flagging a network so that it can be found.

• Network provider options are now accepted in create_network().

• Support added for configuring metadata about networks for a cloud in a list of dicts, rather than in the external_network and internal_network entries. The dicts support a name, a routes_externally field, a nat_destination field and a default_interface field.

• Add new APIs, OperatorCloud.get_network_quotas(), OperatorCloud.set_network_quotas() and OperatorCloud.delete_network_quotas() to manage neutron quotas for projects and users

• Added support for created_at, updated_at, description and revision_number attributes for floating ips.

• Image dicts that are returned are now normalized across glance v1 and glance v2. Extra key/value properties are now both in the root dict and in a properties dict. Additionally, cloud and region have been added like they are for server.

• Added helper method for constructing OpenStack SDK Connection objects.

• server creation errors now include the server id in the Exception to allow people to clean up.

• Add the add_server_security_groups and remove_server_security_groups functions to add and remove security groups from a specific server.

• Added kwargs and argparse processing for session_client.

• Added a set_volume_bootable call to allow toggling the bootable state of a volume.

• Added helper method for constructing shade OpenStackCloud objects.

• Implement update_stack to perform the update action on existing orchestration stacks.

• get_object now supports streaming output directly to a file.

• Added ‘strict’ mode, which is set by passing strict=True to the OpenStackCloud constructor. strict mode tells shade to only return values in resources that are part of shade’s declared data model contract.

• Added update_endpoint as a new function that allows the user to update a created endpoint with new values rather than deleting and recreating that endpoint. This feature only works with keystone v3, with v2 it will raise an exception stating the feature is not available.

• Version discovery is now done via the keystoneauth library. shade still has one behavioral difference from default keystoneauth behavior, which is that shade will use a version it understands if it can find one even if the user has requested a different version. This change opens the door for shade to start being able to consume API microversions as needed.

• Add new APIs, OperatorCloud.get_volume_quotas(), OperatorCloud.set_volume_quotas() and OperatorCloud.delete_volume_quotas() to manage cinder quotas for projects and users

• Add support for listing volume types.

• Add support for managing volume type access.

• Adds a new pair of options to create_image_snapshot(), wait and timeout, to have the function wait until the image snapshot being created goes into an active state.

• Adds a new function wait_for_image() which will wait for an image to go into an active state.

• New wait_for_server() API call to wait for a server to reach ACTIVE status.

Known Issues

• If there was only one cloud defined in clouds.yaml os-client-config was requiring the cloud parameter be passed. This is inconsistent with how the envvars cloud works which WILL work without setting the cloud parameter if it’s the only cloud.

• Fixed a regression when using latest os-client-config with the keystoneauth from stable/newton. Although this isn’t a super common combination, the added feature that broke the interaction is really not worthy of the incompatibility, so a workaround was added.

• Fixed an issue where nodepool could cause config_drive to be passed explicitly as None, which was getting directly passed through to the JSON. Also fix the same logic for key_name and scheduler_hints while we’re in there.

• Fixed the logic in delete_ips and added regression tests to cover it. The old logic was incorrectly looking for floating ips using port syntax. It was also not swallowing errors when it should.

• Images in the cloud with a string property named “properties” caused image normalization to bomb.

• Fixed an issue where glance image list pagination was being ignored, leading to truncated image lists.

Upgrade Notes

• Cluster Templates have data model and normalization now. As a result, the detail parameter is now ignored and detailed records are always returned.

• The attach_volume method now always returns a volume_attachment object. Previously, attach_volume would return a volume object if it was called with wait=True and a volume_attachment object otherwise.

• shade will now only generate file hashes for glance images if both hashes are empty. If only one is given, the other will be treated as an empty string.

• Removed designateclient as a dependency. All designate operations are now performed with direct REST calls using keystoneauth Adapter.

• Server creation calls are now done with direct REST calls.

• troveclient is no longer a hard dependency. Users who were using shade to construct a troveclient Client object should use os_client_config.make_legacy_client instead.

• Nova flavor operations are now handled via REST calls instead of via novaclient. There should be no noticable difference.

• Nova microversion is being requested. Since shade is not yet actively microversion aware, but has been dealing with the 2.0 structures anyway, this should not affect anyone.

• magnumclient is no longer a direct dependency as magnum API calls are now made directly via REST.

• All Nova interactions are done via direct REST calls. python-novaclient is no longer a direct dependency of openstack.cloud.

• The Profile object has been replaced with the use of CloudRegion objects from openstack.config.

• The openstacksdk specific Session object has been removed.

• Proxy objects are now subclasses of keystoneauth1.adapter.Adapter.

• REST interactions all go through TaskManager now.

• Removed swiftclient as a dependency. All swift operations are now performed with direct REST calls using keystoneauth Adapter.

• Renamed bare-metal to baremetal to align with the official service type.

• The block_store service object has been renamed to block_storage to align the API with the official service types.

• Renamed cluster to clustering to align with the official service type.

• Renamed telemetry to meter to align with the official service type.

• If your cloud presents a default split IPv4/IPv6 stack with a public v6 and a private v4 address and you have the expectation that auto_ip should procure a v4 floating ip, you need to set ‘force_ipv4’ to True in your clouds.yaml entry for the cloud.

• keystoneauth version 3.2.0 or higher is required because of version discovery.

Deprecation Notes

• Renamed session_client to make_rest_client. session_client will continue to be supported for backwards compatability.

• external_network and internal_network are deprecated and should be replaced with the list of network dicts.

• Renamed simple_client to session_client. simple_client will remain as an alias for backwards compat.

Bug Fixes

• [bug 2001080] Project update will only update the enabled field of projects when enabled=True or enabled=False is passed explicitly. The previous behavior had enabled=True as the default.

• Fixed caching the volume list when volumes are in use.

• Fix bug where project_domain_{name,id} was set even if project_{name,id} was not set.

• Fixed the volume normalization function when used with cinder v2.

• The create_stack() call was fixed to call the correct iterator method and to return the updated stack object when waiting.

• The create_server() API call would not use the supplied ‘network’ parameter if the ‘nics’ parameter was also supplied, even though it would be an empty list. It now uses ‘network’ if ‘nics’ is not supplied or if it is an empty list.

• The returned data from a create_service() call was not being normalized.

• Delete swift objects uploaded in service of uploading images at the time that the corresponding image is deleted. On some clouds, image uploads are accomplished by uploading the image to swift and then running a task-import. As shade does this action on behalf of the user, it is not reasonable to assume that the user would then be aware of or manage the swift objects shade created, which led to an ongoing leak of swift objects.

• Upload swift Large Objects as Static Large Objects by default. Shade automatically uploads objects as Large Objects when they are over a segment_size threshold. It had been doing this as Dynamic Large Objects, which sound great, but which have the downside of not deleting their sub-segments when the primary object is deleted. Since nothing in the shade interface exposes that the object was segmented, the user would not know they would also need to find and delete the segments. Instead, we now upload as Static Large Objects which behave as expected and delete segments when the object is deleted.

• The delete_object() method was not returning True/False, similar to other delete methods. It is now consistent with the other delete APIs.

• The delete_project() API now conforms to our standard of returning True when the delete succeeds, or False when the project was not found. It would previously raise an expection if the project was not found.

• delete_image used to fail with an AttributeError if an invalid image name or id was passed, rather than returning False which was the intent. This is worthy of note because it’s a behavior change, but the previous behavior was a bug.

• When creating a new server, the timeout was not being passed through to floating IP creation, which could also timeout.

• Fix for list_networks() ignoring any filters.

• Added missing dependency on futures library for python 2. The depend was missed in testing due to it having been listed in test-requirements already.

• Fixed an issue where shade could report a floating IP being attached to a server erroneously due to only matching on fixed ip. Changed the lookup to match on port ids. This adds an API call in the case where the workaround is needed because of a bug in the cloud, but in most cases it should have no difference.

• Fix for update_domain() where ‘name’ was not updatable.

• Fixed magnum service_type. shade was using it as ‘container’ but the correct type is ‘container-infra’. It’s possible that on old clouds with magnum shade may now do the wrong thing. If that occurs, please file a bug.

• Setting and unsetting flavor extra specs now works. This had been broken since the 1.2.0 release.

• Refactor OpenStackConfig._fix_backward_madness() into OpenStackConfig.magic_fixes() that allows subclasses to inject more fixup magic into the flow during get_one_cloud() processing.

• Fixed a bug related to neutron endpoints that did not have trailing slashes.

• Fixed issue with ports not having a created_at attribute.

• Role assignments were being returned as plain dicts instead of Munch objects. This has been corrected.

• Reverse the order of option selction in OpenStackConfig._validate_auth() to prefer auth options passed in (from argparse) over those found in clouds.yaml. This allows the application to override config profile auth settings.

• No longer fail in list_router_interfaces() if a router does not have the external_gateway_info key.

• Keystone service descriptions were missing an attribute describing whether or not the service was enabled. A new ‘enabled’ boolean attribute has been added to the service data.

• Fixed an issue where a section of code that was supposed to be resetting the SwiftService object was instead resetting the protective mutex around the SwiftService object leading to an exception of “__exit__”

• shade now correctly does not try to attach a floating ip with auto_ip if the cloud has given a public IPv6 address and the calling context supports IPv6 routing. shade has always used this logic to determine the server ‘interface_ip’, but the auto floating ip was incorrectly only looking at the ‘public_v4’ value to determine whether the server needed additional networking.

• Added requests and Babel to the direct dependencies list to work around issues with pip installation, entrypoints and transitive dependencies with conflicting exclusion ranges. Packagers of shade do not need to add these two new requirements to shade’s dependency list - they are transitive depends and should be satisfied by the other things in the requirements list. Both will be removed from the list again once the python client libraries that pull them in have been removed.

Other Notes

• HPCloud vendor profile removed due to cloud shutdown.

• RunAbove vendor profile removed due to migration to OVH.

• The contents of x-openstack-request-id are no longer added to object returned. Instead, they are logged to a logger named ‘openstack.cloud.request_ids’.

• The shade and os-client-config libraries have been merged into python-openstacksdk.

• Started using reno for release notes.

• Add citycloud regions for Buffalo, Frankfurt, Karlskrona and Los Angles

• Add new DreamCompute cloud and deprecate DreamHost cloud