Current Series Release Notes¶
41.0.0-19¶
Prelude¶
Tempest dropped the Python 3.8 support.
This is an intermediate release during the 2025.1 development cycle to mark the end of support for 2023.1 release in Tempest. After this release, Tempest will support below OpenStack Releases: * 2024.2 * 2024.1 * 2023.2 Current development of Tempest is for OpenStack 2025.1 development cycle.
New Features¶
Add a new config option [image_feature_enabled]/image_format_enforcement which tells tempest that glance will do image format inspection and enforcement on upload. This will disable tests that require glance to accept a bad image in order to test another service (i.e. nova).
Upgrade Notes¶
The default for
[volume] catalog_type
, which is used to determine the service type to use to identify the block storage service in the service catalog, has changed fromvolumev3
toblock-storage
.
Python 3.8 support has been dropped. Last release of Tempest to support python 3.8 is Temepst 41.0.0. The minimum version of Python now supported by Tempest is Python 3.9.
41.0.0¶
Prelude¶
This release is to tag Tempest for OpenStack 2024.2 release. This release marks the start of 2024.2 release support in Tempest. After this release, Tempest will support below OpenStack Releases: * 2024.2 * 2024.1 * 2023.2 * 2023.1 Current development of Tempest is for OpenStack 2025.1 development cycle. Every Tempest commit is also tested against master during the 2025.1 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2025.1 (or future release) cloud. To be on safe side, use this tag to test the OpenStack 2024.2 release.
New Features¶
Adding a new config options [scenario]/target_dir which allows users to specify the location where timestamps files will be written to. The default value is /tmp that, however, cannot be expected to persist across reboots of an instance.
Add support for project manager and domain manager personas by adding
get_project_manager_creds
andget_domain_manager_creds
to theDynamicCredentialProvider
andPreProvisionedCredentialProvider
classes of the common library.
40.0.0¶
Prelude¶
This is an intermediate release during the 2024.2 Dalmatian development cycle to make new functionality available to plugins and other consumers.
New Features¶
Added a new config option in the image section, http_qcow2_image, which will use qcow2 format image to download from the external source specified and use it for image conversion in glance tests. By default it will download http://download.cirros-cloud.net/0.6.2/cirros-0.6.2-x86_64-disk.img
Add new config option ‘enable_volume_image_dep_tests’ in section [volume-feature-enabled] which should be used in image<->volume<->snapshot dependency tests.
Adds API calls for traits in ResourceProvidersClient.
Adds API calls for traits in PlacementClient.
Add a new config option [compute]/target_hosts_to_avoid which will filter out any hypervisor candidates with a hostname that matches the provided pattern when determining target hosts for migration.
Upgrade Notes¶
The
attr
decorator is no longer available in thetempest.test
module. Use thetempest.lib.decorators
module instead.
Default value of the
[service_available] neutron
option has been updated fromFalse
toTrue
.
All tests which require network features are now skipped when the
[service_available] neutron
option is set toFalse
Changed the default value of ‘http_image’ config option in the ‘image’ group to ‘http://download.cirros-cloud.net/0.6.2/cirros-0.6.2-x86_64-uec.tar.gz’ from ‘http://download.cirros-cloud.net/0.3.1/cirros-0.3.1-x86_64-uec.tar.gz’ as former image is very old and we should always use latest one.
Tests for identity v2 API have been removed.
Deprecation Notes¶
The following options have been formally deprecated. These options were used to test identity v2 API which was removed during Queens cycle. The tests for identity v2 API were removed from tempest and these options have no effect.
[identity] uri
[identity] v2_admin_endpoint_type
[identity] v2_public_endpoint_type
[identity-feature-enabled] api_v2_admin
The following options have been deprecated because only identity v3 API is used.
[identity] auth_version
[identity-feature-enabled] api_v3
39.0.0¶
Prelude¶
This is an intermediate release during the 2024.2 development cycle to mark the end of support for Zed release in Tempest. After this release, Tempest will support below OpenStack Releases: * 2024.1 * 2023.2 * 2023.1 Current development of Tempest is for OpenStack 2024.2 development cycle.
New Features¶
We add a new argument,
--prefix
, totempest cleanup
tool that will allow users delete only resources that match the prefix. When this option is used,saved_state.json
file is not needed (no need to run with--init-saved-state
first). If there is one, it will be ignored and the cleanup will be done based on the given prefix only. Note, that some resources are not named thus they will not be deleted when filtering based on the prefix.
A new config option in the
volume_feature_enabled
section,volume_types_for_data_volume
, is added to allow the user to specify which volume types can be used for data volumes in a new testtest_instances_with_cinder_volumes_on_all_compute_nodes
. By default, this option is set to None.
A new interface
--resource-list
has been introduced in thetempest cleanup
command to remove the resources created by Tempest. A new config option in the default section,record_resources
, is added to allow the recording of all resources created by Tempest. A list of these resources will be saved inresource_list.json
file, which will be appended in case of multiple Tempest runs. This file is intended to be used with thetempest cleanup
command if it is used with the newly added option--resource-list
.
Upgrade Notes¶
The following deprecated alias methods of the
ContainerClient
class has been removed.update_container_metadata
, replaced bycreate_update_or_delete_container_metadata
list_container_contents
, replaced bylist_container_objects
The following decorators are no longer available in the
tempest.test
module. Use thetempest.common.utils
module instead.services
requires_ext
is_extension_enabled
Default value of the
[image-feature-enabled] image_import
has been changed fromFalse
toTrue
, and now the image import feature is tested by default.
Default value of the
[image-feature-enabled] os_glance_reserved
has been changed fromFalse
toTrue
and now the reservation of os_glance namespace is tested by default.
The deprecated
[compute-feature-enabled] xenapi_apis
option has been removed.
Deprecation Notes¶
The
[image-feature-enabled] image_import
option has been deprecated. The image import feature works in both standalone mode and WSGI mode since Victoria and the image import feature can be always tested.
The
[image-feature-enabled] os_glance_reserved
option has been deprecatd because glance reserves the os_glance namespace since Wallaby.
38.0.0¶
Prelude¶
This release is to tag Tempest for OpenStack 2024.1 release. This release marks the start of 2024.1 release support in Tempest. After this release, Tempest will support below OpenStack Releases: * 2024.1 * 2023.2 * 2023.1 * Zed Current development of Tempest is for OpenStack 2024.2 development cycle. Every Tempest commit is also tested against master during the 2024.2 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2024.2 (or future release) cloud. To be on safe side, use this tag to test the OpenStack 2024.1 release.
New Features¶
Add a new config options migration_source_host and migration_dest_host in the compute section, which if is set takes source or destination host from options, otherwise a host is chosen automatically.
Upgrade Notes¶
The deprecated
[network] dns_servers
option has been removed.
37.0.0¶
Prelude¶
This is an intermediate release during the 2024.1 development cycle to mark the end of support for Yoga release in Tempest. After this release, Tempest will support below OpenStack Releases: * 2023.2 * 2023.1 * Zed Current development of Tempest is for OpenStack 2024.1 development cycle.
New Features¶
A new config option in the default section, resource_name_prefix, is added to allow users to customize the name (specifically the prefix) of the resources created by Tempest during a test run. By default it is set to tempest. Tempest cleanup CLI will then use this config option to cleanup only the resources that match the prefix. Make sure this prefix does not match with the resource name you do not want Tempest cleanup CLI to delete.
Add delete image from specific store API to image V2 client
A new tempest.conf option called backup_driver was added to tempest to indicate which backup driver is used by Cinder. This option allows us to create a resource properly and avoid cleanup issues when Swift is used as the backup driver.
Upgrade Notes¶
The following deprecated options in the
[identity-feature-enabled]
section have been removed. Project tags API and application credentials API are now always tested if identity v3 API is available.project_tag
application_credentials
Default value of the
[identity-feature-enabled] access_rule
option has been changed fromFalse
toTrue
and now the access rule API is always tested when identity API is available.
The deprecated
[compute-feature-enabled] block_migrate_cinder_iscsi
option has been removed. Now the[compute-feature-enabled] block_migration_for_live_migration
option is solely used to determine when to run block migration based tests during live migration.
The deprecated
[compute-feature-enabled] nova_cert
option has been removed. The nova-cert service was removed from nova in 16.0.0 release. Tests of compute root certificates API have also been removed.
The deprecated
[compute-feature-enabled] rdp_console
config option has been removed.
The deprecated
[compute-feature-enabled] vnc_server_header
option has been removed.
Deprecation Notes¶
The Keystone access_rule is enabled by default since Train release and we no longer need a separate config in Tempest to enable it. Therefore the
[identity-feature-enabled] access_rule
option has been deprecated and will be removed in a future release.
36.0.0¶
Prelude¶
Glance v1 APIs were removed in Rocky release and last supported release for v1 was Queens. Tempest master does not support the Rocky or Queens release so we removed the Glance v1 tests, config option, and its service clients. If you would like to test the v1 APIs then you can use the old Tempest version.
Tempest remove the nova-network tests and service clients. The nova-network was removed from Rocky release and current Tempest master does not support the Rocky release. Below are the service clients have been removed: * floating_ip_pools_client * floating_ips_bulk_client * fixed_ips_client * list_virtual_interfaces
This release is to tag Tempest for OpenStack 2023.2 release. This release marks the start of 2023.2 release support in Tempest. After this release, Tempest will support below OpenStack Releases:
2023.2
2023.1
Zed
Yoga
Current development of Tempest is for OpenStack 2024.1 development cycle. Every Tempest commit is also tested against master during the 2024.1 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2024.1 (or future release) cloud. To be on safe side, use this tag to test the OpenStack 2023.2 release.
New Features¶
The following
tasks_client
tempest client for glance v2 image task API is implemented in this release.
Adding a new config option [identity]/user_minimum_password_age which allows to specify the number of days that a password must be used before the user can change it. For this option to take effect, identity-feature-enabled.security_compliance must be set to True.
35.0.0¶
Prelude¶
This is an intermediate release during the 2023.2 development cycle to mark the end of support for EM Xena release in Tempest. After this release, Tempest will support below OpenStack Releases: * 2023.1 * Zed * Yoga Current development of Tempest is for OpenStack 2023.2 development cycle.
New Features¶
Add delete_attachment to the v3 AttachmentsClient and terminate_connection to the v3 VolumesClient.
34.2.0¶
Prelude¶
This is an intermediate release during the 2023.2 development cycle to make scenario tests server SSHABLE functionality available to plugins and other consumers.
Upgrade Notes¶
Update default value of config option
CONF.identity.v3_entrypoint_type
from adminURL to public. This was deprecated in Q release, and was missed. The default entrypoint used by tempest should be the public one.
34.0.0¶
Prelude¶
Adding placement service for config options enforce_scope
so that we can switch the scope and new defaults enforcement for placement service.
This release is to tag Tempest for OpenStack 2023.1 release. This release marks the start of 2023.1 release support in Tempest. After this release, Tempest will support below OpenStack Releases:
2023.1
Zed
Yoga
Xena
Current development of Tempest is for OpenStack 2023.2 development cycle. Every Tempest commit is also tested against master during the 2023.2 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2023.2 (or future release) cloud. To be on safe side, use this tag to test the OpenStack 2023.1 release.
New Features¶
Adds a
ssh_allow_agent
parameter to theRemoteClient
class wrapper and the direct sshClient
class to allow a caller to explicitly request that the SSH Agent is not consulted for authentication. This is useful if your attempting explicit password based authentication asparamiko
, the underlying library used for SSH, defaults to utilizing an ssh-agent process before attempting password authentication.
Bug Fixes¶
There was a bug (bug#1964509) in dynamic credentials creation where project credentials with different roles are created with the new projects. Credential of different role of projects must be created within the same project. For exmaple, ‘project_admin’, ‘project_member’, ‘project_reader’, and ‘primary’, credentials will be created in the same projects. ‘alt’, ‘project_alt_admin’, ‘project_alt_member’, ‘project_alt_reader’ will be created within the same project.
‘admin’ credenatials is considered and kept as legacy admin and will be created under new project. If any test want to test with admin role in projectA and non-admin/admin in projectB then test can request projectA admin using ‘admin’ or ‘project_alt_admin’ and non-admin in projectB using ‘primary’, ‘project_member’, or ‘project_reader’/admin in projectB using ‘project_admin’. Many existing tests using the ‘admin’ with new project to assert on the resource list so we are keeping ‘admin’ a kind of legacy admin.
33.0.0¶
Prelude¶
This is an intermediate release during the 2023.1 development cycle to mark the end of support for EM Wallaby release in Tempest. After this release, Tempest will support below OpenStack Releases:
Zed
Yoga
Xena
Current development of Tempest is for OpenStack 2023.1 development cycle.
New Features¶
The
server_external_events
tempest client for compute Server External Events API is implemented in this release.
Add a new config option [image_feature_enabled]/manage_locations which enables tests for the show_multiple_locations=True functionality in glance. In order for this to work you must also have a store capable of hosting images with an HTTP URI.
32.0.0¶
Prelude¶
This release is to tag Tempest for OpenStack Zed release. This release marks the start of Zed release support in Tempest. After this release, Tempest will support below OpenStack Releases:
Zed
Yoga
Xena
Wallaby
Current development of Tempest is for OpenStack 2023.1 development cycle. Every Tempest commit is also tested against master during the 2023.1 cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a 2023.1 (or future release) cloud. To be on safe side, use this tag to test the OpenStack Zed release.
New Features¶
The following
image_cache
tempest client for glance v2 image caching API is implemented in this release.
Add configuration parameter tempurl_digest_hashlib into object-storage-feature-enabled which configures the hashing algorithm to use for the temp_url tests; defaults to ‘sha256’.
Security Issues¶
Swift used to support only ‘sha1’ for temp_url hashing but from many years now ‘sha256’ and ‘sha512’ are also available. These are stronger than ‘sha1’ and tempest now allows configuring which one to use.
31.1.0¶
New Features¶
Added new module net_downtime including the fixture NetDowntimeMeter that can be used to measure how long the connectivity with an IP is lost during certain operations like a server live migration. The configuration option allowed_network_downtime has been added with a default value of 5.0 seconds, which would be the maximum time that the connectivity downtime is expected to last.
31.0.0¶
Prelude¶
This is an intermediate release during the Zed development cycle to mark the end of support for EM Ussuri release in Tempest. After this release, Tempest will support below OpenStack Releases:
Yoga
Xena
Wallaby
Victoria
Current development of Tempest is for OpenStack Zed development cycle.
This is an intermediate release during the Zed development cycle to mark the end of support for EM Victoria release in Tempest. After this release, Tempest will support below OpenStack Releases:
Yoga
Xena
Wallaby
Current development of Tempest is for OpenStack Zed development cycle.
Upgrade Notes¶
As the version of cirros used in OpenStack CI does not support SHA-2 signatures for ssh, any connection from a FIPS enabled machine will fail in case validation.ssh_key_type is set to rsa (the default until now). Using ecdsa keys helps us avoid the mentioned issue. From now on, the validation.ssh_key_type option will be set to ecdsa by default for testing simplicity. This change shouldn’t have any drastic effect on any tempest consumer, in case rsa ssh type is required in a consumer’s scenario, validation.ssh_key_type can be overridden to rsa.
Python 3.6 and 3.7 support has been dropped. Last release of Tempest to support python 3.6 and 3.7 is Temepst 30.0.0. The minimum version of Python now supported by Tempest is Python 3.8.
Remove deprecated config option
api_extensions
fromcompute_feature_enabled
groups.
30.0.0¶
Prelude¶
This release is to tag Tempest for OpenStack Yoga release. This release marks the start of Yoga release support in Tempest. After this release, Tempest will support below OpenStack Releases:
Yoga
Xena
Wallaby
Victoria
Ussuri
Current development of Tempest is for OpenStack Zed development cycle. Every Tempest commit is also tested against master during the Zed cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Zed (or future release) cloud. To be on safe side, use this tag to test the OpenStack Yoga release.
New Features¶
Add parameter to specify the SSH key type. Current options are ‘rsa’ (which is the default) and ‘ecdsa’. Tempest now supports the importing and generation of both ‘rsa’ and ‘ecdsa’ SSH key types.
Add a new client to lists, creates, shows information for, updates and deletes neutron floating ips port forwarding resource.
Deprecation Notes¶
Old APIMicroversionFixture classes
tempest.api.compute.api_microversion_fixture.APIMicroversionFixture
andtempest.api.volume.api_microversion_fixture.APIMicroversionFixture
has been deprecated for removal in favor of new locationtempest.lib.common.api_microversion_fixture.APIMicroversionFixture
Bug Fixes¶
[bug 1948935] The default value of account-generator –concurrency parameter is now set to 2 instead of 1.
29.2.0¶
New Features¶
Added QoS minimum packet rate rule client:
create_minimum_packet_rate_rule
update_minimum_packet_rate_rule
show_minimum_packet_rate_rule
list_minimum_packet_rate_rules
delete_minimum_packet_rate_rule also
29.1.0¶
Bug Fixes¶
Improve cleanup after Swift testing. Ensures containers are empty before deleting to prevent errors due to delayed execution.
29.0.0¶
Prelude¶
This release is to tag Tempest for OpenStack Xena release. This release marks the start of Xena release support in Tempest. After this release, Tempest will support below OpenStack Releases:
Xena
Wallaby
Victoria
Ussuri
Current development of Tempest is for OpenStack Yoga development cycle. Every Tempest commit is also tested against master during the Yoga cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Yoga (or future release) cloud. To be on safe side, use this tag to test the OpenStack Xena release.
28.1.0¶
New Features¶
Add “QoS bandwidth limit rules” APIs to: “tempest/tests/lib/services/network/test_qos_limit_bandwidth_rules_client.py” module.
List bandwidth limit rules for QoS policy
Create bandwidth limit rule
Show bandwidth limit rule details
Update bandwidth limit rule
Delete bandwidth limit rule
Add a new client for keystone’s unified limits API to create and update limits.
Lists neutron’s Loggable resources API service clients are available in
tempest/lib/services/network/loggable_resource_client.py
module.
Upgrade Notes¶
Application credentials are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables application credentials testing (
CONF.identity-feature-enabled.application_credentials
) is now enabled by default.
Project tags are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables project tags testing (
CONF.identity-feature-enabled.project_tags
) is now enabled by default.
Deprecation Notes¶
Application credentials are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables application credentials testing (
CONF.identity-feature-enabled.application_credentials
) is now deprecated.
Project tags are supported by Keystone since Queens. As Tempest currently supports only much newer OpenStack versions (Ussuri or later), the configuration option which enables project tags testing (
CONF.identity-feature-enabled.project_tags
) is now deprecated.
28.0.0¶
Prelude¶
This is an intermediate release during the Xena development cycle to mark the end of support for EM Train release in Tempest. After this release, Tempest will support below OpenStack Releases:
Wallaby
Victoria
Ussuri
Current development of Tempest is for OpenStack Xena development cycle.
Tempest’s identity service client TokenClientJSON and V3TokenClientJSON has been removed.
Tempest Service clients for volume v1 APIs are removed.
New Features¶
Adding new config option for volume tests which allows to specify the size a volume will be extended by (if a test does extend a volume or needs a new bigger volume). The option is beneficial in case such tests are executed on systems where the chunk size (the minimum size a volume can be extended by) is other than 1 (originally hardcoded in the tests).:
CONF.volume.volume_size_extend
Add
placement
API usage method for evaluating resource class utilization of the resource provider. The following API call is available for tempest from now in the resource_providers_client:GET /resource_providers/{uuid}/usages
Add a
TransfersV355Client
to the volume v3transfer_client
library supporting create, list, show, delete, and accept operations for the new Volume Transfers API of the Block Storage API v3. The min_microversion of this API is 3.55.
Add a new client to lists, creates, shows information for, and updates neutron log resource.
Upgrade Notes¶
Tempest’s identity service client TokenClientJSON and V3TokenClientJSON were deprecated since long which are not removed. Please use new service clients TokenClient and V3TokenClient instead.
Cinder removed the volume v1 APIs in queens release and Tempest now support only stable train onwards release so all the Tempest service clients for volume v1 APIs are now removed.
Deprecation Notes¶
Tempest service clients for image v1 APIs (tempest.lib.services.image.v1 module) is deprecated and will be removed once Tempest stop supporting stable Ussuri release which is last release Image v1 APIs are present in glance.
Tempest service clients for volume v2 APIs (tempest.lib.services.volume.v2 module) is deprecated and will be removed once Tempest stop supporting stable wallaby release which is last release volume v2 APIs are present in cinder.
27.0.0¶
Prelude¶
Tempest.scenario.manager is now a stable interface for Tempest plugins.
This release is to tag Tempest for OpenStack Wallaby release. This release marks the start of Wallaby release support in Tempest. After this release, Tempest will support below OpenStack Releases:
Wallaby
Victoria
Ussuri
Train
Current development of Tempest is for OpenStack Xena development cycle. Every Tempest commit is also tested against master during the Xena cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Xena (or future release) cloud. To be on safe side, use this tag to test the OpenStack Wallaby release.
New Features¶
In this release, we made tempest/scenario/manager.py a stable interface ready to be consumed by all tempest plugins. The effort was tracked in the following etherpad [1], and the related review can be listed via [2]:
[2] https://review.opendev.org/#/q/topic:bp/tempest-scenario-manager-stable(status:open+OR+status:merged)
Some time ago, tempest/scenario/manager.py got copied to most of the plugins and therefore, it diverged - every plugin’s copy had slight differences. In this release, we pushed changes to unify the manager’s methods and improved their APIs in order to have them easier consumable:
we added implementations of methods that were often used in plugins’ manager.py however were not implemented in our manager
we improved APIs by f.e. adding a kwargs argument so that the consumers are more in control of the data that are passed to tempest clients
we modified logic of a few methods so that it complies with the plugins’ manager versions in order to prepare for a situation when the plugins can reuse Tempest manager as much as possible rather than keeping their own copy
we made methods consistent w.r.t. names and parameters
we split the lengthy methods to have more readable code
previously private methods which had a potential to be reused were made public so that it’s clear they are expected to be used in tempest plugins
missing docstrings have been added
Add a new client to handle requests to create and delete assisted volume snapshots.
Adds a method to images_client to get tasks relevant to a given image. Also adds has_version() method to image versions_client to probe for availability of a given API version.
Upgrade Notes¶
img_dir scenario option has been deprecated and it’s being removed. Starting Tempest 25.0.0 release, CONF.scenario.img_file needs a full path for the image. Until this release, old behavior was maintained and kept working however a user needs to specify the full path in CONF.scenario.img_file config option from now on.
The old deprecated
data-ultis
fromtempest.common.utils
has been removed. If you are still using this, use the stable version ofdata-utils
from new locationtempest.lib.common.utils
.
Other Notes¶
A new
[compute-feature-enabled]ide_bus
config option has been introduced to indicate if the environment supports attaching disks to an instance using anIDE
bus.This currently defaults to
True
but should be set toFalse
when using the libvirt OpenStack Nova virt driver and theq35
machine type as support for this bus is no longer provided.
26.1.0¶
Prelude¶
This is an intermediate release during the Wallaby development cycle to make new functionality available to plugins and other consumers.
The integrated horizon dashboard test is now moved from tempest-horizon plugin into Tempest. You do not need to install tempest-horizon to run the horizon test which can be run using Tempest itself.
Support for RBAC new system scope is added in Tempest.
New Features¶
The following tempest clients for keystone v3 OS_FEDERATION API were implemented in this release
identity_providers
protocols
mappings
service_providers
Add project alternate admin, member and reader role for dynamic credentials.
Added methods to the identity v3 roles client to support:
PUT /v3/system/users/{user}/roles/{role}
GET /v3/system/users/{user}/roles
GET /v3/system/users/{user}/roles/{role}
DELETE /v3/system/users/{user}/roles/{role}
PUT /v3/system/groups/{group}/roles/{role}
GET /v3/system/groups/{group}/roles
GET /v3/system/groups/{group}/roles/{role}
DELETE /v3/system/groups/{group}/roles/{role}
Added public interface create_loginable_secgroup_rule(). Since this interface is meant to be used by tempest plugins, It doesn’t neccessarily require to be private api.
Added public interface create_security_group_rule(). Since this interface is meant to be used by tempest plugins, It doesn’t neccessarily require to be private api.
Added public interface log_console_output(). It used to be a private method with name _log_console_output(). Since this interface is meant to be used by tempest plugins, It doesn’t neccessarily require to be private api.
Keystone provides the new scoped token called
system
which can be used to query the system scoped API operation. Projects are moving towards the policy with new scope types, Keystone, Nova already provide the new policy for RBAC checks. Tempest has added the support to query the system scoped token from keystone to test the new policy. As next step, we will be moving all the Tempest tests on the project’s new policy.
Dynamic credentials now support the scope type for specific roles too along with
admin
,member
,reader
role. Test can specify the scope in the prefix ofcls.credentials
name. Ifsystem
is prefix incls.credentials
name then creds will be created with scope assystem
. Ifdomain
is prefix incls.credentials
name then creds will be created with scope asdomain
otherwise defaultproject
scope will be used. For Example:credentials = [['my_role', 'role1'], # this will be old style and project scoped ['project_my_role', 'role1'], # this will be project scoped ['domain_my_role', 'role1'], # this will be domain scoped ['system_my_role', 'role1']] # this will be system scoped
And below is how test can access the credential manager of respective credentials type:
cls.os_my_role.any_client cls.os_project_my_role.any_client cls.os_domain_my_role.any_client cls.os_system_my_role.any_client
For backward compatibility, we set the credentials manager class attribute in old style form too which is prefix with
os_roles_*
, examplecls.os_roles_my_role
but we recommend to use the new style attribute as shown above.
Adds new personas that can be used to test service policies for all default scopes (project, domain, and system) and roles (reader, member, and admin). Both dynamic credentials and pre-provisioned credentials are supported.
Known Issues¶
Scope support for specific role is not yet added for pre-provisioned credentials.
Upgrade Notes¶
In this release tempest/manager.py is removed after more than 4 years of deprecation.
The
tempest.lib.common.utils.data_utils.random_bytes()
helper function will no longer allow asize
of more than 1MiB. Tests generally do not need to generate and use large payloads for feature verification and it is easy to lose track of and duplicate large buffers. The sum total of such errors can become problematic in paralllelized and constrained CI environments.
Deprecated config options to select the Volume API version have been removed. Use
CONF.volume.catalog_type
to run volume tests under v3 or v2 APIs.CONF.volume-feature-enabled.api_v2
CONF.volume-feature-enabled.api_v3
Deprecation Notes¶
In this release the following tempest arguments are deprecated and replaced by new ones which are functionally equivalent:
–black-regex is replaced by –exclude-regex
–blacklist-file is replaced by –exclude-list
–whitelist-file is replaced by –include-list
For now Tempest supports both (new and old ones) in order to make the transition for all consumers smoother. However, that’s just a temporary case and the old options will be removed soon.
Bug Fixes¶
[bug 1613819] admin_domain_name and default_credentials_domain_name parameters under [auth] now affect a domain used for creating test users just as they affect it for projects. Previously a domain with an id set to “default” had to be present in order for test user creation to succeed with Keystone v3.
Fixes the bug# 1917168
26.0.0¶
Prelude¶
This is an intermediate release during the Wallaby development cycle to mark the end of support for EM Stein release in Tempest. After this release, Tempest will support below OpenStack Releases:
Victoria
Ussuri
Train
Current development of Tempest is for OpenStack Wallaby development cycle.
New Features¶
Add a new config option can_migrate_between_any_hosts in the compute-feature-enabled section, which can be set to False for environment with non homogeneous compute nodes, so that it can select a destination host for migrating automatically, otherwise the testcase may fail unexpectedly. e.g., if source host is with CPU “E5-2699 v4” and the selected target host is with CPU “E5-2670 v3”, the live-migration will fail because of the downgrade issue.
Add a new config option
[compute-feature-enabled] shelve_migrate
which enable test for environment that support cold migration of qcow2 unshelved instance.
A new config option in the validation section, image_alt_ssh_user, to specify the user name used to authenticate to an alternative instance (instance using image_ref_alt) in tests. By default this is set to root.
A new config option in the validation section, image_alt_ssh_password, to specify the password used to authenticate to an alternative instance (instance using image_ref_alt) in tests. By default this is set to password.
Added missing client methods for keystone’s OS-ENDPOINT-POLICY and OS-EP-FILTER APIs.
Added missing clients and tests for keystone’s v3 EC2 API which already existed for keystone v2.
Add show type API to v3 types_client library.
default_volume_type
Added associate_floating_ip() and dissociate_floating_ip() methods to the scenario manager.
Add glance image import APIs function to v2 images_client library.
stage_image_file
info_import
info_stores
image_import
[blueprint blueprint adopt-oslo-versioned-objects-for-db] Any reference to “tenant_id” in Network objects is replaced with “project_id”.
New config option to
network-feature-enabled
:available_features
. This is a list which can contain features that are not discoverable through Neutron API, or it can be the special entryall
.
Add
placement
API methods for testing Routed Provider Networks feature. The following API calls are available for tempest from now in the new resource_providers_client:GET /resource_providers
GET /resource_providers/{uuid}
GET /resource_providers/{uuid}/inventories
GET /resource_providers/{uuid}/aggregates
Upgrade Notes¶
The following deprecated image scenario options are removed after a ~4 year deprecation period.
ami_img_file
ari_img_file
aki_img_file
Starting Tempest 25.0.0 release, CONF.scenario.img_file need a full path for the image. CONF.scenario.img_dir was deprecated and will be removed in the next release. Till Tempest 25.0.0, old behavior is maintained and keep working but starting Tempest 26.0.0, you need to specify the full path in CONF.scenario.img_file config option.
The test_reboot_server_soft has been skipped for more than 6 years. Take into account that the minimum scenario test uses soft reboot and the nova functional test also covers reboot.
Member
role has been deprecated and replaced bymember
. Therefore the default value of config option[object-storage].operator_role
is changed tomember
. (Fixes bug #1330132)
Remove the deprecated CLI
tempest-account-generator
in favor oftempest account-generator
command. You can usetempest account-generator
CLI to generate the accounts yaml file.
A number of Compute APIs that only worked with the XenAPI virt driver have been removed in the Compute service. As a result, their corresponding tests are now disabled by default. They can be re-enabled using the new
[compute_feature_enabled] xenapi_apis
config option.
Bug Fixes¶
Fixed bug #1890060. tempest subunit_describe_calls –verbose not working with Cliff CLI. The subunit_describe_calls –verbose argument was a boolean and worked in the non Cliff CLI which is now deprecated, but does not work with cliff since –verbase is a standard cliff argument which is an int. Since the tool is in lib directory we cannot change the interface, so we add a new argument -a –all-stdout that will allow cliff CLI to support the feature in subunnit_describe_calls to print request and response headers and bodies to stdout.
is_resource_deleted method of v3 volumes_client might have returned a KeyError exception due to an incorrect accessing of a volume id in the case the volume was in error_deleting state. incorrect code - volume[‘id’] correct code - volume[‘volume’][‘id’] More details about the issue can be found at https://bugs.launchpad.net/tempest/+bug/1887980
Concurrency parameter for account-generator command was accepting negative values and zero. The concurrency parameter now accepts only positive numbers. When a negative value or zero is passed to the program then the program ends and help is displayed.
Other Notes¶
New configuration options
CONF.glance.image_feature_enabled.image_import
has been introduced to enable the image import tests. If your glance deployement support image import functionality then you can enable the image import tests via this flag. Default value of this new config option is false.
24.0.0¶
Prelude¶
Remove the support of python3.5. Tempest, its plugins dependencies in ussuri cycle are python-requires>=py3.6 which makes distro not having python 3.6 to do hack to install py3.6 etc. It time to drop the py3.5 from Tempest. Last supported version of Tempest for py3.5 is 23.0.0.
This release is to tag the Tempest for OpenStack Ussuri release. This release marks the start of Ussuri release support in Tempest. After this release, Tempest will support below OpenStack Releases: * Ussuri * Train * Stein Current development of Tempest is for OpenStack Victoria development cycle. Every Tempest commit is also tested against master during the Victoria cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ussuri (or future release) cloud. To be on safe side, use this tag to test the OpenStack Ussuri release.
New Features¶
A new config option ‘subnet_id’ is added to section ‘network’ to specify subnet which should be used for allocation of IPs for VMs created during testing. It should be used when the tested network contains more than one subnet otherwise test of external connectivity will fail. (Fixes bug
Add the option –worker-file in
tempest run
command. This is to give tempest more granularity to manually configure how the different sets of tests can be grouped to run with the different worker. You can configure tests regex to run under workers. You can also mix manual scheduling with standard one by mentioning concurrency. For example, the user can setup tempest to run with different concurrences, to be used with different regexps.
A new attachments client library has been introduced for the volume service.
Initially only the show_attachment API is provided. This API requires a minimum volume API microversion of
3.27
.
Upgrade Notes¶
Default value of config option
CONF.placement.region
is updated fromRegionOne
to empty string.As per tempest design, if tempest conf is not having any region for a service then identity region should be used. In case of placement the default value is “RegionOne” which is considered as placement region if region missing in tempest conf. In order to have identity region to be used as default we need to change to empty string for placement service. Empty string can be seen being used in other services like volume, image etc.
Python 2.7 support has been dropped. Last release of Tempest to support python 2.7 is Temepst 23.0.0. The minimum version of Python now supported by Tempest is Python 3.6.
Remove the deprecated argument
os-tenant-name
orOS_TENANT_NAME
in favour ofos-project-name
argument.
Remove the deprecated CLI
verify-tempest-config
in favour oftempest verify-config
command. You can usetempest verify-config
CLI to verify the tempest conf file.
Deprecation Notes¶
The
[compute-feature-enabled]/block_migrate_cinder_iscsi
is deprecated ahead of removal in a future release. Once removed the[compute-feature-enabled]/block_migration_for_live_migration
configurable will then be used to determine when to run block migration based tests during live migration.
The config options
CONF.compute.spice_console
andCONF.compute.rdp_console
are deprecated because test cases using them are removed. We can add them back when adding the test cases again.
Bug Fixes¶
Fixed bug #1858417. Adding consistency group exceptions
ConsistencyGroupException
andConsistencyGroupSnapshotException
that didn’t exist before and caused failure in cinder-tempest-plugin.
23.0.0¶
Prelude¶
This is an intermediate release during the Ussuri development cycle to mark the end of support for EM Queens in Tempest. After this release, Tempest will support below OpenStack Releases:
Train
Stein
Rocky
Current development of Tempest is for OpenStack Ussuri development cycle.
This is the last release of Tempest to officially support python2.7.
New Features¶
tempest cleanup CLI is extended about region deletion. Until now, the regions have been neglected by tempest cleanup. From now on, tempest cleanup is able to delete leftover regions as well.
Added new tempest subcommand for subunit-describe-calls
Deprecation Notes¶
Deprecated command for subunit-describe-calls
22.1.0¶
Bug Fixes¶
Bug#1847749. This provides the workaround of Skip Exception raised instead of skipping the CLI tests. If you are running Tempest with stestr > 2.5.0 then use this fix. Ref- https://github.com/testing-cabal/testtools/issues/272
22.0.0¶
Prelude¶
This release is to tag the Tempest for OpenStack Train release. This release marks the start of Train release support in Tempest. After this release, Tempest will support below OpenStack Releases:
Train
Stein
Rocky
Queens
Current development of Tempest is for OpenStack Ussuri development cycle. Every Tempest commit is also tested against master during the Ussuri cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ussuri (or future release) cloud. To be on safe side, use this tag to test the OpenStack Train release.
New Features¶
Add show api version details function to v3 versions_client library for cinder.
show_version
A new kwarg,
log_req_body
, was added to thetempest.lib.common.rest_client.RestClient
methodraw_request()
. This kwarg takes in a string which will be used in place of the request body, which is logged by default. The intent of this option is to be used for security reasons to avoid logging sensitive information that are part of request bodies.
Add
segments
client to Tempest to make possible the testing of the Routed Provider Networks feature. The following API calls are available for tempest from now:POST /segments
PUT /segments/{segment_id}
GET /segment/{segment_id}
DELETE /segments/{segment_id}
GET /segments
Deprecation Notes¶
The config option
CONF.compute.vnc_server_header
is deprecated because it has become obsolete with the usage of different response header fields to accomplish the same goal in accordance with RFC7231 Section 6.2.2.
Bug Fixes¶
Adequately validates WebSocket upgrade in test_novnc and removes unneeded configuration complexity. Closes bug #1838777 and #1840788.
21.0.0¶
Prelude¶
This is an intermediate release during the Train development cycle to mark the end of support for Pike in Tempest. After this release, Tempest will support below OpenStack Releases:
Stein
Rocky
Queens
Current development of Tempest is for OpenStack Train development cycle.
New Features¶
Add
qos-policies
andqos-minimum-bandwidth-rule
clients to Tempest to make possible the testing of the placement based bandwidth allocation feature. The following API calls are available for tempest from now:QoS policies
client:GET /qos/policies
POST /qos/policies
GET /qos/policies/{policy_id}
PUT /qos/policies/{policy_id}
DELETE /qos/policies/{policy_id}
QoS minimum bandwidth rules
client:GET qos/policies/{policy_id}/minimum_bandwidth_rules
POST /qos/policies/{policy_id}/minimum_bandwidth_rules
GET qos/policies/{policy_id}/minimum_bandwidth_rules/{rule_id}
PUT qos/policies/{policy_id}/minimum_bandwidth_rules/{rule_id}
DELETE /qos/policies/{policy_id}/minimum_bandwidth_rules/{rule_id}
Add list host API support to the volume v3 client library. This feature enables callers to list all hosts for a given project.
Add migrate volume API support to the volume v3 client library. This features allows callers to migrate volumes between backends.
Add show default group type details API to v3 group_types_client library.
show_default_group_type
New decorator
unstable_test
is added totempest.lib.decorators
. It can be used to mark some test as unstable thus it will be still run by tempest but job will not fail if this test will fail. Such test will be skipped in case of failure. It can be used for example when there is known bug related which cause irregular tests failures. Marking such test as unstable will help other developers to get their job done and still run this test to get additional debug data or to confirm if some potential fix really solved the issue.
A new parameter, compute/compute_volume_common_az is introduced to specify availability zone where tempest creates instances and volumes for scenario tests, to allow us to run scenario tests in the deployment which has multiple availability zones and cinder/cross_az_attach in nova.conf is set to False.
New library interface to set the API microversion on Service Clients.
APIMicroversionFixture,
can be used to set the API microversion on multiple services. This Fixture will take care of reseting the service microversion to None once test is finished.
Add microversion support for scenario tests. Scenario test calls multiple service API within same test and many services like compute, volume and placement etc support API microversion. With microversion support in scenario test, we can call different service API with different microvesion. Which means we can implement scenario tests for microversion also. Currently Scenario manager support below services microversion:
Compute
Volume
Placement
Upgrade Notes¶
Remove deprecated config option
endpoint_type
fromidentity
group. Usev2_public_endpoint_type
fromidentity
group instead. Remove deprecated config optiontenant_isolation_domain_name
fromauth
group. Usedefault_credentials_domain_name
fromauth
group instead.
Remove deprecated config option
admin_username
fromidentity
groups. Useadmin_username
fromauth
instead. Remove deprecated config optionadmin_tenant_name
fromauth
andidentity
groups. Useadmin_project_name
fromauth
instead. Remove deprecated config optionadmin_password
fromidentity
groups. Useadmin_password
fromauth
instead. Remove deprecated config optionadmin_domain_name
fromidentity
groups. Useadmin_domain_name
fromauth
instead.
Deprecation Notes¶
The config option
CONF.network.dns_servers
is no longer used anywhere, so it is deprecated and will be removed in the future.
Other Notes¶
New configuration options
[compute]/certified_image_ref
and[compute]/certified_image_trusted_certs
have been introduced. These are required in order to run theServerShowV263Test
test and allow a signed image with the required img_signature_* properties set along with a list of trusted certificates to be used during the test.
20.0.0¶
Prelude¶
This release is to tag the Tempest for OpenStack Stein release. This release marks the start of Stein release support in Tempest and the end of support for Ocata in Tempest. After this release, Tempest will support below OpenStack Releases:
Stein
Rocky
Queens
Pike
Current development of Tempest is for OpenStack Train development cycle. Every Tempest commit is also tested against master during the Train cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Train (or future release) cloud. To be on safe side, use this tag to test the OpenStack Stein release.
New Features¶
Add basic read-only Placement client to Tempest to make possible the testing of the placement based bandwidth allocation feature. The following API calls are available for tempest from now:
GET /allocation_candidates
GET /allocations/{consumer_uuid}
Add new config group
placement
, with the config options:endpoint_type
to use for communication with placement service.catalog_type
of the placement service.region
as the placement region name to use.min_microversion
andmax_microversion
as the range between placement API requests are sent.
Add a new config setting
immutable_user_source
in the[identity-feature-enabled]
group that defaults to false. This setting, combined with the usage of the@testtools.skipIf()
decorator, will allow tests that require user creation, deletion, or modification to skip instead of failing in environments that are LDAP-backed. In such environments, the user source is read-only, so this feature flag is needed to allow such tests to gracefully skip without having to blacklist them.
Add support of OSProfiler library for profiling and distributed tracing of OpenStack. A new config option
key
in sectionprofiler
is added, the option sets the secret key used to enable profiling in OpenStack services. The value needs to correspond to the one specified in [profiler]/hmac_keys option of OpenStack services.
A new parameter
follow_redirects
has been added to the classRestClient
, which is passed through toClosingHttp
orClosingProxyHttp
respectively. The default value isTrue
which corresponds to the previous behaviour of following up to five redirections before returning a response. Settingfollow_redirects = False
allows to disable this behaviour, so that any redirect that is received is directly returned to the caller. This allows tests to verify that an API is responding with a redirect.
Adds the new method to AgentsClient that implements agent deletion according to the API [0]. [0] https://developer.openstack.org/api-ref/network/v2/index.html#delete-agent
The
tempest.lib.decorators.attr
decorator now supports acondition
kwarg which can be used to conditionally apply the attr to the test function if the condition evaluates to True.
Add
show_version
function to theNetworkVersionsClient
client. This allows the possibility of getting details for Networking API.
Upgrade Notes¶
Changed the default value of ‘api_v1’ config option in the ‘image-feature-enabled’ group to False from True, because glance v1 APIs are deprecated. Please set True explicitly on the option if still testing glance v1 APIs.
The
tempest-full
,tempest-full-py3
andtempest-slow
zuul v3 job configurations now setENABLE_VOLUME_MULTIATTACH: true
in thedevstack_localrc
variables section. If you have a plugin job configuration that inherits from one of these jobs and the backend cinder volume driver or nova compute driver do not support volume multiattach then you should override and set this variable toENABLE_VOLUME_MULTIATTACH: false
in your job configuration.
Below config option was deprecated for removal since juno release. It’s time to remove it as all supported stable branches and Tempest plugins are good to handle it.
[volume_feature_enabled].api_v1
Also Tempest removes the below corresponding service clients alias from client.py which were being set based on above removed config option.
self.backups_client
self.encryption_types_client
self.snapshots_client
self.volume_availability_zone_client
self.volume_hosts_client
self.volume_limits_client
self.volume_qos_client
self.volume_quotas_client
self.volume_services_client
self.volume_types_client
self.volumes_client
self.volumes_extension_client
tempest.lib.common.utils.misc.find_test_caller
was deprecated during Kilo release cycle in favor oftempest.lib.common.utils.test_utils.find_test_caller
. The deprecated version offind_test_caller
is removed.
CONF.validation.run_validation
default enabled. This option required to be settrue
in order to run api tests stability when the guest cooperation required. For example when the guest needs react on Volume/Interface detach. The ssh test makes sure the VM is alive and ready when the detach needs to happen. The option was enabled on the gate for a long time.
Deprecation Notes¶
The
scheduler_available_filters
option is being deprecated in favor ofscheduler_enabled_filters
. The new name is more indicative of what the option means.scheduler_enabled_filters
’s default value is set to the default value of Nova’senabled_filters
.scheduler_available_filters
’s default was all. There was confusion around this value. Sometimes it was understood to mean the default Nova filters are enabled, other times it was understood to mean all filters are enabled. While all is still allowed forscheduler_enabled_filters
for backwards compatibility, it is strongly recommended to provide an explicit list of filters that matches what’s configured in nova.conf.
Bug Fixes¶
[bug 1616892] Tempest now allows tests to verify that an API responds with a redirect.
Fixed bug #1791007.
tempest workspace register
andtempest workspace rename
CLI will error if None or empty string is passed in –name arguments. Earlier both CLI used to accept the None or empty string as name which was confusing.
Fixed bug #1799883.
tempest workspace register
andtempest workspace move
CLI will now validate the value of the--path
CLI argument and raise an exception if it is None or empty string. Earlier both CLI actions were accepting None or empty path which was confusing.
Fixed bug #1808473.
tempest run
CLI will error if a non-exist config file is input to parameter –config-file. Earlier non-exist config value was silently getting ignored and the default config file was used instead which used to give false behavior to the user on using the passed config file.
Patch https://review.opendev.org/#/c/499575/ introduced support creating Neutron port with certain capabilities. Currently capabilities list interpreted as string this change fix it.
tempest.conf [network] port_profile = capabilities:[switchdev]
result: {‘capabilities’:’[switchdev]’}
expected: {‘capabilities’: [‘switchdev’]}
19.0.0¶
Prelude¶
This release is to tag the Tempest for OpenStack Rocky release. After this release, Tempest will support below OpenStack Releases:
Rocky
Queens
Pike
Ocata
Current development of Tempest is for OpenStack Stein development cycle. Every Tempest commit is also tested against master during the Stein cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Stein (or future release) cloud. To be on safe side, use this tag to test the OpenStack Rocky release.
New Features¶
Add
v3-ext/OS-ENDPOINT-POLICY
API calls to support creation, deletion and retrieval of associations between service endpoints and policies. Such associations enable an endpoint to request its policy.
Add
enable_service
,disable_service
,disable_log_reason
,freeze_host
andthaw_host
API endpoints to volume v3services_client
.
Add a new parameter called
bug_type
totempest.lib.decorators.related_bug
andtempest.lib.decorators.skip_because
decorators, which accepts 2 values:launchpad
storyboard
This offers the possibility of tracking bugs related to tests using launchpad or storyboard references. The default value is launchpad for backward compatibility.
Passing in a non-digit
bug
value to either decorator will raise aInvalidParam
exception (previouslyValueError
).
Add update flavor API to compute flavors_client library.
[blueprint application-credentials] Tempest can test keystone’s application credentials interface. A new client library is added for application credentials, and a new config option,
[identity-feature-enabled]/application_credentials
, can control whether the application credentials feature is tested (defaults to False, indicating the feature is not enabled in the cloud under test).
Add
project_tags_client
to the identity v3 library. This feature enables the possibility of invoking the following API actions:update_project_tag
list_project_tags
update_all_project_tags
check_project_tag_existence
delete_project_tag
delete_all_project_tags
The
update_service
API is added to theservices_client
compute library. This API is introduced in microversion 2.53 and supersedes the following APIs:PUT /os-services/disable
(disable_service
)PUT /os-services/disable-log-reason
(disable_log_reason
)PUT /os-services/enable
(enable_service
)PUT /os-services/force-down
(update_forced_down
)
New string configuration option
vnc_server_header
is added tocompute-feature-enabled
section. It allows the expected VNC server name in the response header to be specified. For example, obvious at hand names are ‘WebSockify’, ‘nginx’.
Adds volume service clients for v3 APIs. As v3 base API should be identical to v2 APIs, we just copy all existing v2 service client for v3 API.
Upgrade Notes¶
Cinder CLI calls have now been updated to use the
--os-endpoint-type
option instead of--endpoint-type
. The latter had been deprecated in Cinder and has been removed in the Rocky release.
Remove deprecated config option
allow_tenant_isolation
fromauth
andcompute
groups. Useuse_dynamic_credentials
directly instead of the removed option.
The volume config option
catalog_type
default is changed tovolumev3
which is v3 API endpoint configured in devstack. With this change Tempest will be testing v3 API as default. User who want to test v2 API can still test by configuring thecatalog_type
to v2 endpoint.
Deprecation Notes¶
The v2 volume API has been deprecated since Pike release. Volume v3 API is current and Tempest volume tests can be run against v2 or v3 API based on config option
CONF.volume.catalog_type
. If catalog_type isvolumev2
, then all the volume tests will run against v2 API. If catalog_type isvolumev3
which is default in Tempest, then all the volume tests will run against v3 API. That makes below config options unusable in Tempest which used to select the target volume API for volume tests.CONF.volume-feature-enabled.api_v2
CONF.volume-feature-enabled.api_v3
Tempest deprecate the above two config options in Rocky release and will be removed in future. Alternatively
CONF.volume.catalog_type
can be used to run the Tempest against volume v2 or v3 API.
Deprecates the volume service clients for v2 APIs. Volume v2 APIs are deprecated in all supported stable branches, so it’s time to deprecate the tempest service clients for v2 APIs and remove in future release.
Security Issues¶
The x-subject-token of a response header is ommitted from log, but clients specify the same token on a request header on Keystone API and that was not omitted. In this release, that has been omitted for a security reason.
Bug Fixes¶
Fix show_image_file interface in v2 ImagesClient: Bug#1756264. The expected success code of show_image_file is changed from
200
to[200, 204, 206]
.
Fix VNC server response header issue when it is behind reverse proxy
18.0.0¶
Prelude¶
This release marks the start of Queens release support in Tempest. This release also marks the end of support for Newton in Tempest.
New Features¶
Add group type specs APIs to v3 group_types_client library.
create_or_update_group_type_specs
list_group_type_specs
show_group_type_specs_item
update_group_type_specs_item
delete_group_type_specs_item
A new config option ‘port_profile’ is added to the section ‘network’ to specify capabilities of the port. By default this is set to {}. When using OVS HW offload feature we need to create Neutron port with a certain capability. This is done by creating Neutron port with binding profile. To be able to test this we need profile capability support in Tempest as well.
Add show default quotas API to network quotas_client library. This feature enables the possibility to show default network quotas for a specified project.
Add show encryption specs item API to v2 encryption_types_client library. This feature enables the possibility to show specific encryption specs for a volume type.
Add extension API show quota details to network quotas_client library. This feature enables the possibility to show a quota set for a specified project that includes the quota’s used, limit and reserved counts per resource.
Add update group types API to v3
group_types_client
library; min_microversion of this API is 3.11.
A function has been added to the common library to allow mounting and unmounting of the config drive consistently.
The Tempest CLI commands have switched from calling testrepository internally to use stestr instead. This means that all of the features and bug fixes from moving to stestr are available to the tempest commands.
Adds a new CLI arg in tempest run,
--black-regex
, which is a regex to exclude the tests that match it.
A new boolean configuration option
[compute-feature-enabled]/volume_backed_live_migration
has been added. If enabled, tests which validate the behavior of Nova’s volume-backed live migration feature will be executed. The option defaults toFalse
.
Upgrade Notes¶
Below config options or feature flags were deprecated for removal. It’s time to remove them as all supported stable branches are good to handle them.
[identity-feature-enabled].forbid_global_implied_dsr
[image-feature-enabled].deactivate_image
[default].resources_prefix
config group
orchestration
[service_available].heat
Remove the deprecated decorator
skip_unless_attr
in lib/decorators.py.
The tox ostestr job (normally invoked with
tox -eostestr
) has been removed. This was lightly used, and in the near future ostestr will be removed from the tempest requirements file. If you were relying on this functionality you can replicate it by using the venv-tempest tox job. For example, simply runningtox -evenv-tempest -- ostestr
will do the same thing the old ostestr job did.
Tempest CLI commands will no long rely on anything from testr. This means any data in existing testr internals that were being exposed are no longer present. For example things like the .testr directories will be silently ignored. There is a potential incompatibility for existing users who are relying on test results being stored by testr. Anything relying on previous testr behavior will need to be updated to handle stestr.
Bug Fixes¶
Fixes tempest run CLI args mutually exclusive behavior which should not be the case anymore (Bug#1751201).
Other Notes¶
The CLIClient class, when it calls a command line client, uses
--os-project-name
instead of--os-tenant-name
for the project, and passes--os-identity-api-version
(default empty). All CLI clients still available in supported releases of OpenStack which are wrapped by thecmd_with_auth()
method support those switches.
A new configuration option
[compute-feature-enabled]/volume_multiattach
has been added which defaults to False. Set this to True to enable volume multiattach testing. These tests require that compute API version 2.60 is available and block storage API version 3.44 is available.Note
In the Queens release, the only compute driver that supports volume multiattach is the libvirt driver, and only then when qemu<2.10 or libvirt>=3.10. The only volume backend in Queens that supports volume multiattach is lvm.
OpenStack Releases supported after this release are Queens, Pike, and Ocata. The release under current development of this tag is Rocky, meaning that every Tempest commit is also tested against master during the Rocky cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Rocky (or future release) cloud.
17.2.0¶
New Features¶
Add support of args and kwargs when calling func in call_until_true, also to log the cost time when call_until_true returns True or False for debugging.
Add
list_auth_domains
API endpoint to the identity v3 client. This allows the possibility of listing all domains a user has access to via role assignments.
Upgrade Notes¶
Replace any call in your code to credentials_factory.DEFAULT_PARAMS with a call to config.service_client_config().
Remove deprecated APIs (
show_pools
andshow_backend_capabilities
) from volume v2 volumes_client, and the deprecated APIs are re-realized in volume v2 scheduler_stats_client (list_pools
) and capabilities_client (show_backend_capabilities
) accordingly.
Bug Fixes¶
The credentials_factory module used to load configuration at import time which caused configuration being loaded at test discovery time. This was fixed by removing the DEFAULT_PARAMS variable. This variable was redundant (and outdated), the same dictionary (but up to date) can be obtained via invoking config.service_client_config() with no service parameter.
17.1.0¶
Prelude¶
This is an intermediate release during the Queens development cycle to make new functionality available to plugins and other consumers.
New Features¶
Adds a new cli option to tempest run,
--load-list <list-file>
to specify target tests to run from a list-file. The list-file supports the output format of the tempest run--list-tests
command.
Add reset group snapshot status API to v3 group_snapshots_client library, min_microversion of this API is 3.19. This feature enables the possibility to reset group snapshot status.
Add reset group status API to v3 groups_client library, min_microversion of this API is 3.20. This feature enables the possibility to reset group status.
Add the validation_resources module to tempest.lib. The module provides a set of helpers that can be used to provision and cleanup all the resources required to perform ping / ssh tests against a virtual machine: a keypair, a security group with targeted rules and a floating IP.
Add the proxy_url optional parameter to the get_credentials method in tempest/lib/auth.py so that that helper can be used when going through and HTTP proxy.
Add a new function called
compare_version_header_to_response
totempest.lib.common.api_version_utils
, which compares the API microversion in the response header to another microversion using the comparators defined intempest.lib.common.api_version_request.APIVersionRequest
.It is now possible to determine how to retrieve an attribute from a response body of an API call, depending on the returned microversion.
Add a new exception type called
InvalidParam
totempest.lib.exceptions
, allowing the possibility of raising an exception if an invalid parameter is passed to a library function.
Adds a new config options,
proxy_url
. This options is used to configure running tempest through a proxy server.
The RestClient class in tempest.lib.rest_client has a new kwarg parameters,
proxy_url
, that is used to set a proxy server.
A new class was added to tempest.lib.http, ClosingProxyHttp. This behaves identically to ClosingHttp except that it requires a proxy url and will establish a connection through a proxy
A new boolean config option
domain_specific_drivers
is added to the sectionidentity-feature-enabled
. This option must be enabled when testing an environment that is configured to use domain-specific identity drivers.
Define below object storage service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes.
account_client
container_client
object_client
Two extra modules are now marked as stable for plugins, test.py and clients.py. The former includes the test base class with its automatic credentials provisioning and test resource managing fixtures. The latter is built on top of ServiceClients and it adds aliases and a few custom configurations to it.
Upgrade Notes¶
As of the Queens release, tempest no longer tests the identity v2.0 API because the majority of the v2.0 API have been removed from the identity project. Once the Queens release reaches end-of-life, we can remove the v2.0 tempest tests and clean up v2.0 testing cruft.
Remove deprecated APIs from volume v2 volumes_client, and the deprecated APIs are re-realized in volume v2 transfers_client.
create_volume_transfer
show_volume_transfer
list_volume_transfers
delete_volume_transfer
accept_volume_transfer
Remove two deprecated skip decorators in
config
module:skip_unless_config
andskip_if_config
.
Remove deprecated get_ipv6_addr_by_EUI64 method from data_utils. Use the same method from oslo_utils.netutils.
Bug Fixes¶
Allow to specify new domain parameters:
user_domain_name
user_domain_id
project_domain_name
project_domain_id
for CLIClient class, whose values will be substituted to
--os-user-domain-name
,--os-user-domain-id
,--os-project-domain-name
and--os-project-domain-id
respectively during command execution.This allows to prevent possible test failures with authentication in Keystone v3. Bug: #1719687
Fix list_group_snapshots API in v3 group_snapshots_client: Bug#1715786. The url path for list group snapshots with details API is changed from
?detail=True
to/detail
.
17.0.0¶
Prelude¶
This release marks the start of support for the Pike release in Tempest.
New Features¶
Add create_group_from_source to groups_client in the volume service library.
Implement the rest_client method is_resource_deleted in the network security group client.
The
list_backups
method of the v2BackupsClient
class now has an additional**params
argument that enables passing additional information in the query string of the HTTP request.
Add show volume image metadata API to v2 volumes_client library. This feature enables the possibility to show volume’s image metadata.
Add update_group to groups_client in the volume service library.
Add group_snapshots client for the volume service as library. Add tempest tests for create group snapshot, delete group snapshot, show group snapshot, and list group snapshots volume APIs.
Add list_group_type and show_group_type in the group_types client for the volume service. Add tests for create/delete/show/list group types.
The credentials_factory.py module is now marked as stable for Tempest plugins. It provides helpers that can be used by Tempest plugins to obtain test credentials for their test cases in a format that honors the Tempest configuration in use. Credentials may be provisioned on the fly during the test run, or they can be setup in advance and fed to test via a YAML file; they can be setup for identity v2 or identity v3.
Add the
disable_log_reason
and theupdate_forced_down
API endpoints to the computeservices_client
. Add ‘2.11’ compute validation schema for compute services API.
Enhances the v3 identity client with the
check_token_existence
endpoint, allowing users to check the existence of tokens
The tempest module tempest.common.dynamic creds which is used for dynamically allocating credentials has been migrated into tempest lib.
Define below object storage service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes.
bulk_middleware_client
capabilities_client
The tempest module tempest.common.preprov_creds which is used to provide credentials from a list of preprovisioned resources has been migrated into tempest lib at tempest.lib.common.preprov_creds.
The InvalidTestResource exception class from tempest.exceptions has been migrated into tempest.lib.exceptions
The tempest module tempest.common.fixed_network which provided utilities for finding fixed networks by and helpers for picking the network to use when multiple tenant networks are available has been migrated into tempest lib at tempest.lib.common.fixed_network.
When registering service clients from installed plugins, all registrations are now processed, even if one or more fails. All exceptions encountered during the registration process are recorded. If at least one exception was encountered, the registration process fails and all interim errors are reported.
The __repr__ method is now implemented for the base tempest.Exception class, its implementation is identical to __str__: it reports the error message merged with input parameters.
Add a new identity catalog client. At this point, the new client contains a single functionality, “show_catalog”, which returns a catalog object.
Upgrade Notes¶
Tempest checks a volume delete by waiting for NotFound(404) on show_volume(). Sometime a volume delete fails and the volume status becomes error_deleting which means the delete is failed. So Tempest doesn’t need to wait anymore. A new release of Tempest raises an exception DeleteErrorException instead of waiting.
Deprecation Notes¶
Remove the support of python3.4, because in Ubuntu Xenial only python3.5 is available (python3.4 is restricted to <= Mitaka).
Bug Fixes¶
Add more accurate ip version check in addresses schema which will limit the ip version value in [4, 6].
Add a missing return statement to the retype_volume API in the v2 volumes_client library: Bug#1703997
This changes the response body from None to an empty dictionary.
Fix RemoteClient default ssh_shell_prologue: Bug#1707478
The default ssh_shell_proloque has been modified from specifying erroneous PATH=$$PATH:/sbin to PATH=$PATH:/sbin.
Other Notes¶
OpenStack Releases supported after this release are Pike, Ocata, and Newton. The release under current development of this tag is Queens, meaning that every Tempest commit is also tested against master during the Queens cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Queens (or future release) cloud.
16.1.0¶
Prelude¶
This is an intermediate release during the Pike development cycle to make new functionality available to plugins and other consumers.
New Features¶
Add a new client to handle the OAUTH token feature from the identity API.
A new boolean config option
serial_console
is added to the sectioncompute-feature-enabled
. If enabled, tests, which validate the behavior of Nova’s serial console feature (an alternative to VNC, RDP, SPICE) can be executed.
Add a new client to handle the domain configuration feature from the identity v3 API.
A new config option in the network-feature-enabled section, floating_ips, to specify whether floating ips are available in the cloud under test. By default this is set to True.
Add force detach volume feature API to v2 volumes_client library. This feature enables the possibility to force a volume to detach, and roll back an unsuccessful detach operation after you disconnect the volume.
Defines the identity v3 OS-EP-FILTER EndPoint Groups API client. This client manages Create, Get, Update, Check, List, and Delete of EndPoint Group.
Defines the identity v3 OS-EP-FILTER extension API client. This client manages associations between endpoints, projects along with groups.
The
delete_volume
method of theVolumesClient
class now has an additional**params
argument that enables passing additional information in the query string of the HTTP request.
Add list volume transfers with details API to v2 transfers_client library. This feature enables the possibility to list volume transfers with details.
A new config option ‘manage_snapshot_ref’ is added in the volume section, to specify snapshot ref parameter for different storage backend drivers when managing an existing snapshot. By default it is set to fit the LVM driver.
The
list_endpoints
method of the v3EndPointsClient
class now has an additional**params
argument that enables passing additional information in the query string of the HTTP request.
Add
--save-state
option to allow saving state of cloud before tempest run.
Add validation schema for Nova server diagnostics API
Add show host API to the volume v2 hosts_client library. This feature enables the possibility to show details for a host.
Add show snapshot metadata item API to v2 snapshots_client library. This feature enables the possibility to show a snapshot’s metadata for a specific key.
Add show volume metadata item API to v2 volumes_client library. This feature enables the possibility to show a volume’s metadata for a specific key.
Define v3 volumes_client for the volume service as a library interface, allowing other projects to use this module as a stable library without maintenance changes. Add show volume summary API to v3 volumes_client library, min_microversion of this API is 3.12.
volumes_client(v3)
Define v3 backups_client for the volume service as a library interface, allowing other projects to use this module as a stable library without maintenance changes. Add update backup API to v3 backups_client library, min_microversion of this API is 3.9.
backups_client(v3)
Add groups and group_types clients for the volume service as library. Add tempest tests for create group, delete group, show group, and list group volume APIs.
Define v2 quota_classes_client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.
quota_classes_client(v2)
A new configuration flag api_v2_admin is introduced in the identity feature flag group to allow for enabling/disabling all identity v2 admin tests. The new flag only applies when the existing api_v2 flag is set to True
Add additional API endpoints to the identity v2 client token API: - list_endpoints_for_token - check_token_existence
A new
attr
decorator has been added in thetempest.lib.decorators
module. For example, use it to tag specific tests, which could be leveraged by test runners to run only a subset of Tempest tests.
A new
related_bug
decorator has been added totempest.lib.decorators
. Use it to decorate and tag a test that was added in relation to a launchpad bug report.
Move base_client from tempest.lib.services.volume.v3 to tempest.lib.services.volume, so if we want to add new interfaces based on a v2 client, we can make that v2 client inherit from volume.base_client.BaseClient to get microversion support, and then to make the new v3 client inherit from the v2 client, thus to avoid the multiple inheritance.
Define v2.0
tags_client
for the network service as a library interface, allowing other projects to use this module as a stable library without maintenance changes.tags_client(v2.0)
Pause teardown When pause_teardown flag in tempest.conf is set to True a pdb breakpoint is added to tearDown and tearDownClass methods in test.py. This allows to pause cleaning resources process, so that used resources can be examined. Closer examination of used resources may lead to faster debugging.
Added tempest workspace remove
--name <workspace_name> --rmdir
feature to delete the workspace directory as well as entry.
Upgrade Notes¶
Switch to use Glance v2 APIs in volume tests, by adding the Glance v2 client images_client.
The Heat API tests have been removed from tempest, they were unmaintained. The future direction of api test for heat is their in-tree Gabbi tests
The volume config option ‘api_v3’ default is changed to
True
because the volume v3 API is CURRENT.
verify_tempest_config command starts using extension_client of cinder v2 API only, because cinder v3 API is current and v2 and v1 are deprecated and v3 extension API is the same as v2. Then we can reuse the v2 client for v3 API also.
Deprecation Notes¶
Image APIs in compute are deprecated, Image native APIs are recommended. And Glance v1 APIs are deprecated and v2 APIs are current. Image client compute_images_client and Glance v1 APIs are removed in volume tests.
The config option
forbid_global_implied_dsr
from theIdentityFeature
group is now deprecated. This feature flag was introduced to support testing of old OpenStack versions which are not supported anymore.
Deprecate default value for configuration parameter v3_endpoint_type of identity section in OpenStack Pike and modify the default value to publicURL in OpenStack Q release.
Deprecate class BaseClient from volume.v3.base_client and move it to volume.base_client.
tempest.lib.services.volume.v3.base_client.BaseClient
(newtempest.lib.services.volume.base_client.BaseClient
)
Bug Fixes¶
When receiving nullable list as a response body, tempest.lib rest_client module raised an exception without valid json deserialization. A new release fixes this bug.
16.0.0¶
Prelude¶
This release indicates end of support for Mitaka in Tempest.
New Features¶
Add a new client to handle the OAUTH consumers feature from the identity API.
Add missing API call, list all role inference rules, to the roles_client library. This feature enables the possibility of listing all role inference rules in the system.
Add cascade parameter to volumes_client. This option provides the ability to delete a volume and have Cinder handle deletion of snapshots associated with that volume by passing an additional argument to volume delete, “cascade=True”.
Define the compute server evacuate client method in the servers_client library.
Add the list auth projects API to the identity client library. This feature enables the possibility to list projects that are available to be scoped to based on the X-Auth-Token provided in the request.
Add versions_client module for image service. This new module provides list_versions() method which shows API versions from Image service.
Add the list security groups by server API to the servers_client library. This feature enables the possibility to list security groups for a server instance.
Add versions_client module for identity service. This new module provides list_versions() method which shows API versions from Identity service.
Add versions_client module for volume service. This new module provides list_versions() method which shows API versions from Volume service.
Interface show_quota_set of compute quotas_client has been extended to include the argument “detail”, which allows for detailed quota set information for a project to be retrieved, if set to True.
Add remote_client under tempest.lib. This remote_client under tempest.lib is defined as stable interface, and now this module provides the following essential methods.
exec_command
validate_authentication
ping_host
Adds a new cli option to tempest run,
--combine
, which is used to indicate you want the subunit stream output combined with the previous run’s in the testr repository
Add update encryption type API to the v2 encryption_types_client library. This feature enables the possibility to update an encryption type for an existing volume type.
Add the unmanage volume API service method in v2 volumes_client library. Define v2 volume_manage_client client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.
volume_manage_client(v2)
Add server tags APIs to the servers_client library. This feature enables the possibility of updating, deleting and checking existence of a tag on a server, as well as updating and deleting all tags on a server.
Define volume transfers service clients as libraries. The following volume transfers service clients are defined as library interface.
transfers_client(v2)
Upgrade Notes¶
The
JSON_ENC
andTXT_ENC
option in the_error_checker
section have been added with additional content-type which are defined in RFC7231 but missing in the current rest_client.py file. The lack of these additional content-type will cause defcore test to fail for OpenStack public cloud which uses tomcat module in the api gateway. The additions areapplication/json;charset=utf-8
,text/html;charset=utf-8
,text/plain;charset=utf-8
The default value of rand_name()’s prefix argument is changed to ‘tempest’ from None to identify resources are created by Tempest.
The volume config option ‘api_v1’ default is changed to
False
because the volume v1 API has been deprecated since Juno release.
The call_until_true of test module is removed because it was marked as deprecated and Tempest provides it from test_utils as a stable interface instead. Please switch to use test_utils.call_until_true if necessary.
Remove Cinder v1 API tests. Cinder v1 API has been deprecated since Juno release, and Juno is not supported by current Tempest. Then Cinder v1 API tests are removed from Tempest.
The deprecated config option
allow_port_security_disabled
fromcompute_feature_enabled
group has been removed.
Below deprecated config options from compute group have been removed. Corresponding config options already been available in validation group.
compute.image_ssh_user
(available asvalidation.image_ssh_user
)compute.ssh_user
(available asvalidation.image_ssh_user
)scenario.ssh_user
(available asvalidation.image_ssh_user
)compute.network_for_ssh
(available asvalidation.network_for_ssh
)compute.ping_timeout
(available asvalidation.ping_timeout
)
The deprecated config option ‘dvr_extra_resources’ from network group has been removed. This option was for extra resources which were provisioned to bind a router to Neutron L3 agent. The extra resources need to be provisioned in Liberty release or older, and are not required since Mitaka release. Current Tempest doesn’t support Liberty, so this option has been removed from Tempest.
The deprecated config option ‘reseller’ from identity_feature_enabled group has been removed.
The deprecated
volume_services
option in thevolume_feature_enabled
section has now been removed.
Tempest now defaults to using Keystone v3 API for the authentication, because Keystone v3 API is CURRENT and the v2 API is deprecated.
Deprecation Notes¶
The
deactivate_image
configuration switch from theconfig
module is deprecated. It was added to support the older-than-kilo releases which we don’t support anymore.
The
dvr_extra_resources
configuration switch from theconfig
module is deprecated. It was added to support the Liberty Release which we don’t support anymore.
Glance v1 APIs are deprecated and v2 are current. Tempest should tests only v2 APIs. Below API version selection config options for glance have been deprecated and will be removed in future.
CONF.image_feature_enabled.api_v2
CONF.image_feature_enabled.api_v1
The resources_prefix is marked as deprecated because it is enough to set ‘tempest’ as the prefix on rand_name() to identify resources which are created by Tempest and no projects set this option on OpenStack dev community.
The
skip_unless_attr
decorator in lib/decorators.py has been deprecated, please use the standardtesttools.skipUnless
andtesttools.skipIf
decorators.
The
skip_unless_config
andskip_if_config
decorators in theconfig
module have been deprecated and will be removed in the Queens dev cycle. Use thetesttools.skipUnless
(or a variation of) instead.
Volume v1 API is deprecated and the v3 are CURRENT. Tempest doesn’t need to test the v1 API as the default. The volume config option ‘api_v1’ has been marked as deprecated.
Deprecate the client_parameters argument in tempest.lib.services.clients.ServiceClients. The parameter is actually not honoured already - see https://bugs.launchpad.net/tempest/+bug/1680915
Deprecate volume v2 transfers resource methods from volumes_client(v2) libraries. Same methods are available in new transfers service client: transfers_client(v2) The following methods of volume v2 volumes_clients have been deprecated:
create_volume_transfer (v2.volumes_client)
show_volume_transfer (v2.volumes_client)
list_volume_transfers (v2.volumes_client)
delete_volume_transfer (v2.volumes_client)
accept_volume_transfer (v2.volumes_client)
Bug Fixes¶
Fix below volume v2 service clients to make v2 API call: Bug#1667354
SchedulerStatsClient
CapabilitiesClient
The ‘sahara’ config option in the ‘service-available’ group has been moved to the sahara plugin (openstack/sahara-tests) along with tests and service client during the Ocata timeframe. A ‘sahara’ config option was left over on Tempest side, and it’s removed now. As long as the sahara plugin is installed, this change as no impact on users of sahara tests.
Other Notes¶
OpenStack Releases Supported after this release are Newton and Ocata
The release under current development as of this tag is Pike, meaning that every Tempest commit is also tested against master branch during the Pike cycle. However, this does not necessarily mean that using Tempest as of this tag will work against Pike (or future releases) cloud.
15.0.0¶
Prelude¶
This release is marking the start of Ocata release support in Tempest
New Features¶
As in the [doc]: http://developer.openstack.org/api-ref/image/v2/metadefs-index.html, there are some apis are not included, add them.
namespace_objects_client(v2)
namespace_tags_client(v2)
Add the implied roles feature API to the roles_client library. This feature enables the possibility to create inferences rules between roles (a role being implied by another role).
Define v2 snapshot_manage_client client for the volume service as library interfaces, allowing other projects to use this module as stable libraries without maintenance changes.
snapshot_manage_client(v2)
Added customized JSON schema format checker for ‘date-time’ format. Compute response schema will be validated against customized format checker.
Upgrade Notes¶
The default value for the
allow_port_security_disabled
option in thecompute-feature-enabled
section has been changed fromFalse
toTrue
.
The default value for the
reseller
option in theidentity-feature-enabled
section has been changed fromFalse
toTrue
.
The default value for the
volume_services
option in thevolume_feature_enabled
section has been changed fromFalse
toTrue
.
The deprecated compute microversion config options from ‘compute-feature-enabled’ group have been removed. Those config options are available under ‘compute’ group to configure the min and max microversion for compute service. * CONF.compute.min_microversion * CONF.compute.max_microversion
Below deprecated config options from compute group have been removed. Corresponding config options already been available in validation group.
compute.use_floatingip_for_ssh
(available asvalidation.connect_method
)compute.ssh_auth_method
(available asvalidation.auth_method
)compute.image_ssh_password
(available asvalidation.image_ssh_password
)compute.ssh_shell_prologue
(available asvalidation.ssh_shell_prologue
)compute.ping_size `` (available as ``validation.ping_size
)compute.ping_count `` (available as ``validation.ping_count
)compute.floating_ip_range `` (available as ``validation.floating_ip_range
)
The deprecated input-scenario config options and group have been removed. The input scenarios functionality already being removed from tempest and from this release, their corresponding config options too.
Below deprecated network config options have been removed. Those config options already been renamed to below meaningful names.
tenant_network_cidr
(removed) ->project_network_cidr
tenant_network_mask_bits
(removed) ->project_network_mask_bits
tenant_network_v6_cidr
(removed) ->project_network_v6_cidr
tenant_network_v6_mask_bits
(removed) ->project_network_v6_mask_bits
tenant_networks_reachable
(removed) ->project_networks_reachable
Deprecation Notes¶
The
allow_port_security_disabled
option in thecompute-feature-enabled
section is now deprecated.
The
reseller
option in theidentity-feature-enabled
section is now deprecated.
The
volume_services
option in thevolume_feature_enabled
section is now deprecated.
Other Notes¶
OpenStack releases supported at this time are Mitaka, Newton, and Ocata.
The release under current development as of this tag is Pike, meaning that every Tempest commit is also tested against master during the Pike cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Pike (or future releases) cloud.
14.0.0¶
Prelude¶
This release is marking the end of Liberty release support in Tempest
New Features¶
The cred_provider abstract class which serves as the basis for both of tempest’s cred providers, pre-provisioned credentials and dynamic credentials, is now a library interface. This provides the common signature required for building a credential provider.
The cred_client module was added to tempest.lib. This module provides a wrapper to the keystone services client which provides a uniform interface that abstracts out the differences between keystone api versions.
As in the [doc]: http://developer.openstack.org/api-ref/image/v2/metadefs-index.html, there are some apis are not included, add them.
namespace_properties_client(v2)
Define the identity service role_assignments_client as a library. Add role_assignments_client to the library interface so the other projects can use this module as a stable library without any maintenance changes.
A Neutron Service Providers client is added to deal with resources of the ‘/service-providers’ route.
A new optional parameter port for ssh client (tempest.lib.common.ssh.Client) to specify destination port for a host. The default value is 22.
Define the Volume v3 service clients as library interfaces, allowing other projects to use these modules as stable libraries without maintenance changes.
messages_client(v3)
The volume_limits client was added to tempest.lib.
Define volume service clients as libraries. The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
volumes_client(v1)
volumes_client(v2)
capabilities_client(v2)
scheduler_stats_client(v2)
Upgrade Notes¶
Add an error translation to list_versions() of versions_client of both compute and network. This can affect users who are expecting that these clients return error status code instead of the exception. It is needed to change the code for handling the exception like the other clients code.
The Stress tests framework and all the stress tests have been removed.
All tests for the Ironic project have been removed from Tempest. Those exist as a Tempest plugin in the Ironic project.
The Negative Tests Generator has been removed (it was not used by any Tempest tests).
All tests for the Sahara project have been removed from Tempest. They now live as a Tempest plugin in the
openstack/sahara-tests
repository.
Deprecation Notes¶
The api_extensions config option in the compute-feature-enabled group is now deprecated. This option will be removed from tempest when all the OpenStack releases supported by tempest no longer support the API extensions mechanism. This was removed from Nova during the Newton cycle, so this will be removed at the Mitaka EOL.
The bootable config option in the volume_feature_enabled group is removed because the corresponding feature os-set_bootable has been implemented 2.5 years ago and all OpenStack versions which are supported by Tempest should support the feature.
Other Notes¶
OpenStack releases supported at this time are Mitaka and Newton.
The release under current development as of this tag is Ocata, meaning that every Tempest commit is also tested against master during the Ocata cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ocata (or future releases) cloud.
13.0.0¶
Prelude¶
This release is marking the start of Newton release support in Tempest
New Features¶
Define identity service clients as libraries. Add new service clients to the library interface so the other projects can use these modules as stable libraries without any maintenance changes.
identity_client(v2)
groups_client(v3)
trusts_client(v3)
users_client(v3)
identity_client(v3)
roles_client(v3)
inherited_roles_client(v3)
credentials_client(v3)
Define volume service clients as libraries. The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
backups_client
encryption_types_client (v1)
encryption_types_client (v2)
qos_clients (v1)
qos_clients (v2)
snapshots_client (v1)
snapshots_client (v2)
Define the v1 and v2 types_client clients for the volume service as library interfaces, allowing other projects to use these modules as stable libraries without maintenance changes.
Upgrade Notes¶
the already deprecated tempest-cleanup standalone command has been removed. The corresponding functionalities can be accessed through the unified tempest command (tempest cleanup).
Deprecation Notes¶
Oslo.utils provides same method get_ipv6_addr_by_EUI64, so deprecate it in Newton and remove it in Ocata.
The
call_until_true
function is moved from thetempest.test
module to thetempest.lib.common.utils.test_utils
module. Backward compatibility is preserved until Ocata.
Other Notes¶
OpenStack releases supported at this time are Liberty, Mitaka, and Newton.
The release under current development as of this tag is Ocata, meaning that every Tempest commit is also tested against master during the Ocata cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Ocata (or future releases) cloud.
12.2.0¶
New Features¶
RestClient now supports setting timeout in urllib3.poolmanager. Clients will use CONF.service_clients.http_timeout for timeout value to wait for http request to response.
KeystoneAuthProvider will accept http_timeout and will use it in get_credentials.
Define identity service clients as libraries. The following identity service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
endpoints_client(v3)
policies_client (v3)
regions_client(v3)
services_client(v3)
projects_client(v3)
The Tempest plugin interface contains a new optional method, which allows plugins to declare and automatically register any service client defined in the plugin.
tempest.lib exposes a new stable interface, the clients module and ServiceClients class, which provides a convenient way for plugin tests to access service clients defined in Tempest as well as service clients defined in all loaded plugins. The new ServiceClients class only exposes for now the service clients which are in tempest.lib, i.e. compute, network and image. The remaining service clients (identity, volume and object-storage) will be added in future updates.
A new optional interface TempestPlugin.get_service_clients is available to plugins. It allows them to declare any service client they implement. For now this is used by tempest only, for auto-registration of service clients in the new class ServiceClients.
A new singleton class clients.ClientsRegistry is available. It holds the service clients registration data from all plugins. It is used by ServiceClients for auto-registration of the service clients implemented in plugins.
A new helper method service_client_config has been added to the stable module config.py that returns extracts from configuration into a dictionary the configuration settings relevant for the initialization of a service client.
Define volume service clients as libraries The following volume service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
availability_zone_client(v1)
availability_zone_client(v2)
extensions_client(v1)
extensions_client(v2)
hosts_client(v1)
hosts_client(v2)
quotas_client(v1)
quotas_client(v2)
services_client(v1)
services_client(v2)
Upgrade Notes¶
The
nova_cert
option default is changed toFalse
. The nova certification management APIs were a hold over from ec2, and are not used by any other parts of nova. They are deprecated for removal in nova after the newton release. This makes false a more sensible default going forward.
The previously deprecated Javelin utility has been removed from Tempest. As an alternative Ansible can be used to construct similar yaml workflows to what Javelin used to provide.
Deprecation Notes¶
The new clients module provides a stable alternative to tempest classes manager.Manager and clients.Manager. manager.Manager only exists now to smoothen the transition of plugins to the new interface, but it will be removed shortly without further notice.
12.1.0¶
New Features¶
Adds a network version client for querying Neutron’s API version discovery URL (“GET /”).
Tempest library auth interface now supports scope. Scope allows to control the scope of tokens requested via the identity API. Identity V2 supports unscoped and project scoped tokens, but only the latter are implemented. Identity V3 supports unscoped, project and domain scoped token, all three are available.
Adds the tempest run command to the unified tempest CLI. This new command is used for running tempest tests.
Adds tempest workspaces command and WorkspaceManager. This is used to have a centralized repository for managing different tempest configurations.
Adds subunit-describe-calls. A parser for subunit streams to determine what REST API calls are made inside of a test and in what order they are called.
Input can be piped in or a file can be specified
Output is shortened for stdout, the output file has more information
Tempest library auth interface now supports filtering with catalog name. Note that filtering by name is only successful if a known service type is provided.
Define identity service clients as libraries The following identity service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
endpoints_client(v2)
roles_client(v2)
services_client(v2)
tenants_client(v2)
users_client(v2)
Define image service clients as libraries The following image service clients are defined as library interface, so the other projects can use these modules as stable libraries without any maintenance changes.
image_members_client(v1)
images_client(v1)
image_members_client(v2)
images_client(v2)
namespaces_client(v2)
resource_types_client(v2)
schemas_client(v2)
A new test_utils module has been added to tempest.lib.common.utils. It should hold any common utility functions that help writing Tempest tests.
A new utility function called call_and_ignore_notfound_exc has been added to the test_utils module. That function call another function passed as parameter and ignore the NotFound exception if it raised.
Define routers_client as stable library interface. The routers_client module is defined as library interface, so the other projects can use the module as stable library without any maintenance changes.
The RestClient (in tempest.lib.common.rest_client) now supports POSTing and PUTing data with chunked transfer encoding. Just pass an iterable object as the body argument and set the chunked argument to True.
A new generator called chunkify is added in tempest.lib.common.utils.data_utils that yields fixed-size chunks (slices) from a Python sequence.
Upgrade Notes¶
The input scenarios functionality no longer exists in tempest. This caused a large number of issues for limited benefit and was only used by a single test, test_server_basic_ops. If you were using this functionality you’ll now have to do it manually with a script and/or tempest workspaces
The integrated dashboard scenario test has been removed and is now in a separate tempest plugin tempest-horizon. The removed test coverage can be used by installing tempest-horizon on the server where you run tempest.
The deprecated legacy credential provider has been removed. The only way to configure credentials in tempest now is to use the dynamic or preprovisioned credential providers
All tests for the Trove project have been removed from tempest. They now live as a tempest plugin in the trove project.
The location on disk that the tempest init command looks for has changed. Previously it would attempt to use python packaging’s data files to guess where setuptools/distutils were installing data files, which was incredibly unreliable and depended on how you installed tempest and which versions of setuptools, distutils, and python you had installed. Instead, now it will use either /etc/tempest, $XDG_CONFIG_PATH/.config/tempest, or ~/.tempest/etc (attempted in that order). If none of these exist it will create an empty ~/.tempest/etc directory. If you were relying on the previous behavior and none of these directories were being used you will need to move the files to live in one of these directories.
Deprecation Notes¶
tempest.lib.common.utils.misc.find_test_caller has been moved into the tempest.lib.common.utils.test_utils module. Calling the find_test_caller function with its old location is deprecated.
All the options in the input-scenario group are now deprecated. These were only used in tree by the now removed input scenarios functionality in test_server_basic_ops. They were only deprecated because there could be external consumers via plugins. They will be removed during the Ocata cycle.
12.0.0¶
Prelude¶
This release is marking the end of Kilo release support in Tempest
Other Notes¶
OpenStack Releases Supported after this release are Liberty and Mitaka The release under current development as of this tag is Newton, meaning that every Tempest commit is also tested against master during the Newton cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Newton (or future releases) cloud.
11.0.0¶
Prelude¶
This release is marking the start of Mitaka release support in tempest
New Features¶
Tempest library interface addition(API Microversion testing interfaces).
Compute Microversion testing support in Service Clients.
Other Notes¶
OpenStack Releases Supported at this time are Kilo, Liberty, Mitaka The release under current development as of this tag is Newton, meaning that every Tempest commit is also tested against master during the Newton cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Newton (or future releases) cloud.
10.0.0¶
Prelude¶
This release includes the addition of the stable library interface for tempest. This behaves just as tempest-lib did prior to this, but instead it lives directly in the tempest project. For more information refer to the library docs.
New Features¶
Tempest library interface
Other Notes¶
OpenStack Releases Supported at this time are the same as in the previous release 9, Kilo and Liberty.
The release under current development as of this tag is Mitaka, meaning that every Tempest commit is also tested against master during the Mitaka cycle. However, this does not necessarily mean that using Tempest as of this tag will work against a Mitaka (or future releases) cloud.
Start using reno for managing release notes.