Current Series Release Notes

0.10.0

Upgrade Notes

  • Python 2.7 support has been dropped. Last release of sahara-tests to support python 2.7 is 0.9.1, the first version that can be used with OpenStack Train. The minimum version of Python now supported by sahara-tests is Python 3.5.

  • Removed nova-network configuration. Nova network has been fully removed from the OpenStack codebase, in all releases supported by sahara-tests.

0.9.1

Bug Fixes

  • Fix a strange error where the internally generated test does not start because sahara_tests.scenario is not found when running inside a virtualenv.

0.9.0

Prelude

Stein test templates are now available, while Ocata test templates have been removed.

New Features

  • Allow to enable boot_from_volume on node group templates when running scenario tests with APIv2.

  • sahara-scenario now supports testing the S3 API for job binaries and data sources, a feature introduced in Rocky. The code can be enabled using the “s3” feature and various templates now runs an S3-based job too when the feature is enabled from the command line.

  • A folder with scenario templates for Stein has been added. It is a subset of the templates in the main directory.

Deprecation Notes

  • The Ocata-specific job templates have been removed. This means that starting from this release Ocata is not supported (it is under Extended Maintenance now).

Bug Fixes

  • Properly handle more use cases when only Keystone v3 is enabled and/or its service URI is missing the /v3 suffix.

0.8.0

Prelude

Tempest tests now support APIv2.

Scenario tests now support APIv2.

New Features

  • The Tempest plugin provides an APIv2 DataProcessing client and tempest tests can be executed against APIv2 too. The type of API used is driven by a tempest.conf configuration key (data_processing.use_api_v2 for API tests, data_processing.api_version_saharaclient for client and CLI tests)

  • sahara-scenario supports feature sets. When passing specific feature tags to sahara-scenario, additional job templates and EDP jobs marked with those tags will be loaded.

  • The fully generated YAML file is printed when the verbose mode is enabled.

  • Scenario tests can be executed against APIv2. The usage of APIv2 is enabled through a new command line argument for sahara-scenario (–v2, -2).

Bug Fixes

  • Fix the test template for storm 1.2.

  • When passing the plugin/version parameters to sahara-scenario, users can now specify additional YAML templates which will be merged to the default YAMLs, instead of being ignored.

0.7.0

Prelude

Rocky test templates are now available.

New Features

  • New basic scenario test template for CDH 5.13.0.

  • The api_version_saharaclient variable now controls the Sahara API version used not only by the Tempest.lib-based clients tests, but also by the Tempest CLI tests.

  • Added basic S3 API tests (job binaries and data sources) to the Tempest plugin. The tests are disabled by default and can be enabled using a new tempest.conf key (data-processing-feature-enabled.s3).

  • A folder with scenario templates for Rocky was added. It is a subset of the templates in the main directory, and includes all non-deprecated plugin/versions.

  • The available default test templates now supports also Spark 2.3, Storm 1.2, Vanilla 2.7.5, and the Ambari-based HDP 2.6 and 2.5.

Upgrade Notes

  • sahara-scenario now requires stestr.

Deprecation Notes

  • The saharaclient_version option in the data-processing group has been renamed to api_version_saharaclient.

Bug Fixes

  • The node group template creation test for the CLI client does not fail anymore when the Sahara plugin selected for testing provides more than one version.

  • The CDH 5.11 test template, previously only available for queens, was added to main (unversioned) templates directory.

Other Notes

  • Updated the list of supported plugin/versions used by some Tempest and Tempest-based tests to cover all the combinations available in the Sahara releases supported by sahara-tests.

0.6.0

Prelude

Queens test templates are now available, while Newton test templates have been removed.

New Features

  • A folder with scenario templates for Queens was added. It is a subset of the templates in the main directory.

  • The available templates now supports Spark 2.2 and Vanilla 2.8.2.

  • The public network field can be omitted from the configuration file of the scenario, enabling the testing when only the project network is used.

Deprecation Notes

  • The Newton-specific job templates have been removed. This means that starting from this release Newton is not supported (it has been EOL for a while).

0.5.0

Prelude

Pike test templates are now available, while Mitaka test templates have been removed.

New Features

  • A folder with scenario templates for Pike was added. It is a subset of the templates in the main directory.

  • Some requirements have been raised (especially Tempest).

Deprecation Notes

  • The Mitaka-specific job templates have been removed. This means that starting from this release Mitaka is not supported (it has been EOL for a while).

0.4.1

Prelude

Create new cli test in Sahara Tempest plugin for node group templates

Clean resources created in CLI tests

Create new cli tests in Sahara Tempest plugin for job types

Long overdue general updates of the test templates

New Features

  • Now we can filter node group template with column flag by plugin and check if it was filtered successfully

  • Now we can filter job types with column flag and also save config file of any job type to the specified file.

Bug Fixes

  • Fix bug in which after running CLI tests, projects and networks created for these tests were not deleted at the end of the run.

  • The default templates used by tests have been updated; added (MapR, Ambari and Storm, and some versions of CDH), or removed (obsolete versions of Vanilla and CDH).

Other Notes

  • The dependency on tempest.scenario.manager.py has been removed. There are still dependencies on internal Tempest interfaces but they are more difficult to replace due to lack of tempest.lib alternatives, and scenario.manager.py is undergoing an heavy refactoring.

0.4.0

Prelude

Fixes and addition for the API and CLI tests.

Support for CDH 5.9 in scenario tests.

Less parameters required for scenario tests.

Ocata test templates are now available, while Liberty test templates have been removed.

New Features

  • The Tempest-based tests have received an increase of the coverage for both API and CLI tests (job, plugins, templates; negative testing).

  • CDH 5.9 can be tested thanks to the addition of the specific templates.

  • Few parameters are not required anymore by the templates in scenario tests; a default value is provided (name of templates, etc).

  • The flavors used in templates are now parameters too.

  • If a flavor name is provided in addition to its specification and a flavor with that name exists, it is used and not created again.

  • The dependencies on non-public Tempest interfaces have been removed.

  • A folder with test templates with Ocata has been created and initialized starting from the templates in the main directory, following the status of the jobs tested on the Sahara CI.

  • Capture and report the timestamp in scenario tests when an event starts and when an exception is triggered.

Upgrade Notes

  • the name of the variables/parameters used for the name of the images in the scenario tests have been changes to follow a more consistent pattern.

Deprecation Notes

  • The Liberty-specific job templates have been removed. This means that starting from this release Liberty is not supported (it has been EOL for a while).

  • The MapR 5.0.0 test template have been removed from the master branch as well.

Critical Issues

  • The Tempest plugin was fixed after the removal of the data_processing plugin from the tempest repository. Most of the work was in place, only a small change was missing.

Bug Fixes

  • The artifacts created during the execution of CLI tests are properly cleaned at the end of the tests.

Other Notes

  • The documentation was improved (scenario tests) and extended (Tempest plugin).

0.3.0

Prelude

Added ability to use clouds.yaml with scenario tests

Client tests have been imported from the Sahara repository.

Migrate sahara cli tests from saharaclient to sahara-tests

Yaml-files for deprecated plugins was removed

Tests no longer depend on fake plugin to run

Sahara Tests plugin now uses tempest stable interface

New Features

  • User can now use clouds.yaml in the format defined by os-client-config to specify the auth values wanted on a scenario test.

  • The tests for the official Python clients have been moved here from the Sahara repository. They are based on the Tempest libraries even they do not follow the Tempest guidelines (as they need to test the Python clients, they do not use the native Tempest clients).

Upgrade Notes

  • Migration from novaclient.v2.images to glanceclient

Bug Fixes

  • Removed yaml-files for Kilo release

  • Removed unused yaml-files for master branch

Other Notes

  • Adapt Sahara Tests code to stop relying only on the fake plugin and use the default plugin available. However, it’s worth noting that - if available - the fake plugin will be used.

  • Sahara Tests plugin is adapted to use in-tree client, which was migrated from Tempest code. Also, there’s a new stable interface for Service Clients in Tempest, so this change adapts the code to use it.

0.2.0

Prelude

Discovery of data sources with relative paths is now fixed.

Fix default resource discovery from the installed package.

Migrate auth system from keystoneclient to keystoneauth

Removed the need of a .testr.conf file when calling the test runner.

Bug Fixes

  • Datasources with relative paths are now properly found from the default resources.

  • The default set of resources (test templates for each plugin, etc) can now be properly discovered when the package is installed.

  • A .testr.conf file was previously required in the runner execution directory, now this is handled internally.

Other Notes

  • The default timeout for cluster polling was raised from 1800 to 3600 seconds.

0.1.0

New Features

  • Sahara API tests have been imported from Tempest and made available using the Tempest Plugin Interface.

Other Notes

  • OpenStack reno integration was added for managing release notes