CHANGES
=======

0.0.3
-----

* Release 0.0.3
* Add aborted and broken flags to sla checker
* Fix consuming in broker
* Add scenario to boot and rebuild server
* Remove DeployEngines in favor of DeploymentEngine
* Replaced deprecated 'use deployment' with 'deployment use'
* Replaced incorrect BenchmarkScenarios list command in doc
* Remove keystone v3 jobs from MOS
* Updated from global requirements
* Add required_cinder_services validator
* Fix kwargs in nova migrate scenatio
* Specify alt_user in tempest config
* [Vm] Restore fixed IP support in boot_runcommand_delete
* Add heat list_stack_and_events scenario
* Add keystone create_and_list_services scenario
* Remove redundant option "detailed: True"
* [Docs] Remove subsection numberings in the Rally tutorial
* Add "pause and unpause" scenario for Nova
* Add unit test for keystone._get_service_by_name
* Pass username/password to client
* Remove redundant type
* [Docs] Add tutorial chapter about task templates
* configure ssl in deployment config
* Adds endpoint to --fromenv deployment creation
* Add "suspend and resume" scenario for Nova
* Add murano benchmarks
* Lazy import of keystoneclient in osclients
* Add a --uuids-only option to rally task list
* Load OpenStack clients only if those are used
* Modifies VMScenario._run_command function, to handle single bash cmd
* [Sahara] Add support for anti-affinity
* [Sahara] Improved provisioning plugin support
* Adds uninstall_tempest to API
* Fixes Tempest path update to use abspath
* Fix 'rally task results' output for running task
* Fixed typo in rally/benchmark/validation.py#L415
* Add keystone create_and_delete_service scenario
* Fix rally task start output format
* Add the ability to specify versions for clients
* Add boot server scenario for EC2 API
* [spec] Proposal for new task input file format
* Updated from global requirements
* Fix permissions issue in docker image
* Show precision of 2 in coverage job
* Add keystone get_entities scenario
* [Tempest]: Try to find py27 interpreter
* Removes from tox.ini flake configuration _ as builtin
* [Ceilometer] Add CeilometerAlarms.create_alarm_and_get_history scenario
* Added 'alias' decorator
* [Tempest] add cleanup for clone method
* Fix some typos in comments/doc strings
* Improve hacking and remove useless unit tests
* Add a maximum concurrency option to rps runner
* [unblock-gates] Reduce cirteria of success for Neutron
* Move create-update-delete-stack-inplace test

0.0.2
-----

* Release 0.0.2
* Use get_client when creating ceilometer client instance
* Fix texts in docs
* [Tempest]: Handle errors from `requests` in ConfigGen
* Updated from global requirements
* Add murano client to osclient
* Updated from global requirements
* Add Python 3.4 as supported interpreters
* Fix texts and images in docs
* Fix printing too much output for show secgroup
* Fix bash autocompletion of "rally task sla_check"
* Introduce in-tree Rally specs
* Unify RPS and constant runners and tune constant runner
* Implement create_and_upload_volume_to_image scenario for Cinder
* Fix unstable job
* Unblock gates
* Add create-check-delete stack scenario
* Add create-suspend-resume-delete stack scenario
* [Sahara] Fix Job Binaries download
* [Sahara] cleanup clusters correctly
* Add scenarios for live migration of an instance with volumes
* Mark 'rally use {deployment|task|verification}' deprecated
* Add security group for neutron ports
* [Sahara] added a cluster delete timeout
* Updated from global requirements
* Fix iterate_per_tenants and adds docstring
* Add the context custom_image
* Refactor ImageGenerator and validation
* Adds Designate server-list scenario
* Use `DEFAULT_CONFIG' to specify context's config
* Default DNS servers for servers in Neutron network
* **kwargs in create_image_and_boot_instances only for server
* Fix db configuration in rally_install.sh
* Updated from global requirements
* Add warnings to not use Tempest benchmarks against production cloud
* Add base for Swift API Benchmarks - Patch-1
* Removed **kwargs for volume in create_and_attach_volume
* Optional arguments for boot_and_delete_server_with_secgroups
* Creating OS clients from the environment variables
* Sets new users' endpoint_type to default to admin's
* Remove all non-incubated oslo code
* Adds Nova keypair tests
* Updated from global requirements
* Updated from global requirements
* Supress warning in cli methods that have sensetive output
* Add new script for gate jobs
* Fix security group setup when users_per_tenant is > 1
* Add list-stack-with-resources scenario
* Add few words in docs about installing docker
* More verbose WrongStatusException
* Add user-friendly message if the task cannot be deleted
* Updated from global requirements
* Add context that pre-creates heat stacks
* Add create and delete routers scenarios for Neutron
* Add floating-IP aware VM boot/delete utils
* Prefixing private atomic actions with `_`
* Change error message in secgroups scenario
* Abort scenario execution on SLA failure
* Updated from global requirements
* Fix rally task detailed --iterations-data
* Store keypair and secgroup data on per-user basis
* Mention Rally ReadTheDocs in README
* Don't create task db record during validation
* Add update + add/delete resource cases
* Add "update-inplace/replace resource" cases
* Don't launch stress cleanup in TempestContext
* Add create-update-delete stack scenario
* Support scenarios for updating neutron quotas
* Fix get_floating_ip lookup for {"id": net_id"}
* Looking for floating IP in all networks
* Publishing functional tests reports
* Make unstable-neutron job more stable
* Adds support for multiple tempest sources
* Fix issues in Mistral code
* Implement abort in scenario runners
* Add Mistral benchmark
* Shifting ._generate_random_name() call
* Replace ' with " in rally/benchmark and tests/unit
* Replace ' with " in tests/unit/verification
* Replace ' with " in tests/unit/[aas,cmd]
* Fix template paths in heat samples
* Add option to keep db settings in functional tests
* Replace ' with " in tests/unit/[common,deploy,doc,fixtures]
* Switch to the new Rally API
* Refactor rally.api
* Adds Designate server tests
* Neutron: remove ports before removing network
* Add aliases "rally task|deployment|verify use"
* Adds "scenario" to available verify sets
* Add plugin base
* Add a Python context that logs exceptions
* Replace ' with " in tests/unit/benchmark
* Replace ' with " in [doc, samples]
* Unblock Rally gates
* Replacing ' with " in rally/cmd
* Replace ' with " in rally/[objects,ui,verification]
* Log resource details on TimeoutException
* Replace ' with " in rally/deploy
* Adds postgresql to install_rally
* Disable security group for some neutron driver
* Replace deprecated max_failure_precent sla

0.0.1
-----

* Fix all py3 related issues
* Fix cinder scenario: create nested snapshots
* Correcting minor typos
* Refactor oslo.foo as oslo_foo
* Updated from global requirements
* Move tempest install to rally.api
* Allow setting DB for unittests via env & add psql support
* Replace ' with " in rally/benchmark
* Replace ' with " in rally/common
* Updated from global requirements
* Add cinder scenario: create volume from volume(clone)
* Add image arg for creating volume from image
* Allow using builtin functions in task templates
* Avoid mapping non-boot volume to boot device
* Improving error message
* Update location to samples
* [Sahara] Add Cloudera plugin support
* Refactoring benchmark/validation.py
* Add cold migrate scenario for Nova test
* Add extend_volume scenario for Cinder test
* Add list_snapshots scenario for Cinder test
* Removing "failed" column entirely from rally cli as well as db
* Adds Tempest source repo support for "Tempest" context
* Re-enabling Heat/Neutron resources test
* Allow user to choose Tempest remote repo
* Restructure Rally docs
* Improvements for scenario VMTasks.boot_runcommand_delete
* Make Rally gates more stable by moving things to unstable job
* Removes duplicate for generate_random_name
* Add unstable job file + return SLA for boot from volume
* Adding duration header to rally verify list query
* Replace ' with " in rally/*.py
* Moving rally/doc/samples to rally/samples
* Added support to password auth with key pair
* Unblock Rally gates cause by nova bug 1411525
* Adding duration header
* Created new single vm workload sample script using linpack
* Fix "regex" switch for verify
* Input task templates and task cmd cleanup
* Neutron network context support
* Adding dict.items() concatenate validation
* Pulls up _test_atomic_action_timer to TestCase
* Updated from global requirements
* Optional parameter "endpoint" is added to deployment configuration
* Adding documentation
* Replace ' with " in aas/rest
* Use proper 'open' mocking in unit tests
* Changes Timestamp to hold float instead of datetime
* Adds timestamps to scenarios iterations
* fix devstack URL
* Move files to common lib(Part 4)
* Add list_volumes scenario for Cinder test
* Updated from global requirements
* Move JSON_SCHEMA to rally.consts
* Updated from global requirements
* Use json.dumps builtin sorting during plot
* Support generation of report from N task
* Don't hard set keystone endpoint
* Python 3 fix mocking
* Skipping CliUtilsTestCase
* Python 3 fix '+' operand in dict in test_osclients.py
* Python 3 fix '+' operand in dict_items
* Fix py3k issues
* Add the flavor context
* use function next() instead of next() method on iterator objects
* Refactoring structure of rally/verification
* Python 3 maxint compatibility
* Wrongly formatted input task
* Adds support for instances with IPv6 addresses in VMScenario
* Move files to common lib(Part 3)
* [Sahara] refactoring _launch_cluster
* Use string.ascii_lowercase and string.ascii_uppercase only
* Move CinderVolumes tasks from rally.yaml to rally-neutron.yaml
* [Sahara] make job executions delete syncronized
* Use six.moves.range instead of xrange
* Moved module "uuidutils" from oslo-incubator to oslo.utils
* Use six.moves.map instead of itertools.imap
* Use from six.moves.urllib.parse instead of urlparse
* Fix router GW creation
* Refactor validation (part 1)
* syntax change for py3 incompatible
* Add Dockerfile
* Use six.moves.StringIO instead of StringIO.StringIO
* Use six.string_types instead of basestring
* Move files to common lib(Part 2)
* Move files to common lib(Part 1)
* Fix docstring checker
* Cover Rally with docstrings & Test this coverage
* DataBase: Delete BigText class
* Adding iteritems validation
* Sahara auto_security_group parameter added
* Use Network Context in security groups scenario
* Add Network Context Class
* Add list_servers scenario for Nova
* Remove SLA checks for NovaServers.boot_server_from_volume_and_delete
* Updates in Context order
* Fix cli tests vs MOS clouds
* Converting scenario context to dict
* Python 3 incompatible issue: iteritems
* Fix html rendering on failed functional tests
* Add swift service to pre-defined service list
* orchestrator refactor
* Adds Zaqar consumer-producer scenario
* Context tenants refactoring
* Small refactor in run function
* Changing default new user password
* 100% coverage in functional tests
* Updated from global requirements
* Add 'Stop scenario after several errors' feature req
* Configure gate-rally-dsvm-verify
* Fix nested snapshot job
* Updated from global requirements
* Change TaskNotFound traceback info to user-friendly message
* Fix the world (heat role was remove from devstack)
* Use HTTPS to fetch external js resources
* Add cinder stress scenario: nested snapshots
* Updated from global requirements
* Incorrect work rally info find for SLA
* Add scenario for booting vm with security groups
* Adds trove python client to rally.osclients
* Workflow documentation is now in infra-manual
* Updated from global requirements
* Added hacking rule for assertEqual(A in/not in B, True/False)
* add 'rally task list' filters
* Fix rally deployment check fails with https auth_url
* Further improvements in rally info
* Fix rps runner to use random users from context
* Fix functional test doc
* Add several new hacking rules
* rally info: smart substitutions in the non-ambiguous case
* Reduce amount of time in gates
* Add benchmark overview page to html report
* Add scenarios for security group
* Introudce benchmark.context() decorator
* Adds Nova vm live migration benchmark scenarios
* Engine changed to not crash on context creation
* Add a servers context and a cinder stress scenario
* Add aliases for debug levels in rally.log
* Fix typo in rally tempest config generator
* Add force_delete to safely delete created VMs
* Improvements for HTML output of rally verify results
* Remove redunant BenchmarkEngine.bind() method
* Refactoring: Chaning atomic ation usage in _resource_delete
* Refactoring: removing unused parameter **kwards
* Rename rally-scenarios to rally-jobs & add readme
* TEMPEST: refactor error handling while installation
* Improve unit tests for UserGenerator
* Modify sla failure rate
* Updated from global requirements
* Fix ZeroDivision when no results
* Fix cli arguments backward compatibility
* Fix problem with compare algorithm
* Updated from global requirements
* Changes in getting image id from image name functionality
* Fix invalid Ceilometer task samples
* add ability to refer deployment by uuid or name
* Fix defect of 'separate log level for Rally'
* Port to use oslo.config.fixture
* Fix issue with Pre-Grizzly deployment
* Updated from global requirements
* Base HTML templates and improvements for task report
* Remove check of verification smoke tests
* Sync oslo-incubator
* Fix Zaqar's cleanup
* Added separate log level for Rally debugging
* Updated from global requirements
* Fix typo in Ceilometer's class in the cleanup
* Refactor generic cleanup mechanism part 3
* Updated from global requirements
* Refactor generic cleanup mechanism part 2
* Refactor UserGenerator
* Fix tempest.conf subnets configuration
* Refactor generic cleanup mechanism part 1
* Add SLA to Designate scenarios
* Removing redundancy in (a==b) and 1 or 0
* Updated from global requirements
* Updated from global requirements
* Fix 2 typos and 2 invalid links in doc
* Sahara Cluster error state handling
* Enhance rally info
* Fix rally plugins
* Improve HTML report
* Modify rally commands deployment, task, and verify
* Adds Zaqar Basic scenarios
* Simplify and improve "rally info"
* Fixes fakes and adds version param
* Jobs with scaling benchmark
* Sahara Cluster scaling benchmark
* Add ability to compare two verification results
* Updated from global requirements
* Add functional tests for rally verify
* Refactor tempest verification and tests for it
* Add simple multi thread broker
* Improve plugins and apply loading for runners and contexts plugins
* Port to use oslo.i18n
* Port to use oslo.serialization
* Fix wrong argument in rally verify
* Add detailed description for rally commands
* Adding logic to clean up old rally installs
* Allow user specified tempest.conf
* Remove the obsolete tests_ci file
* Fix TempestConfig generator keystone API v3 issue
* Using Neutron names in Sahara
* Port to use oslo.utils
* Fix test__generate_quota_values_* fails
* Updated from global requirements
* Make full bash-completion
* Adding new scenarious for Heat
* Improvements for task plot2html
* Implement max_avg_duration criteria
* Fix CONFIG_SCHEMA in IterationTime in sla/base.py
* Updated from global requirements
* Fix bash completition setup
* Display proper missing argument name
* Fix messages in rally exceptions
* Add misspellings handling to "rally info"
* Improve the deployment's creation and deletion
* Adds Neutron update Port and router benchmarks:
* Fix error status for get_from_manager method
* don't call client.authenticate() only when it is authenticated
* Updated from global requirements
* Reorganize test module structure
* Change default endpoint_type to "public"
* Sahara plugins support
* Add volume option to boot server from volume and run script
* Expand ~ in file paths, specified in rally commands
* Remove run_tests.sh
* Improve CLI functional tests
* Call correct method to reset quotas
* Catch IOError in cliutils
* Add config file for rally-designate job
* Allow user specified endpoint_type in config
* Remove two unused exceptions
* Fixes InvalidBenchmarkConfig error while running tempest scenario
* Stop using intersphinx
* Added cleanup for Zaqar objects
* Adds Neutron update Network and subnets benchmarks:
* Sahara Job sequence benchmark added
* Adding optional parameters to Sahara cluster
* Fix bug in atomic actions
* User story - booting 400 servers on real cluster
* Add unit tests for rally.benchmark.scenarios
* Updated from global requirements
* Fix percentile in sla check message
* Remove extraneous vim editor configuration comments
* Fix AttributeError in admin_cleanup
* add bash completion script
* Store sla result to db
* Add task file validate command
* Adds the Zaqar client
* Add atomic action names tracking
* Add unit tests for rally.benchmark.context
* update rally.conf.sample
* Updated from global requirements
* Fix "rally info" failure in case of empty docstrings
* Rally cli functional testing
* Add hacking checks
* Fix ping_ip_address method in order to be run under BSDs
* Add @validation.require_openstack() validator
* Moved preprocess function from base to types
* Add nova scenarios to rally-neutron.yaml
* Add a cinder volume attachment scenario
* Sahara Job Execution benchmark
* Sahara Cluster and EDP contexts
* User_stories and feature requests appended to sphinx
* Fix "Resource" typo in debug output
* Fix some typos and mistakes in the doc string
* Updated from global requirements
* Fix CIDR overlapping in _generate_subnet_cidr
* Adds unit tests for rally.cmd.cliutils
* Updated from global requirements
* Minor modification of the `rally info find` command
* Allow user specified tempest config file
* Fix syntax errors in JSON doc samples
* Fix disassociation of nova-networks from tenants
* Sahara Job Binary internals cleanup
* Removed admin_user from benchmark.runners.base
* Fix one more time rally perf job
* Fix rally job in projects that don't have rally-scenario/plugins dir
* Updated from global requirements
* Moved context creation to benchmark.engine
* Rename doc environment to docs
* Make rally perf job work in other projects
* Fix rally verify fail (cause of rally KeyError: 'admin')
* Improve Rally gate job
* Rally info: deploy engines & server providers
* Support for benchmarking with existing users (part 1)
* Add cinder snapshot scenario tests
* Use https for all script sources
* Refactor searchutils module to base entity classes
* Fix rps._worker_process
* Sahara clusters networking parameters
* Designate Records scenarios and tests
* Updated from global requirements
* Improve usability of base.Scenario.validate()
* Impove benchmark validation and cleanup benchmarks decorators
* Designate - update example quotas in scenarios
* Fix wrong parameter used by raise of InvalidBenchmarkConfig
* Updated documentation of "DummyProvider to "ExistingServers"
* Add support for setting up Designate quotas
* Pass named arguments to users.create
* Trying to ingore races related to Nova + restoring missing benchmark
* Mention 'rally info' command in docs
* Add feature_request mechanism
* Add designate support with domain scenarios
* Fix test_rps.test__worker_process test
* Updated from global requirements
* Fix failing tests in gates
* Add new benchmark type VolumeType
* Refactoring rps runner unit tests
* Updated keystone authenticate user_story
* Updated from global requirements
* Add "rally info" command
* Adding http_requests scenario
* Disassociate Nova networks during cleanup
* Fix semantic validation of context images
* Decrease jobs time in gates
* Do not assume networks should be assigned
* Update the scenario_base to base in benchmarks
* Refactor benchmark/context/quota.py
* Workaround for different floating_ip_pool results
* Added Sahara Clusters scenario
* Remove magic() method from context manager
* Add required_contexts validator
* Fix partial results sending in rps runner
* Modify config semantic validation of benchmark engine
* Removed concurency from Quotas.nova_update_and_delete
* Fixes the check for admin role to be case-insenstive
* Fix rally-gate.sh exit status
* Fix copyright in doc/source/config.py
* Split user and admin cleanup to 2 separate classes
* Adds --json,--pprint flags to cmd
* All scenario runs time measurement
* Amended sphinx documents
* Add cinder-update-and-delete quotas scenario
* Extend sla output result
* Refactor Move atomic actions from benchmark scenarios
* Remove openstack-rally entry point
* Update rally neutronclient ca_cert option
* Mock open to prevent file creation
* Updated from global requirements
* Adds deleting scenarios for Neutron:
* Renames *.rst files in /rally/doc/samples/tasks
* Adding sla_check to gate jobs
* Add db model and api for worker
* Added Sahara Image context
* Added Sahara Node Group Templates scenarios
* Fix side menu depth
* Updated from global requirements
* Add resize-server for nova benchmark
* Add support for Keystone API v3
* Adds html result generator for tempest runs
* Modify wrong comment in existing.py
* Periodic runner refactoring
* Updated _boot_server() to handle floating ip/external networks
* Refactored test_images.py and images.py
* Updated rally gate scenarios
* Updated from global requirements
* Set any_order to True in some unit tests
* Fix cleaning up neutron resources
* Added cleanup for sahara objects
* Reduce number of points in generated charts
* Remove use of random uuid from unit tests
* fixes TypeError coercing of tempfile in subunit2json.py
* Update Tempest Config Generator
* Added Sahara client
* Update the way hard & soft reboots are displayed in task result
* Handle keyboard interrupt in task run
* Revert "Fix failing unit tests"
* Add unit tests for vm benchmark scenarios
* Add basic tools for integration tests
* Add new "volumes" context
* Add support for images context semantic validation
* Add rally task detailed info to rally-plot dir
* Updated test cases in benchmark types
* Updated mock return values to decorators
* Add unit tests for benchmark validation
* Do not use mktemp for creating rally.conf
* Sort by created at column in cli output
* Fix failing unit tests
* Add unit tests for benchmark scenarios
* Add unit tests for tempest context
* Add unit tests for processing utils
* Scenario output should have proper format in case of timeout
* Asynchronous result yielding for scenario runners
* Patched version of hacking library
* Add json results to the logs bundle, as well as html
* Check if required services are available before starting the scenario
* Add tempest unit test cases
* Updated from global requirements
* Change the default value of use_public_urls
* Add nova floating ip management in VM scenario
* Ensure temporary results_dir is created
* Add abstract layer for benchmark context semantic validation
* Check python>=2.7 before running tempest
* Updated from global requirements
* Change the default value of scenario_output
* Fix creation of ICMP security group rule when using Neutron
* Fix wrong attribute name used for http status
* Add sla checking
* A test to verify that scenarios in rally-scenarios are correct
* Use wget instead of curl in install script
* Updated from global requirements
* Restructure doc/samples/tasks directory
* Added a test to check json<->yaml configs equality
* Delete commented codes from test_vmtasks.py
* Adding Image Context Class
* Improve doc/source
* use oslo.db module
* small fixes on rest/base.py
* Add Neutron benchmarks scenario for creating and listing ports
* Benchmark to validate a keystone token N times at service endpoint
* Sync necessary modules from oslo incubator for worker service
* Updated from global requirements
* Add unit tests for rest
* Refactor tempest config generator
* Changes tempest subunit log handling
* Minor fixes in Ceilometer benchmark scenarios
* Updated from global requirements
* Fix hacking 0.9.x issues
* Update Rally gates to support VM scenarios
* Add Neutron benchmarks scenario for creating and listing routers
* Rename api directory to rest
* Make quotas context cleanup lazy
* Fixes wrong indent in rally/objects/task.py
* Fixes typo error in Rally - orchestrator/api.py
* Renaming in validation system
* Correct scenario specific results table headers
* Use import from six.moves to import the httplib module
* Removed usage of the py3kcompat module
* Fix wrong result data for constant_duration runner
* Added user_stories section
* Bugfix of bug causing security groups from allow_ssh to remain
* Fixes typo error in rally/README.rst
* fix installation failure if pip is not installed yet
* Add error handling to install_rally.sh
* Auto install tempest during "rally verify"
* Prepare documentation for readthedocs
* Use FakeCeilometerClient instead of mocks
* Add result table to atomic actions in html
* Add neutron scenario
* Add missing keys to rally runner result
* Heat rally scenario tests
* Add benchmark scenarios for ceilometer queries API
* Fix typos and pep257
* Log deployment id and benchmark key from task
* pass id or name of resources in scenario config
* Ensure file handle cleanup during image creation
* Add scenario name to Scenario Configuration for html
* corresponding result with name in output html
* sorted benchmark scenario selection
* Increase http client timeout from 30s to 180s
* Add benchmark scenarios for ceilometer statistics api
* Add FuelEngine
* Fix typos in file names for ceilometer resources API
* Use plugins from project in rally-gate
* Use function "print_list" to output tables in rally cmd
* Modify install_rally.sh to install under BSDs
* Fix inconsistencies in docs for factory classes
* Added a class to measure the duration of atomic operations
* The configs in osclients.py are not in rally.conf.sample
* Add output message for rally task command if uuid is not specified
* Add documentation for plugins
* Unit tests for TempestConf
* replaced e.message
* Add benchmark scenarios for ceilometer resources GET API
* Add benchmark scenarios for ceilometer meters GET API
* Refactoring for random names generation in scenarios
* Enable plots with no successful iterations
* Add probabliity based dummy exception scenario
* Add command-line integrated tests
* Fix typo, rename libxslt1 to libxslt
* Make cleanup methods lazy
* Invalid result for unexisting task in rally task results
* Add Neutron benchmarks scenario for creating and listing subnets
* Add benchmark for tempest. Part 2
* debug level logs should not be translated
* add possibility of using public_urls for benchmarking
* Skip non yaml/json files when validating samples
* Correction of tempest verification
* Rename number validator parameter none_ok
* Fix check for tempest tests name
* Add benchmark scenarios for ceilometer alarms api
* Replace non-documented ThreadPool with multiprocessing.Pool
* Fixed issue Unit Test requested Image from Internet
* Add dev libs for xml2 and xslt to install_rally.sh
* Add Neutron benchmarks scenarios
* Modify the CLI output for 'task detailed'
* Updated from global requirements
* Add mock.patch.stopall() to rally test cleanup
* Sync code from oslo incubator
* Load plugins for scenarios
* Refactored cleaner and added cleanup/utils
* Add argument to curl for following redirects
* Integrate the new oslotest library
* Minor fixes in the visualization of histograms
* Allow to provide a list of uuids for rally task delete
* Don't run rally in debug mode in ci
* Replace non-documented ThreadPool with multiprocessing.Pool
* Add Ironic client
* Adding Dummy benchmark scenarios
* Enlarge Scenario Selector Label
* Create utils module for processing task results
* Pass region_name to neutron client for consistency
* Make a bigger column for storing task results in mysql
* Runner Type as consts
* Add scenarios directory for ci
* Add new "role" context
* Quotas scenario for nova/cinder
* Use "https" instead of "git" for cloning code
* Add benchmark for tempest. Part 1
* Improve User context
* Name Change for "active_users" to "concurrency"
* Updated from global requirements
* Renamed provider 'name' property to 'type'
* Fix deploy engine "name" property in a docstring
* Fix DevStack contrib
* Clean rally ENV after recreate database
* Changed all deploy engine's 'name' property to 'type'
* Columns name change for ScenarioRunnerResult
* Fix ubuntu-12 issues in LxcHost
* Updated from global requirements
* Fix broken Glance cleanup context
* Allow to call task delete without specifying --uuid parameter
* Fix column index for sorting in rall verify list
* Add LxcEngine
* Fix cleanup in LxcProvider
* move cleaner context users/admin into setup()
* Create vm benchmark module
* Add number and file validators
* Remove createfromenv test environment dependency
* Improve tempest.conf generation
* Split ContinuousScenarioRunner to constant and constant duration
* Region support
* Replace inline schema urls with reference to single piece of data
* Add multihost engine
* Add neutron quotas management in benchmark context
* Added create_and_list_users and create_and_list_tenants benchmark
* Add --filename to rally task start
* Create class "Connection" for sqlalchemy.api
* Add required_parameters validator
* Adds commands for verification results
* Adds saver for results from `rally verify`
* Updated from global requirements
* Skip large-ops benchmarks
* Add histogram to the report of tasks' results
* move install script to project root dir
* Remove file AUTHORS
* Re-Add default db connection string
* Improve WaitForTestCase
* Remove _change_port
* Add task uuid to iteration start/end logs
* Add Scenario meta() to extract meta information from method
* There is no mock_magic.assert_called_once method
* Create copy of scenario context before modifying
* Add benchmark-context manager
* Improve @base.scenario to specify it is admin only
* non-zero exit status on error for CLI
* Add support for the neutron client in the osclients object
* remove openstack-patches
* Sync common db code from Oslo
* Use pool of threads in UserGenerator for deleting tenants and users
* Add tenants create, with users per tenant benchmark for keystone
* Use float values in cfg.FloatOpt() options
* Add nova/cinder quotas management in benchmark context
* add +x to pre_test_hook.sh
* Add @cleanup usage in each scenario method
* Add tests_ci/pre_test_hook.sh
* Add support to upload image from a local directory
* Fix default dummy engine name in devstack plugin
* Fix dependencies for rpm based systems
* Add execrise script to devstack
* Simple fix for bug with periodic runner
* add create_and_list_image benchmark
* Pass context to secnarios
* Moved fuelclient.py
* Fix samples in README.rst
* Improve generic cleanup performance
* Add a ScenarioRunner subclass which executes scenarios serially
* Use pool of threads in UserGenerator for creating tenants and users
* Sync modules from oslo, excluding db related
* decorator for benchmark scenarios method
* Fix Nova crap
* Add cliutils.args for task_id in use task command
* Rename deploy_id argument to uuid in deployment commands
* Rename task_id argument to uuid in task commands
* Updated from global requirements
* Add default task_id management by using @default_from_global
* Use testr directly in tempest verifier
* rename DummyProvider to ExistingServers
* rename DummyEngine to ExistingCloud
* Refactor text in use deployment command
* Fix envutils.get_global (prev. default_deployment_id) errors management
* Add fix __ctx_name__ in keypair context
* add create_and_list_volume benchmark
* Add shell script for integrated tests
* show detailed exception when doing cleanup
* remove a mixed style (**{} and param=value)
* Add boot_and_list_server benchmark
* Use ConfigParser to generate tempest config
* standardized output for scenario runners
* Updated from global requirements
* Refactor Tempest installation directories structure
* unify log_task_wrapper and log_deploy_warpper
* Improve OpenstackProvider
* Fix CONFIG_SCHEMA for runners
* Move serverprovider under deploy
* Base part of a server REST API
* Added tenant create benchmark for keystone
* remove unused code in serverprovider and tests
* improve rally output (exception part)
* Remove underscore (_use) import in deployment command
* Remove base.Scenario.register()
* Refactor the decorator which sets a default deploy_id
* New task config and verification refactoring
* Add print openrc information for use deployment
* Add unit test for deployment list subcommand
* print all debug msg from clients when using debug flag
* remove some unused codes
* Validator for image checking
* Use itertools instead of custom count
* Use six.add_metaclass instead of __metaclass__
* Add unit test for Authenticate scenario
* Make tempest integration works without "sudo"
* Show atomic operations results without aggregation with CLI
* Add keypair context
* Fix for Authentication scenario to correctly use self.clients
* Capture the pkey errors
* Switch to oslosphinx
* Don't raise exception if deploy_id not found
* Fix typo in deployment check
* Set 'period', 'duration' config parameters in seconds
* Add the install_rally.sh file
* Expose clients directly from osclients.Clients
* Updated from global requirements
* Make usage serverprovider in common way
* Updated from global requirements
* catch exception when deployment not found
* Remove unnecessary flush in task detailed
* exclude INFO log about "Starting new HTTP connection..."
* move wait_for & wait_for_delete into benchmark.utils
* Added "active" column on deployment list
* remove keypair from secgroup context
* Add git branches support in DevstackEngine
* Fix cleanup in DevstackEngine
* Add count to atomic actions display
* Make lxc ssh port fowarding optional
* don't delete network during benchmark
* Unify Context classes and introduce context object
* added decorator-based mocks declaration to
* added decorator-based mocks declaration to
* ignore build dir in tox.ini
* use CONF in glance benchmark
* Updated from global requirements
* Add unit tests for the show command
* Add 'flavor' to fakes.py
* Changed import statement
* Add support to tag a task
* Removes old obsolete task.plot function and all related code
* Fix user creation in devstack engine
* Fix and unify structure of modules in benchmark engine
* Fix ScenarioRunnerTestCase:test_run_scenario_timeout
* Prepare for real benchmark context support
* Proper fix for test_run_scenario_exception_inside_test
* Remove cinder volume type deletion
* Fix CIDR key error bug
* Move scenario args validation to BenchmarkEngine
* Add use task subcommand
* Fix test_run_scenario_exception_inside_test
* Add deploy-id to command 'use deployment'
* Updated from global requirements
* Refactor use deployment command
* Improve CLI: rally task list (of empty sequence)
* Improve CLI: rally deployment list (of empty sequence)
* Add tests for utils._prepare_for_instance_ssh
* Add yaml file support
* Add unit test for use deployment with a deployment_id not found
* Remove unused methods from rally.benchmark.uitls
* Finish unit tests for ScenarioRunner
* Improve benchmark.runner.UserGenerator
* Updates _cleanup_users_resources to use imap instead of map
* Add v3 uri to tempest configuration
* Fix typo in use deployment subcommand
* Add ceilometer python client to rally.osclients
* Improve README.rst
* Add a new rally command: show
* Add task result graphic ploter command
* Updated from global requirements
* Rally & Tempest integration (base) - part 2
* Updated from global requirements
* UPDATED README to make it detailed
* UPDATED README to make it detailed
* Do not use openstack clients as a thread safe objects
* Use Endpoint objects instead of endpoint dictionaries
* Show 90 and 95 percentile results
* Print exceptions from scenarios only in case of debug
* Add permission to validators
* Fix relative path in Tempest verification UTs
* Import missing gettextutils
* Rename _run_scenario_loop --> _run_scenario_once
* Rally & Tempest integration (base) - part 1
* Rename TestEngine --> BenchmarkEngine
* Add support for user/admin permissions in endpoints
* Move every rally command to its own module
* Fix openstack server provider resource cleanup
* Added SSL support for OpenStack services clients
* Validate flavor_id in glance benchmark
* Added Heat client initialization
* Fix mocking in scenario runner tests
* Update OS serverprovider schema to allow specifying image by uuid only
* Replace the call to default_deployment_id() by a decorator
* Add ssh port forwarding to lxc provider
* Show whether task failed when printing detailed information
* Implement ScenarioRunner execution types via inheritance
* Fix rally benchmark's timeout mechanism
* Fix py26 timeouts
* Do not cache not connected ssh client
* Use new deployemnts by default, add --no-use option
* Add atomic actions timer for glance scenarios
* Don't initalise deployment engine when starting a task
* Don't require glance to use OpenStack server provider
* Add authenticate benchmark for Keystone
* Fix misspellings in rally
* Add benchmark tests for glance
* Remove magic numbers from benchmark utils
* Mark task as failed if benchmark validation fails
* Use assertIsInstance to test the results of get_engine and get_provider
* Start refactoring scenario runner
* Use task.set_failed when task is failed
* Add display of atomic actions timer in CLI
* Add rally use command - part 2
* Sshutils refactoring
* Fix benchmark.utils.get_from_manager()
* Add timer for atomic actions
* Fix idle time in timeout case
* Validator for flavor_id
* Temporary fix testtols version to unblock gate
* Validator for image_id
* Move cleanup functions to benchmark.utils
* Call rally-manage with warning when openstack-rally-manage is used
* Add deployment check subcommand
* Validators adder for benchmarks preconditions
* Add FuelClient
* Add option to use deployment on creation
* Create rally deployment from OS_* environment variables
* Remove debug print from fileutils
* Add name into dummy engine schema and make it required
* Add rally use command
* LxcProvider network refactoring
* Fix tests/benchmark/scenarios/nova/test_utils.py
* Use cls instances instead of object in benchmarks
* Add keystone host unreachable checks
* Print task information when creating task from CLI
* Benchmark scenarios for Rally: run task in instance
* Improve devstack install script
* Add management of unauthorized keystone exceptions
* Added missing test for rally.cmd.main.task.results
* Fix CLI output on task detailed command
* Introduce the Endpoint object
* Print deployment id when a deployment is created
* Refactor unit tests for dummy engine
* Remove results before a task
* Extra scripts with DevStack support
* Omit openstack/common/ in test coverage reports
* Add support to build docs by Sphinx
* Add a separate class for a mutable JSONEncodedDict
* Improve tests of DevstackEngine.deploy
* Clarify usage of an endpoint
* Deprecate openstack- console scripts, introduce rally, rally-manage
* Rename Server's 'ip' to 'host'
* Add boot_from_volume benchmark
* Fix icmp security group rule creation
* Fix devstack engine test coverage
* Check endpoints with keystone client
* Add missing tests for rally/benchmark/scenarios/nova/utils.py
* DummyProvider refactoring
* Create FK for parent_uuid on ALTER TABLE
* Switch rally.sshutils from 'eventlet' to 'time'
* Move rally.test to tests.test
* Make cli result table columns more clean
* Adding cinder benchmark scenarios for Rally
* Remove nullable=True because it is set by default in db
* Fix comments and typos in db/api, orchestrator/api and cmd/main
* Sync with global requirements
* Fix H302 pep8
* Add base utils for keystone benchmarks
* Move nova benchmark samples to nova directory
* Log warnings instead of infos about cleanup fails
* Make resource status checking letter case independent
* Add generic keystone cleanup mechanism
* Show results of last task without requring user to lookup UUID
* Refactor the way benchmark engine returns its results
* Fix invalid port range in security group rule create
* Prepare instances for SSH access
* Preserve consistency in module namings
* Fix incorrect exit status checking in sshutils
* Remove duplicate mocks from ScenarioTestCase
* Move ScenarioRunner class to a separate module
* Remove mox
* Use hacking import_exceptions for gettextutils._
* Add missing tests for rally/cmd/manage.py
* Fix usage of get() methods in python clients
* Add `port` attribute to Server object
* Fix tcp port in sshutils
* Fix ssh port in devstack engine
* Add `parent` field to deploy object
* Rename "destroy_vms" to "destroy_servers"
* Remove unused sh requirments
* Add .venv to git ignore
* Get rid of fuel-ostf-tests
* Rename "create_vms" to "create_servers"
* Update oslo code
* Fix too slow .VirshProviderTestCase.test_create_vm
* Add start and finish time of the deployment
* Add ability to get stdout/stderr to sshutils
* Show task details without results
* Removed `tests' section from config samples
* Implement periodic execution of benchmark scenarios
* Move Fake objects to a special tests.fakes module
* Add CONFIG_SCHEMA to virsh provider
* Add CONFIG_SCHEMA to dummy provider
* fix the wrong file name for devstack-in-openstack sample json file
* Benchmark scenarios for Rally: Nova rescue server
* Add cover to gitignore
* Add CONFIG_SCHEMA to DummyEngine
* Add caching to osclients.Clients
* Cosmetic change to cleanup output
* Separate deployments from tasks
* Fix bug in serverprovider.test_openstack_provider
* Store/restore Server from a dict with credentials
* Fix bug in tests.serverprovider.test_virsh
* LxcProvider track resources
* Store/restore Server from a dict with credentials
* Add CONFIG_SCHEMA to openstack provider
* Add common config validation to ProviderFactory
* Add CONFIG_SCHEMA to devstack engine
* Add common config validation to EngineFactory
* Fix bug in tests.benchmark.test_utils
* Generic benchmark cleanup
* Refactoring for generic benchmark cleanup
* OpenStackProvider track resources
* Remove argument vm_uuids from destroy_vms
* VirshProvider track resources
* Quick fix for boot_and_bounce_server
* Cover VirshProvider by unit tests
* Introduce a manager of resources
* Remove a task from providers
* Resource model with DB API
* Rename ServerDTO to Server
* Run benchmark scenarios for a given period of time
* Renamed sample with OpenStack provider
* Separate samples
* Add CLI command for deployments
* Update completed tasks status to "finished"
* Data processing: aggregation plots
* Tracking deployment status into context on error
* Fix bug in tests.benchmark.test_utils
* Add openstack based server provider
* Separate command to perform deploy and cleanup
* Enable access to admin openstack clients from benchmarks
* Add uuid to temporary tenant names
* Fix bug in ScenarioRunner
* Fix passing the context to multiprocessing pool
* Switch sshutils.SSH from subprocess to paramiko
* Introduce the deployment
* Change benchmark config format
* Refactor nova scenarios benchmark tests
* Proper update of objects
* Make Deploy and Task objects similar
* Add tests for rally.objects.task.Task
* Move task and deploy modules in rally.objects
* Fail task when engine not found
* Remove old sshutils functions
* Fix debian/ubuntu related issues
* Cover rally.orchestrator.api by unit tests
* Fix typo in name of the orchestration test file
* Abstract out DB operations on deploys in db.deploy
* Added Deployment model
* Canonical form of the test_db_api.py filename
* Fix incorrect TaskCommandsTestCase.test_start test
* Add btrfs support to LxcContainer
* LxcProvider refactoring
* Switch LxcProvider from sshutils
* Improve sample configs
* Delete task by command
* Add sleep_between() method to benchmark scenarios
* Fedora/CentOS specific fixes in devstack engine
* Fixes rally boot_server benchmark bug
* Benchmark scenario to boot server and bounce it N times
* Delete objects completely
* Optimize the scenario runner
* Fix intermitten py test failure
* Use a dash in --task-id argument
* Use ServerDTO.ssh in devstack engine
* Add ssh object to ServerDTO
* Add SSH class to rally.sshutils
* Cleanup logging
* Rename NoDBTestCase to TestCase
* Enable different tenants in the scenario runner
* Fix local test case failure for test_lxc_install()
* Fully initialize the keystone client in osclients
* Use random network in boot_server scenario
* Performance changes for UTs
* Add lxc provider
* Catch proper exception on MissignArgs error
* Fixed typos into constants
* Improve task results cli cmd
* Adds boot server benchmark test
* Add task to provider object
* Improve rally.utils.log_task_wrapper()
* Add utils.log_task_wrapper() method
* Work out the sample task config
* Simplify the test config format
* Move fixtures from test-requirments to requirments
* Added python clients patches
* Get rid of rally.benchmark.tests module
* Remove legacy code from test engine
* Fix bug in DevstackEngineTestCase.test_deploy
* Fix NovaServers.snapshot benchmark
* Fixed some formatting and typos in the README
* Update short description language
* Check for VM ERROR state rather than FAILED
* Make DevstackEngine work out of the box
* update rally conf sample
* Update oslo code
* Fixed typo in DevstackEngine docstring
* Rename FakeEngine DummyEngine
* Fix wrong usage of Pool
* Add new method "raw_results" to Task CLI
* Cleanup db.TaskResource model
* Remove vim header
* Benchmark scenarios for Rally: Nova servers
* Benchmark scenarios for Rally: Nova utils
* Improve utils.wait_for()
* Add self to AUTHORS file
* Rewrite rally.utils.sync_execute() to wait_for()
* Make devstack engine more flexible
* Store test scenario results in DB
* Fixed default value for ssh public key
* Fixing Rally config json schema
* Make ProviderFactory.get_provider more simple
* Adding two more options to verification_tests options
* Fixing incorrect mocks usage in unit tests
* Renaming a test case
* Add DummyProvider
* Various fixes in benchmark.engine
* Fix rally.benchmark.scenarios autoload
* Make rally.utils.import_modules_from_package recursive
* Fix verification output
* Reducing the running time for test_sync_execute()
* Rename benchmark.engine.Tester to benchmark.engine.Verifier
* Add new Benchmark engine
* Move rally.task to rally.db.task
* Remove old test engine stuff
* Fix orhestrator.api test
* Fixing decorators for rally.benchmark.base.Scenario
* Add benchmark scenarios base class
* Add Timer to rally.utils
* Slight improvement for utils.sync_execute
* Removing a needless verification test
* Renaming benchmark folders
* Rally logging: deploy engine
* Function synchronization utility
* Fix logging
* Rally logging: test engine
* Patched virsh serverprovider, devstack deploy engine to use ServerDTO
* Added Devstack deploy engine
* Add Server Provider DTO objects
* Move and refactor ImmutableMixin & EnumMixin to rally.utils
* Fix verification results are not stored in db
* Added virsh vm provider engine
* Fixed import utils code
* Fix Tester cleanup crashes rally execution
* Move deploy() out of __enter__ method
* Bugfix for TestEngine
* Bump oslo config
* Implement task cli list method
* Rally integration with fuel-ostf-tests
* Removed except blocks from cli code
* Fix bug in CLI task.start method
* Cancel remaining tests if one test was timed out
* Add task status tracking in TestEngine
* Add Task object for tracking task status
* Added cli scripts to package config
* Bugfix for ConfigManagers: defauls overwriting
* Move task creation from get_engine to orcehstrator
* Fixed incorrect timeout tests
* Change create_image signature
* Rename vmprovider to serverprovider module
* Add task status tracking
* Adding new test execution layer
* Benchmark test framework for Rally
* Add utils to work with OpenStack python clients
* VMProviderFactory.*_image() shouldn't be abstract
* Improve readme
* Added openstack patches for nova
* Config manager
* Partial implement fake deploy engine
* Rename rally.cmd.task_cli module to rally.cmd.main
* Add CLI utils for DB recreate
* db.db_create() shouldn't call hiddenly db.db_drop()
* Fix VM provider get_available_providers() test
* Slight refactoring of the testcase for Tester.run()
* Exception raising in TestEngine.verify()
* Test engine redundant code removal
* Import modules from package
* Add .gitreview file
* Test engine & Test cases
* Add CLI interface for Rally
* Add cliutils
* Add version module
* Add orchestrator
* Test engine
* Force tox version to 1.6
* Repaire benchmark test
* Add author and fix .gitignore
* A fix for tox.ini
* Authors info
* Adding cleanup for all test runner calls
* Add VM Provider factory
* Add deploy engine factory
* Add itersubclasses() method that allows to iter through all subclass
* Init benchmark framework
* README for Rally
* Move tests to root dirrectory and fix tox
* Add Task Model
* Add base for DB API
* Add DB utils from Oslo
* Add run_tests.sh as alternative for tox
* Add venv tools
* Add oslo fixtures
* Fix test-requirements.txt
* Add oslo test module
* Update gettextuils and jsonutils form oslo
* Add rally.sample.conf to project
* Add AUTHORS file
* Add tox, pep and requirements to project
* Rename README.md to README.rst
* Integrate with oslo
* Initial commit
