Yoga Series Release Notes

17.0.1-5

Bug Fixes

  • Fix a bug where Swift client in the trove guest instance failed to verify certification against the self-signed certificate authority. Story 2010674

17.0.1

Bug Fixes

  • check if the user input is legal, currently, Trove may have a RCE vulnerability. For more details see: Stroy 2010004

  • Fix guest-agent failed to start PostgreSQL container due to execution of the “CREATE DATABASE” statement within the context manager of psycopg library. See the following for details Stroy 2010761

16.0.0

New Features

  • Added iptables-persistent package in the guest image element for diskimage-builder. The Trove service administrator is able to perform iptables command (leveraging cloud-init config) and persist the change.

  • The project user can query the project’s own resource quota.

  • Show network ID for the instance private address.

Upgrade Notes

  • The default value of the Trove guest agent config option [postgresql] backup_docker_image is changed to openstacktrove/db-backup-postgresql:1.1.1. There is nothing to do if the option is not configured explicitly.

Security Issues

  • Changed the network mode of the database container to “bridge” and exposed the service ports. Cloud operator could adjust the iptables to restrict network access from the database container to the outside. An example:

    iptables -t filter -I DOCKER-USER 1 -d [restricted-network-range] -i docker0 ! -o docker0 -j REJECT

Bug Fixes

  • Fixed backup creation failed issue when using custom container image registry.

  • Added a periodic task for the PostgreSQL datastore to clean up the archived WAL files. Added a check when creating incremental backups for PostgreSQL. A new container image openstacktrove/db-backup-postgresql:1.1.2 is uploaded to docker hub.

  • Fixed an issue that the replication configuration is lost after resizing the instance.

15.0.0.0rc1

New Features

  • Added the ability to create the Cinder volume in the same AZ as the Nova instance. Set enable_volume_az to True (defaults to False)

  • Added the ability to quota on the total amount of RAM in MB used per project. Set quota.max_ram_per_tenant to enable. Default is -1 (unlimited) to be backwards compatible. Existing installations will need to manually backfill quote usage for this to work as expected.

  • Support image tags for the datastore version. When using image tags, Trove is able to get the image dynamically from Glance for creating instances. If both are specified, image ID takes precedence over image tags.

  • Added custom container registry configuration for Trove guest agent, it’s now possible to use images in the private registry rather than Docker hub.

  • Added MySQL 8 support, docker image openstacktrove/db-backup-mysql8.0 is created to support backup and restore.

  • In a multi-region deployment with geo-replicated Swift, the user can restore a backup in one region by manually specifying the original backup data location created in another region.

Upgrade Notes

  • The default value of [oslo_policy] policy_file config option has been changed from policy.json to policy.yaml. Operators who are utilizing customized or previously generated static policy JSON files (which are not needed by default), should generate new policy files or convert them into YAML format. Use the oslopolicy-convert-json-to-yaml tool to convert a JSON to YAML formatted policy file in a backward compatible way.

  • Any 3rd party clients that are talking with Trove API to wait for status should be notified to use operating_status instead.

Deprecation Notes

  • Use of JSON policy files was deprecated by the oslo.policy library during the Victoria development cycle. As a result, this deprecation is being noted in the Wallaby cycle with an anticipated future removal of support by oslo.policy. As such operators will need to convert to YAML policy files. Please see the upgrade notes for details on the migration of any custom policy files.

Bug Fixes

  • Fixed an issue that orphan volumes left after removing instances, especially for the case that instance creation failed because of timeout when waiting for the volume available.

  • Fixed a race condition that instance becomes ERROR when Trove is handling creating and deleting at the same time.

14.0.0.0rc1

New Features

  • Added a new field named addresses in the instance API response which including the IP address and type, either ‘private’ or ‘public’.

  • Added a config option enable_access_check (default True) to decide if Trove should check the subnet of the user port is associated with a Neutron router. This check is needed for creating public-facing instances and the instance initialization. This check could be skipped When using Neutron provider network.

  • Database service (MySQL and MariaDB) is now running as a Docker container inside the trove instance. The image is defined by docker_image config option for each datastore.

  • The database backup and restore operations are performed by the Docker container inside the Trove instance.

  • Only one trove guest image is needed for all the datastores.

  • The admin user is able to get backups of a specific project.

  • Support db instance rebuild. The rebuild operation is mainly for Trove upgrades, especially when the interface between the Trove controller and guest agent changes. After the Trove controller is upgraded, the cloud administrator needs to send a rebuild request with the new guest image ID. Communication with the end users is needed as the database service is offline during the process. User’s data in the database is not affected.

  • When resizing volume for an instance which is the primary of a replication cluster, Trove also resizes the volume for all the replicas automatically.

  • Added replicas attribute for listing instances when the instance is the master of the replication cluster.

  • Added support to show and update the access configuration for the instance.

  • Trove now supports to resize volume without downtime. To use this feature, the version of Nova and Cinder needs to be at least Pike, the config option cinder_service_type needs to be set to volumev3. The cloud admin can disable this feature by setting online_volume_resize=False, default is enabled.

  • The Trove admin user is able to remove the default configuration parameters for datastore version, e.g. trove-manage db_remove_datastore_config_parameters mysql 5.7.29

Upgrade Notes

  • Existing database services are not affected. However, in order for Trove to communicate with Trove guest agent, a new guest image needs to be built and existing trove instances need to be backed up and restored.

Deprecation Notes

  • The ‘ip’ field of getting instance response is deprecated and will be removed in W release.

  • Most of the options related to backup and restore are removed, e.g. backup_namespace, restore_namespace, backup_incremental_strategy

Bug Fixes

  • When the trove-guestagent failed to update the datastore service status, the instance status should be ERROR.

13.0.0

New Features

  • A new field named service_status_updated is added to the instance API response which e.g. could be used to validate if the instance ‘HEALTHY’ status is stale or not

  • A new database service status HEALTHY is introduced to indicate that the service is responsive. HEALTHY is the final status after ACTIVE.

  • Admin user can delete datastore if there are no instances or backups associated.

  • Add XFS disk format for database data volume, cloud admin can config ‘ext3’, ‘ext4’ or ‘xfs’ in volume_fstype option.

Upgrade Notes

  • Python 2.7 support has been dropped. The last release of Trove to support Python 2.7 was OpenStack Train. The minimum version of Python now supported by Trove is Python 3.6.

  • Trove is now using admin clients by default to communicate with Nova, Cinder, Neutron and Glance. Deployers want to stick to the old clients need to explicitly config the following options:

    • remote_nova_client

    • remote_cinder_client

    • remote_neutron_client

    • remote_glance_client

  • Any existing scripts that rely on the database instance ACTIVE status should now rely on HEALTHY status.

Deprecation Notes

  • The following config options are deprecated in favour of a separate configuration section service_credentials introduced to define the Trove service user credentials for communication with other OpenStack services.

    [DEFAULT]
trove_auth_url
os_region_name
nova_proxy_admin_user
nova_proxy_admin_pass
nova_proxy_admin_tenant_id
nova_proxy_admin_tenant_name
nova_proxy_admin_user_domain_name
nova_proxy_admin_project_domain_name

Bug Fixes

  • Fixed the issue that the datastore version cannot be deleted because of dependency of deleted instances. When an instance or backup is deleted, the datastore version attribute is set to NULL in the database. When the datastore configuration parameter is deleted, the record is deleted from the database rather than only set the ‘deleted’ field to 1.

12.0.0.0rc1

New Features

  • Added support for designate v2 API with a new DNS driver. To use this driver set dns_driver = trove.dns.designate.driver.DesignateDriverV2

  • Adds new fields “instance_ids”, which is supposed to contain ids of cluster instances, in payloads of two cluster events - DBaaSClusterShrink and DBaaSClusterGrow. Moreover, additional end notifications after growing and shrinking cluster have been added. It allows better integration with tools for monitoring resources usage.

  • Support for upgrading Redis instance.

  • Support for upgrading Redis cluster.

  • Added a new config option nova_keypair to specify an existing Nova keypair name for the database instance creation, the cloud administrator is responsible for the keypair management and configuration. It’s recommended to create Trove database instance in the admin project for security reasons, so only the cloud administrator who has the private key can access the database instance. With the keypair support, ssh keys are no longer injected into Trove guest agent image at build time.

  • Support to filter backups by instance_id, additionally, the admin user can get backups of all the projects by specifying all_projects in the query string parameters.

  • The cloud admin is able to apply a security group to the management port(with the purpose of communicating with the control plane and other management tasks) of the Trove instance, by setting the management_security_groups config option. The cloud admin is responsible for managing the security group rules. The security group and its rules need to be created before deploying Trove.

  • Users can create public trove instance that has a floating IP attached but have the ability to define what CIDRs could access the user’s database service. Refer to the API doc for more details.

Upgrade Notes

  • Cloud administrator needs to create a Nova keypair and specify the keypair name for config option nova_keypair, the private key is used to ssh into new database instances created. The previous private key is also needed to ssh into the existing database instances.

  • The management security group won’t affect the Trove instances created before the upgrade.

Deprecation Notes

  • The config option default_neutron_networks is deprecated and will be removed in the future release, use management_networks instead.

11.0.0

Prelude

Added new tool trove-status upgrade check.

New Features

  • Added /instances/detail endpoint to fetch list of instances with details.

  • User can specify the number and volume of mongos/configserver with extended_properties argument when creating mongodb cluster. Currently, the supported parameters are, num_configsvr, num_mongos, configsvr_volume_size, configsvr_volume_type, mongos_volume_size and mongos_volume_type.

  • New framework for trove-status upgrade check command is added. This framework allows adding various checks which can be run before a Trove upgrade to ensure if the upgrade can be performed safely.

Upgrade Notes

  • Operator can now use new CLI tool trove-status upgrade check to check if Trove deployment can be safely upgraded from N-1 to N release.

10.0.0

Bug Fixes

  • MariaDB allows a server to be a master and a slave simultaneously, so when migrating masters, if the old master is reactivated before attaching the other replicas to the new master, new unexpected GTIDs may be created on the old master and synced to some of the other replicas by chance, as the other replicas are still connecting to the old one by the time. After that these diverged slaves will fail to change to the new master. This will be fixed by first attaching the other replicas to the new master, and then dealing with the old master. Fixes #1754539

  • Remove support for creating volume from Nova. The former configuration “use_nova_server_volume” is not used anymore, for creating volumes, cinderclient will be always used. Fixes bug #1673408.

  • Previously root disable API returns a HTTP 200 response without any content, a HTTP 204 response which is more appropriate will be returned now.

9.0.0

New Features

  • OpenStack Trove now supports enable or disable authentication for Redis datastore via the root-enable and root-disable API’s.

Bug Fixes

  • Fixes an issue in galera_common shrink that the wrong load removed nodes which could be missing a ClusterShrinkMustNotLeaveClusterEmpty exception or meet a NotFound error. Bug 1699953

8.0.0

New Features

  • Added the ability to associate datastore versions with volume types. This enables operators to limit the volume types available when launching datastores. The associations are set via the trove-manage tool commands datastore_version_volume_type_add, datastore_version_volume_type_delete, and datastore_version_volume_type_list. If a user attempts to create an instance with a volume type that is not on the approved list for the specified datastore version they will receive an error.

  • Support for the new ‘reapply’ command. This allows a given module to be reapplied to all instances that it had previously been applied to. Bug 1554903

Security Issues

  • Configuration show masks any password values.

Bug Fixes

  • It fixes an issue with Redis configuration, it uses a wrong min value for repl-backlog-size in validation rules. Bug 1697596

  • The module-instances command now returns a paginated list of instances. A –count_only flag was added to the command to return a summary of the applied instances based on the MD5 of the module (this is most useful for live_update modules, to see which ones haven’t been updated). Bug 1554900

7.0.0.0rc1

New Features

  • Support attaching and detaching of configuration groups on clusters.

  • Add support for cluster restart.

7.0.0.0b3

New Features

  • Modules can now be applied in a consistent order, based on the new ‘priority_apply’ and ‘apply_order’ attributes when creating them. Blueprint module-management-ordering

Upgrade Notes

  • For module ordering to work, db_upgrade must be run on the Trove database.

Bug Fixes

  • Module list/show now returns boolean values as True/False instead of 1/0. Bug 1656398

  • Pass instance nic and az to cluster grow. Add specific Fix for MongoDB to use the instance nic and az.

7.0.0.0b2

New Features

  • Adds a region property to the instance model and table. This is the first step in multi-region support.

  • Add RBAC (role-based access control) enforcement on all trove APIs. Allows to define a role-based access rule for every Trove API call (rule definitions are available in /etc/trove/policy.json).

Bug Fixes

  • The payload for cluster GET now returns IPs for all networks, not just the first one found for each instance. Bug 1642695

  • Fix race condition in cluster-show that returned erroneous not found error. Bug 1643002

  • Case where a new instance_modules record is written for each apply has been fixed. This issue would have potentially made it impossible to delete a module. Bug 1640010

  • Improved mountpoint detection by running it as root. This prevents guests that have undiscoverable mount points from failing to unmount.

  • After upgrading the guest agent was in an inconsistent state. This became apparent after restarting or resizing the instance after upgrading.

Other Notes

  • Add Compute ID (server_id) and Volume ID (volume_id) to Trove show output for admin users. Bug

7.0.0.0b1

New Features

  • Add ‘max-prepared-stmt-cnt’ to the list of valid values which can be used in configuration groups for MySQL, Percona, MariaDB and PXC

Bug Fixes

  • Since SafeConfigParser is deprecated in Python version 3.2, Trove uses ConfigParser for versions of Python >= 3.2 and SafeConfigParser for earlier versions. (Bug 1618666)

  • Fix Postgresql promote (bug 1633515).

Other Notes

  • Add ephemeral column in flavour-list (Bug 1617980)

6.0.0.0rc1

New Features

  • Add support for configuration group management for DB2 Express-C.

  • Add support for full online backup and restore for DB2 Express-C by enabling archive logging.

  • The reset-status command will set the task and status of an instance to ERROR after which it can be deleted.

  • The force-delete command will allow the deletion of an instance even if the instance is stuck in BUILD state.

  • The –incremental flag for backup-create will add the ability to create incremental backup based on last full or incremental backup. If no full or incremental backup exists a new full backup will be created.

  • New instance upgrade API supports upgrading an instance of a datastore to a new datastore version. Includes implementation for MySQL family of databases.

  • A locality flag was added to the trove REST API to allow a user to specify whether the instances of a cluster should be on the same hypervisor (affinity) or on different hypervisors (anti-affinity).

  • Support was added for applying modules to cluster instances through cluster-create and cluster-grow. Bug 1578917

  • Support for standard WAL based streaming replication for PostgreSQL guests. Sets up read-only hot standby servers.

  • New quota management APIs for reviewing and changing the quota for a particular tenant. Requires admin privileges.

Deprecation Notes

  • Make ‘long query time’ manageable via configuration groups (see bug 1542485). Deprecate the global ‘guest_log_long_query_time’ option in preference of datastore-specific configurations. MySQL long_query_time Percona long_query_time Percona XtraDB Cluster long_query_time MariaDB long_query_time PostgreSQL log_min_duration_statement

Bug Fixes

  • Allow specification of volume-type on cluster create. (Bug 1623005)

  • Applying a module again will now reflect the update name, type, datastore and datastore_version values. Bug 1611525

  • Updating a module with all_datastores and all_datastore_versions now works correctly. Bug 1612430

  • Close the race condition window in user-list call. Closes-Bug 1617464

Other Notes

  • In the Mitaka release, support was added for full offline backup and restore using the default circular logging. In this release, the name of the strategy for offline backup and restore was changed from DB2Backup to DB2OfflineBackup. Hence, to enable offline backups, we should set backup_strategy=DB2OfflineBackup and for online backups, backup_strategy=DB2OnlineBackup. The property backup_namespace and restore_namespace will be the same for both types of backup and restore.

  • Add disk column in flavour-list Bug 1617987.

6.0.0.0b3

New Features

  • Enable database log retrieval on Cassandra instances.

  • Add icmp option for DB security group. if icmp=True, users will be allowed to ping to DB instances.

  • Added a module driver for New Relics licenses. This allows activation of any New Relic software that is installed on the image. Bug 1571711

  • Errors that occur in Trove are now persisted in the database and are returned in the standard ‘show’ command.

Bug Fixes

  • Use SET PASSWORD and RENAME USER queries to update user properties.

  • Fixed parsing of GTID references containing a list of GTIDs from xtrabackup_binlog_info file on MySQL replicas.

  • Increased agent_call_high_timeout config setting to 10 minutes. This configures the length of time that the taskmanager will wait for an asynchronous guest agent call to complete.

  • Fix IniCodec to deserialize Python objects. This also brings it in line with other codecs. guestagent_utils.to_bytes return the byte values as ints. See bug 1599656

Other Notes

  • Add vCPUs column in flavour-list Bug 1261876.

6.0.0.0b2

New Features

  • A locality flag was added to the Trove REST API to allow a user to specify whether new replicas should be on the same hypervisor (affinity) or on different hypervisors (anti-affinity).

  • Full and incremental backup and restore strategy for PostgreSQL based on pg_basebackup and WAL shipping.

Bug Fixes

  • This would apply to any type of cluster that uses the galera strategy while setting the nics on a create call. When we called cast to set() the object was a list of lists. The set method can not has a list so this was causesing a unhashable error. The change is to make the instance_nics a list of strings (what we originally expected) to resolve this issue. Bug 1570602.

  • Fixed issue where module-apply after module-remove caused module-query to skip reporting on that module. Bug 1571799

  • Fixes an issue with a failure to establish a new replica for MySQL in some cases where a replica already exists and some data has been inserted into the master. Bug 1563574

  • Fixed default configuration template for MySQL to ensure that replication uses binlog_format. Bug 1563541.

  • An invalid module driver is now logged correctly. Bug 1579900

  • Fixed wrong call in conductor when reporting a guest notification exception. Bug 1577848

  • MariaDB historically leveraged the MySQL manager for guest support including the configuration groups implementation. With MariaDB now having its own manager class that inherits from MySQL, it needs to have validation_rules and a ConfigParser setup. Bug 1532256

  • Make ‘default_password_length’ per-datastore-property. Bug 1572230

  • Do not remove MySQL root user on root-disable so that the proper status can be reported on restore. Bug 1549600

  • Implement PostgreSQL guestagent models for databases and users.

  • Implement RootController extension for the PostgreSQL datastore.

  • Remove unused ‘override.config.template’ files. Bug 1575852

  • Make guestagent reuse Cassandra connections to eliminate resource leaks. Bug 1566946.

  • Backups to Swift will now use Static Large Objects for larger backups. A new configuration option ‘backup_segment_max_size’ can be set to adjust the segment size of the SLO. Backups that are smaller than the segment size will be uploaded as regular objects. This is an improvement over old Dynamic Large Object implementation. Bug 1489997.

  • Replace the deprecated ‘myisam-recover’ option with its newer counterpart ‘myisam-recover-options’.