19.10¶
Summary¶
The 19.10 OpenStack Charms release includes updates for the following charms. Additional charm support status information is published in the OpenStack Charm Guide which ultimately supersedes Release Notes contents.
Always use the latest stable charm revision before proceeding with topological changes, application migrations, workload upgrades, series upgrades, or bug reports.
Supported Charms¶
aodh
barbican
barbican-vault
ceilometer
ceilometer-agent
ceph-mon
ceph-osd
ceph-proxy
ceph-radosgw
ceph-rbd-mirror
cinder
cinder-ceph
cinder-purestorage
designate
designate-bind
glance
gnocchi
hacluster
heat
keystone
keystone-ldap
lxd
neutron-api
neutron-openvswitch
neutron-gateway
neutron-dynamic-routing
nova-cloud-controller
nova-compute
octavia
octavia-dashboard
octavia-diskimage-retrofit
openstack-dashboard
percona-cluster
placement
rabbitmq-server
swift-proxy
swift-storage
vault
Preview Charms¶
barbican-softhsm
ceph-fs
cinder-backup
keystone-saml-mellon
manila
manila-generic
masakari
masakari-monitors
mysql-innodb-cluster
mysql-router
neutron-api-plugin-ovn
ovn-central
ovn-chassis
ovn-dedicated-chassis
pacemaker-remote
tempest
Removed Charms¶
n/a
New Charm Features¶
With each new feature, there is a corresponding example bundle in the form of a test bundle, and/or a OpenStack Charms Deployment Guide section which details the use of the feature. For example test bundles, see the src/tests/bundles/ directory within the relevant charm repository.
OpenStack Train¶
The 19.10 OpenStack Charms release introduces support for OpenStack Train on Ubuntu 18.04 LTS (via UCA) and Ubuntu 19.10.
Please note the placement charm must be deployed and related to the
nova-cloud-controller charm as of OpenStack Train. See section Placement
Charm for more details.
nova-cloud-controller: instance migration - DNS caching is now the default¶
Note
This has nothing to do with Designate or Neutron DNS.
The 19.07 release introduced the option to enable caching of DNS lookups of the nova-compute units which are used by nova to perform migrations of tenant instances between nova-compute units. This was not enabled by default. The 19.10 release “flips the switch” and enables this option by default so that after upgrade, if the option was not explicitly set, DNS results will be cached automatically. New clouds will also, unless the option is explicitly set to false, cache DNS host lookups.
The reason for caching DNS host lookups is to reduce the time to add nova-compute units to an existing cloud, or during first deployment.
Please see the config option cache-known-hosts on the
nova-cloud-controller charm for further details.
Policy Overrides¶
The Policy Overrides feature provides operators with a mechanism to override policy defaults on a per-service basis.
Policy defaults for an OpenStack service are defined via “policy-in-code” and/or via a default policy YAML file provided by the charm. The operator can use the new feature by providing a ZIP file consisting of at least one YAML file which contains policy rules that the service will observe when responding to API queries. This allows operators to selectively override the default policies of that service.
This feature is being provided in the following charms:
cinder
designate
glance
keystone
neutron-api
nova-cloud-controller
For further details consult appendix Policy Overrides in the OpenStack Charms Deployment Guide.
Please consult the README for each charm to determine exactly what is provided with respect to this feature.
Ceph Nautilus¶
Along with OpenStack Train, the 19.10 charm release includes support for the Nautilus release of Ceph.
Ceph Placement Group Autotuning¶
In Ceph Nautilus, the OpenStack Charms now support autotuning of placement groups. The new pg_autoscaler module allows the cluster to consider the amount of data stored, or expected, in each pool and manage the correct pg_num values automatically.
This feature can be disabled entirely by setting a new configuration option, pg-autotune, to “false”. This option defaults to “auto” which will cause new deployments on Ceph Nautilus to enable the autoscaler, but older releases upgraded to Nautilus will need to explicitly opt-in by setting pg-autotune to “true”.
Neutron Port Forwarding¶
Neutron Port forwarding extension can now be optionally enabled with 19.10 charms for OpenStack Rocky and later. Be aware that the Train openstack-client version (or later) may be required in order to interact with the feature from the command line.
For more information on this feature see the Neutron documentation.
Migration to FQDN for agent registration¶
Starting with the 19.10 charms when deploying OpenStack Stein or newer, the
Nova Compute agent as deployed by the nova-compute charm and Neutron agents
as deployed by the neutron-openvswitch charm will use a fully qualified
domain name (FQDN) when registering with the API services.
As the name an agent registers with is referenced across multiple services in a OpenStack cloud this change will only apply to newly deployed units. Upgrading the above two charms or upgrading OpenStack will not trigger the new behaviour.
The backdrop of the change is:
Fix bugs in third party services relating to consistency of hypervisor naming in Nova.
Avoid deploy time issues in the event a node is configured without a search domain, and also support clouds with a segregated DNS layout.
Switch to a more sensible default mode of operation enabling easier integration with new technologies such as OVN.
Ceph RADOS Gateway tenant namespacing¶
The ceph-radosgw charm now supports deployment with tenant namespaces.
This is enabled during initial deployment using the namespace-tenants
configuration option. Enabling this option post deployment will have no effect
as it is not possible to migrate a deployment without tenant namespacing to one
with tenant namespacing enabled.
Endpoint URLs for object storage will contain the tenant ID in the form:
http://<cephradosgw-unit-ip-or-vip>:80/swift/v1/AUTH_<tenant-id>
This feature allows per-tenant bucket namespaces, rather than a global bucket namespace, which is aligned to the behaviour of OpenStack Swift.
Placement Charm¶
The 19.10 OpenStack Charms release introduces a new charm for the placement
API. The placement API service was extracted from the Nova project in
OpenStack Train and moved to its own project. Therefore, the new placement
charm must be deployed and related to the nova-cloud-controller charm for
OpenStack Train deployments. See section Upgrading OpenStack for more
details on how to introduce the placement charm into existing deployments when
upgrading to OpenStack Train.
Cinder Integration with Pure Storage Array¶
The 19.10 OpenStack Charms release introduces a new charm which can be used to integrate cinder with a Pure Storage array. To use the new subordinate charm:
juju deploy cinder
juju deploy cinder-purestorage
juju add-relation cinder-purestorage cinder
The cinder-purestorage charm needs to be configured with the IP address of the storage array and provided with an API token for authentication. Typically the settings that need configuring are:
protocol: iscsi
volume-backend-name: cinder-pure
san-ip: PURESTORAGE_IP
pure-api-token: API_TOKEN
Preview Charm Features¶
mysql-innodb-cluster and mysql-router¶
The 19.10 OpenStack Charms release introduces two new charms to deploy MySQL 8 for OpenStack: mysql-innodb-cluster and mysql-router.
Note
These charms are in preview state and are not production-ready. The charms are ready for testing in OpenStack clouds.
Note
Both charms are only deployable on Ubuntu 19.10 and greater.
The mysql-innodb-cluster charm deploys MySQL 8 in an InnoDB cluster with a read/write node and N number of read-only nodes.
Note
The mysql-innodb-cluster charm is intended for deploying a cluster and therefore does not support single-unit or non-clustered deployments.
The mysql-router charm deploys a MySQL 8 Router which will proxy database requests from the principle charm application to a MySQL 8 InnoDB Cluster. MySQL Router handles cluster communication and understands the cluster schema.
Note
The mysql-router charm is deployed as a subordinate on the principle charm application.
A simple example deployment:
juju deploy cs:keystone
juju deploy cs:~openstack-charmers-next/mysql-router
juju deploy -n 3 cs:~openstack-charmers-next/mysql-innodb-cluster
juju add-relation mysql-router:db-router mysql-innodb-cluster:db-router
OVN¶
The 19.10 OpenStack Charms release introduces a suite of four preview charms that allows you to model Open Virtual Network (OVN). OVN provides open source network virtualization for Open vSwitch (OVS).
One of the main drivers for this enablement work is the prospect of being able to hardware-offload everything. This is possible due to how OVN programs everything in Open vSwitch with OpenFlow rules. This in turn provides a uniform way of programming the hardware forwarding tables of supported NICs.
Hardware-offloading is a prerequisite for effective handling of workloads with high bandwidth consumption.
OVN also provides a more flexible way of configuring external Layer3 networking
as OVN does not require every node (Chassis in OVN terminology) in a
deployment to have direct external connectivity. This plays nicely with
Layer3-only datacenter fabrics (RFC 7938).
East/West traffic is distributed by default. North/South traffic is highly available by default. Liveness detection is done using the Bidirectional Forwarding Detection (BFD) protocol.
Please refer to appendix Open Virtual Network (OVN) in the OpenStack Charms Deployment Guide for more details.
Known feature gaps at this point in time:
No validation has been done with DPDK, SR-IOV or hardware-offloading in the charms.
Support for the Octavia OVN provider driver has not been implemented in the charms and no validation has been done with LBaaS in general.
Only limited validation has been done with other Neutron extensions, and it may be possible to configure unsupported combinations of features with undefined results.
There is an unresolved issue with security groups rules that reference remote security groups. Please remove any such rules while testing.
Example of how you could reset your default security group rules:
PROJECT_ID=$(openstack project list -f value -c ID \
--domain admin_domain)
SECGRP_ID=$(openstack security group list --project $PROJECT_ID \
| awk '/default/{print$2}')
openstack security group rule delete \
$(openstack security group rule list $SECGRP_ID| awk '/IPv./{print$2}')
openstack security group rule create --egress --protocol any \
--ethertype IPv4 $SECGRP_ID
openstack security group rule create --egress --protocol any \
--ethertype IPv6 $SECGRP_ID
openstack security group rule create --ingress --protocol any \
--ethertype IPv4 $SECGRP_ID --remote-ip YOUR_IPV4_LAB_NETWORK_CIDR
openstack security group rule create --ingress --protocol any \
--ethertype IPv6 $SECGRP_ID --remote-ip YOUR_IPV6_LAB_NETWORK_CIDR
Upgrading charms¶
Always use the latest stable charm revision before proceeding with topological changes, charm application migrations, workload upgrades, series upgrades, or bug reports.
Please ensure that the keystone charm is upgraded first.
To upgrade an existing deployment to the latest charm version simply use the
upgrade-charm command. For example:
juju upgrade-charm keystone
Charm upgrades and OpenStack upgrades are functionally different. Charm upgrades ensure that the deployment has the latest charm revision, containing the latest charm fixes and charm features available for that deployment, whereas OpenStack upgrades influence the software package versions of OpenStack itself.
Charm upgrades do not trigger OpenStack upgrades. However, OpenStack upgrades do require the latest charm version as pre-requisite.
Upgrading OpenStack¶
Note
Upgrading an OpenStack cloud is not without risk; upgrades should be tested in pre-production testing environments prior to production deployment upgrades.
See appendix OpenStack Upgrades in the OpenStack Charms Deployment Guide for more details.
Before upgrading OpenStack, all OpenStack Charms should be running the latest stable charm revision.
To upgrade an existing Stein-based deployment on Ubuntu 18.04 to the Train release, re-configure the charm with a new openstack-origin configuration. For example:
juju config nova-compute openstack-origin=cloud:bionic-train
As of Train, the placement API is managed by the new placement charm and is
no longer managed by the nova-cloud-controller charm. The upgrade to Train
therefore requires some coordination to transition to the new API endpoints.
Prior to upgrading nova-cloud-controller to Train, the placement charm must be deployed for Train and related to the Stein-based nova-cloud-controller. It is important that the nova-cloud-controller unit leader is paused while the API transition occurs (paused prior to adding relations for the placement charm) as the placement charm will migrate existing placement tables from the nova_api database to a new placement database. Once the new placement endpoints are registered, nova-cloud-controller can be resumed.
Here’s an example of the steps just described:
juju deploy --series bionic --config openstack-origin=cloud:bionic-train cs:placement
juju run-action nova-cloud-controller/leader pause
juju add-relation placement mysql
juju add-relation placement keystone
juju add-relation placement nova-cloud-controller
openstack endpoint list # ensure placement endpoints are listening on new placment IP address
juju run-action nova-cloud-controller/leader resume
Only after these steps have been completed can nova-cloud-controller be upgraded. Here we upgrade all units simultaneously but see the Paused-single-unit method in the OpenStack Charms Deployment Guide for a more controlled approach:
juju config nova-cloud-controller openstack-origin=cloud:bionic-train
New Bundle Features¶
n/a
Deprecation Notices¶
n/a
Removed Features¶
Ceph Nautilus has removed support for directory backed OSDs. The charms will allow for the creation of directory backed OSDs on older Ceph releases but will log a warning about their use from Nautilus onwards. Existing directory backed OSDs will continue to function after an upgrade to Nautilus.
Known Issues¶
Masakari and Masakari Monitors¶
Both Masakari charms remain as previews. Bugs LP #1728527 and LP #1839715 need to be resolved in order to arrive at a successful instance HA deployment. Bug LP #1773765 is likely to affect on-going support of a Masakari deployment.
Glance Simplestreams Sync¶
When deploying the glance-simplestreams-sync charm on Bionic a more recent
version of the simplestreams package must be installed by configuring a PPA:
juju config glance-simplestreams-sync source=ppa:simplestreams-dev/trunk
See bug LP #1790904 for details.
Designate and Vault at Ocata and earlier¶
The designate charm for OpenStack releases Pike and earlier does not yet
support SSL via Vault and the certificates relation. See bug LP #1839019
Current versions of OpenStack with Vault and the certificates relation are supported by the Designate charm.
Restart Nova services after adding certificates relation¶
A race condition exists with the use of the ‘certificates’ relation. When SSL certificates are issued Nova services may attempt to talk to the placement API over HTTP while the API has already changed to HTTPS. See bug LP #1826382.
To mitigate against this, restart nova-compute and nova-scheduler services once certificates have been issued:
juju run --application nova-compute "systemctl restart nova-compute"
juju run --application nova-cloud-controller "systemctl restart nova-scheduler"
Juju Version¶
The 19.10 set of OpenStack charms was validated with Juju 2.6.9 and 2.6.10 (candidate). While some validation has been done with edge versions of Juju 2.7.x, a full qualification of Eoan series and cross-model-relation remains to be done. This validation is considered important, and will be covered in conjunction with 2.7.x RCs or beta versions.
Bugs Fixed¶
This release includes 41 bug fixes. For the full list of bugs resolved for the 19.10 charms release please refer to the 19.10 milestone in Launchpad.
Next Release Info¶
Please see the OpenStack Charm Guide for current information.
