2023.2 Series Release Notes

23.2.0-25

Security Issues

  • A ML2/SR-IOV port with status=DOWN will always set the VF link state to “disable”, regardless of the propagate_uplink_status port field value. The port disabling, to stop any transmission, has precedence over the link state “auto” value.

Bug Fixes

  • Fixes an issue when associating floating IPs to OVN load balancers. See LP#2068644 for more details.

23.2.0

Prelude

The OVN changed support for NAT rules including a new column and auto-discovery logic to know about logical router gateway ports for NAT on a Logical Router.

New Features

  • A new OVN driver Northbound DB column has been added to allow configuring gateway port for NAT rule. If the OVN backend supports the gateway_port column in the Northbound DB NAT table, the gateway port uuid will be configured to any floating IP to prevent North/South traffic issues. Previously created FIP rules will be updated only once during the maintenance task to include the gateway_port reference (if OVN backend supports it). In case all FIP entries are already configured no maintenance action will be performed.

  • A new ovn-cms-options option called enable-chassis-as-extport-host is now recognized by ML2/OVN and is used to identify nodes that are eligible for scheduling OVN’s external ports. This feature is backward compatible and if no nodes contain this new option the external ports will continue to be scheduled using the enable-chassis-as-gw option as before. This change also introduces a limit to the number of members for each HA Chassis Group to 5, matching the limit of gateway router port replicas. This is because OVN uses BFD to monitor the connectivity of each member and having an unlimited number of members could potentially put a lot of stress in OVN.

  • Remote address group support was added to the iptables-based firewall drivers (IptablesFirewallDriver and OVSHybridIptablesFirewallDriver), Previously it was only available in the OVSFirewallDriver. For more information, see bug 2058138.

  • Added the tags policies for the following resources: network, subnet, port, router, floating IP, network segment, network segment range, security group and security group rule. The policies control the creation, the update and the deletion of the resource tags.

Known Issues

  • The fix of bug 2048785 only fixes newly created trunk parent ports. If the fix of already existing trunks is needed, then either delete and re-create the affected trunks or set tpt ports’ vlan_mode and tag manually: ovs-vsctl set Port tpt-... vlan_mode=access tag=0

Bug Fixes

  • [bug 2036423] Now it is not possible to delete a subnet gateway IP if that subnet has a router interface; the subnet gateway IP modification was already forbidden.

  • When synchronizing the OVN databases, either when running the migration command or during startup, the code responsible for synchronization will only clean up segment-to-host mappings for hosts with agent_type OVN Controller agent. Before, the synchronization would clean up (delete) segment-to-host mappings for non-OVN hosts. Fixes bug: 2040172.

  • [bug 2045889] The ports bound to ML2/OVN now contain the OVS bridge name and datapath type in the VIF details dictionary. NOTE: in the ML2/OVS to ML2/OVN migration, the local host OVN bridge (integration bridge) per port is not known; “br-int” will be used by default (that value is rarely changed).

  • [bug 2036705] The Neutron port.status field (“ACTIVE”, “DOWN”) is now set based on the ML2/OVN Logical Switch Port up and enabled flags. The user can now set the port.admin_state_up, that is replicated in the lsp.enabled flag, to enable or disable the port. If the port is disabled, the traffic is stopped and the port.status is set to “DOWN”.

Other Notes

  • When the following configuration is enabled at the same time:

    • OVN L3 service plugin (ovn-router)

    • Port forwarding service plugin (port_forwarding)

    • “vlan” or “flat” network types configured in the ML2 configuration variable tenant_network_types

    • The OVN floating IP traffic is distributed (enable_distributed_floating_ip = True)

    the Neutron server will report a warning during plugin initialization because this is an invalid configuration matrix. Floating IPs need to always be centralized in such a case. For more details see bug report.

  • The new value for ‘device_owner’ for OVN loadbalancer health monitor ports (ovn-lb-hm:distributed) is now supported by Neutron, providing a LOCALPORT behavior to these ports. The responsibility to define these ports with the new value instead of the old one (network:distributed) is under the OVN-Octavia Provider driver, which will take care of database conversion for these ports.

  • Added extension subnetpool-prefix-ops to the ML2/OVN mechanism driver.

23.1.0

Bug Fixes

  • Fixed the scenario where the DHCP agent is deployed in conjunction with the OVN metadata agent in order to serve metadata for baremetal nodes. In this scenario, the DHCP agent would not set the route needed for the OVN metadata agent service resulting in baremetal nodes not being able to query the metadata service. For more information see bug 1982569.

  • During the port bulk creation, if an IPAM allocation fails (for example, if the IP address is outside of the subnet CIDR), the other IPAM allocations already created are deleted before raising the exception. Fixes bug 2039550.

23.0.0

New Features

  • A new API which allows a cloud administrator to define their own set of security group rules added automatically to every new default and/or custom security group created for projects.

  • Neutron allows cloud administrators to limit the rate at which VMs query the Nova metadata service in order to protect the OpenStack deployment from DoS or misbehaved instances. This new feature can be configured in the neutron.conf file. Please see the “Metadata service query rate limiting” section under Neutron configuration in the documentation for more details.

  • A new API extension network-ha has been added. This extension adds a new field to the network API: “ha”. This field is not visible and can be passed only in POST (create) operations. That will define that this network is a high availability (HA) network and will create, in the same database transaction, a ha_router_networks register.

  • The OVN L3 scheduler will assign chassis explicitly configured as gateways to the router gateway ports (OVN logical router ports). If no candidates are available, the “neutron-ovn-invalid-chassis” will be used instead and a warning message will be printed in the logs.

  • The first hint using the new hints port attribute is the ovs-tx-steering hint. The availability of this is marked by the extension: port-hint-ovs-tx-steering. This hint is specific to the openvswitch mechanism driver. It enables the control of Open vSwitch’s Userspace Tx packet steering feature. For details about the Open vSwitch feature please see: https://docs.openvswitch.org/en/latest/topics/userspace-tx-steering/ The valid values for the hints attribute introduced by the 2nd extension are: {"openvswitch": {"other_config": {"tx-steering": "hash|thread"}}}

  • Ports now have a hints attribute, in which backend specific tuning options can be passed to Neutron. The availability of the hints attribute is signaled by the port-hints extension. The hints attribute is admin-only by default. Its value is a dict, keyed by mechanism driver aliases. The possible values are defined by the mechanism drivers. An admin user can ask for a hint in a port create or update request. As the name suggests a hint is not a demand - Neutron applies the hint when it can, but it is free to ignore it, when it can’t.

Known Issues

  • When ML2/OVN backend and it’s L3 service plugin (ovn-router) are used together with “vlan” or “flat” network types used as tenant network type, floating IP port forwarding (FIP PF) will not work if distributed Floating IPs are enabled (enable_distributed_floating_ip = True). To workaround this issue one of the following changes needs to be done:

    • tunnel networks (“geneve” or “vxlan”) should be used as tenant network types or

    • Floating IPs should be centralized (enable_distributed_floating_ip = False).

    For more details see bug 2028846.

  • The high availability of metadata service on isolated networks is limited or non-existent. IPv4 metadata is redundant when the DHCP agent managing it is redundant, but recovery is tied to the renewal of the DHCP lease, making most recoveries very slow. IPv6 metadata is not redundant at all as the IPv6 metadata address can only be configured in a single place at a time as it is link-local. Multiple agents trying to configure it will generate an IPv6 duplicate address detection failure.

    Administrators may observe the IPv6 metadata address in “dadfailed” state in the DHCP namespace for this reason, which is only an indication it is not highly available. Until a redesign is made to the isolated metadata service there is not a better deployment option. See bug 1953165 for information.

  • In OVN 22.09 the option “localnet_learn_fdb” was added, enabling localnet ports to learn MAC addresses and store them at the FDB table. There is no aging mechanism for those MACs (that is the reason for not having this option enabled by default) and therefore it needs to be used with care, specially when provider networks are big. It is recommended to perform periodic manual cleanups of FDB table, to avoid scalability issues – until OVN implements an aging mechanism for this, tracked at https://bugzilla.redhat.com/show_bug.cgi?id=2179942.

  • The redirect-type=bridged option is only used if all the tenant networks connected to the router are of type VLAN or FLAT. In this case their traffic will be distributed. However, if there is a mix of VLAN/FLAT and geneve networks connected to the same router, the redirect-type option is not set, and therefore the traffic for the VLAN/FLAT networks will also be centralized but not tunneled.

  • When using ML2/OVN, during an upgrade procedure, the OVS system-id stored value can be changed. The ovn-controller service will create the “Chassis” and “Chassis_Private” registers based on this OVS system-id. If the ovn-controller process is not gracefully stopped, that could lead to the existence of duplicated “Chassis” and “Chassis_Private” registers in the OVN Southbound database.

Upgrade Notes

  • During the upgrade process, a set of four default security group rules will be created in the Neutron database. Those rules are the same default rules added to every new security group up to now:

    • rule to allow all egress IPv4 traffic (for all default and custom Security groups),

    • rule to allow all egress IPv6 traffic (for all default and custom Security groups),

    • rule to allow all ingress IPv4 traffic from the same security group (for default security group in each project),

    • rule to allow all ingress IPv6 traffic from the same security group (for default security group in each project).

    Those rules can now be modified by a cloud administrator using the default-security-group-rules API.

  • The Neutron service enable the API policies (RBAC) new defaults and scope by default. The Default value of config options [oslo_policy] enforce_scope and [oslo_policy] oslo_policy.enforce_new_defaults have been changed to True.

    This means if you are using system scope token to access Neutron API then the request will be failed with 403 error code. Also, new defaults will be enforced by default. To know about the new defaults of each policy rule, refer to the Policy New Defaults. For more detail about the Neutron API policies changes, refer to Policy Concepts.

    If you want to disable them then modify the below config options value in neutron.conf file:

    [oslo_policy]
enforce_new_defaults=False
enforce_scope=False

  • In ML2/OVN, any new router gateway port (OVN logical router port) will be scheduled only on those chassis configured as gateway. Any existing router gateway port will preserve the current chassis assignation.

Deprecation Notes

  • Support for Neutron on Windows operating systems is deprecated since 2023.2 release and will be removed in 2024.2.

  • “ipv6_pd_enabled” has been marked as deprecated and marked as experimental. To continue using it, deployers have to set to True the “ipv6_pd_enabled” option in the “experimental” section of neutron.conf. See Dibbler project concluded.

  • The tool neutron-debug is now removed. With removal of the neutron client shell code this tool is no longer usable. It had been marked for deprecation since the Newton (9.0) cycle and unmaintained.

Bug Fixes

  • The config option agent_down_time is now limited to a maximum value of 2147483, as neutron-server will fail to start if it is configured higher. See bug 2028724 for more information.

  • 1986003 Fixed an issue with concurrent requests to activate the same port binding where one of the requests returned a 500 Internal Server Error. With the fix one request will return successfully and the other will return a 409 Conflict (Binding already active). This fixes errors in nova live-migrations where those concurrent requests might be sent. Nova handles the 409/Conflict response gracefully.

  • [bug 1999209] Neutron-API now prevents internal IP change for floating IP. It will raise an error when deleting/changing the fixed IP which is linked to a floating IP.

  • [bug 2022914] Neutron-API supports using relays as the southbound connection in a ML2/OVN setup. Before the maintenance worker of the API required a leader_only connection, which was removed.

  • By default localnet ports don’t learn MAC addresses and therefore they are not stored in the FDB table at OVN SB DB. This leads to flooding issues when the destination traffic is an unknown IP by OpenStack. In OVN 22.09 the option “localnet_learn_fdb” was added, enabling those ports to learn MAC addresses and store them at the FDB table. Note there is no aging mechanism for those MACs, thus this is not enabled by default and needs to be used carefully, specially when provider networks are big, and/or performing manual cleanup of FDB table over time to avoid scalability issues, until OVN implements it at https://bugzilla.redhat.com/show_bug.cgi?id=2179942.

  • For OVN versions v22.09.0 and above, the mcast_flood_reports option is now set to false on all ports except “localnet” types. In the past, this option was set to true as a workaround for a bug in core OVN multicast implementation.

  • Fix an issue in the OVN driver where network metadata could become unavailable if the metadata port was ever deleted, even if accidental. To re-create the port, a user can now disable, then enable, DHCP for one of the subnets associated with the network using the Neutron API. This will try and create the port, similar to what happens in the DHCP agent for ML2/OVS. For more information, see bug 2015377.

  • [bug 2003455] As part of a previous commit (https://review.opendev.org/c/openstack/neutron/+/875644) the redirect-type=bridged option was set in all the router gateway ports (cr-lrp ovn ports). However this was breaking the N/S traffic for geneve tenant networks connected to the provider networks through those routers with the redirect-type option enabled. To fix this we ensure that the redirect-type option is only set if all the networks connected to the router are of VLAN or FLAT type, otherwise we fall back to the default option. This also means that if there is a mix of VLAN and geneve tenant networks connected to the same router, the VLAN traffic will be centralized (but not tunneled). If the traffic for the VLAN/FLAT needs to be distributed, then it should use a different router.

  • A new OVN maintenance method remove_duplicated_chassis_registers is added. This method will periodically check the OVN Southbound “Chassis” and “Chassis_Private” tables looking for duplicated registers. The older ones (based on the “Chassis_Private.nb_cfg_timestamp” value) will be removed when more than one register has the same hostname, that should be unique.

Other Notes

  • The external_mac entry in the NAT table is used to distribute/centralize the traffic to the FIPs. When there is an external_mac set the traffic is distributed (DVR). When it is empty it is centralized through the gateway port (no DVR). Upon port status transition to down, the external_mac was removed regardless of DVR being enabled or not, leading to centralize the FIP traffic for DVR – though it was for down ports that won’t accept traffic anyway.

  • Introduced a database constraint to limit the number of ha_router_networks registers per project to one only. This register is used to bind projects and networks, defining the corresponding network as high availability (HA) network. By definition, only one HA network per project can exist.

  • PluginReportStateAPI has a new version (1.3) in which has_alive_neutron_server() no longer returns always True, but performs a DB connection check and returns True/False accordingly. Using this, an agent can check not just MQ but the server’s DB connectivity too.

  • Adds a maintenance task that runs once a day and is responsible for cleaning up Hash Ring nodes that haven’t been updated in 5 days or more. See LP #2033281 for more information.

  • Add a new config option in DEFAULT section named my_ip. This new config option will be set by default to the local IPv4 configured to reach the default gateway. This new setting might be useful for operators that need to configure this IP in another part of their config (like OVS/local_ip) without relying on any external templating system.

  • Add a new config option in DEFAULT section named my_ipv6. This new config option will be set by default to the local IPv6 configured to reach the default gateway. This new setting might be useful for operators that need to configure this IP in another part of their config (like OVS/local_ip) without relying on any external templating system.

  • Added the missing extension uplink-status-propagation to the ML2/OVN mechanism driver. This extension is used by the ML2/SR-IOV mechanism driver, that could be loaded with ML2/OVN. Now it is possible to create ports with the “uplink-status-propagation” flag defined.

  • A ML2/OVN virtual port cannot be bound to a virtual machine. If a port IP address is assigned as an allowed address pair into another port, the first one is considered a virtual port. If the second port (non-virtual) is bound to ML2/OVN, the virtual port cannot be bound to a virtual machine; a virtual port is created only to reserve a set of IP addresses to be used by other ports. The OVN mechanism driver prevents that a virtual port has a device ID; a device ID is provided when the port is being bound.