Networking Guide

The Tricircle is to provide networking automation across Neutron servers in multi-region OpenStack clouds deployment, many cross Neutron networking modes are supported. In this guide, how to use CLI to setup typical networking mode will be described.

Networking Terms

There are four important networking terms will be used in networking automation across Neutron.

Local Network

  • Local Network is a network which can only reside in one OpenStack cloud.

  • Network type could be VLAN, VxLAN, Flat.

  • If you specify a region name as the value of availability-zone-hint during network creation, then the network will be created as local network in that region.

  • If the default network type to be created is configured to “local” in central Neutron, then no matter you specify availability-zone-hint or not, the network will be local network if the network was created without explicitly given non-local provider network type.

  • External network should be created as local network, that means external network is explicitly existing in some specified region. It’s possible that each region provides multiple external networks, that means there is no limitation on how many external networks can be created.

  • For example, local network could be created as follows:

    openstack --os-region-name=CentralRegion network create --availability-zone-hint=RegionOne net1
Local Router

  • Local Router is a logical router which can only reside in one OpenStack cloud.

  • If you specify a region name as the value of availability-zone-hint during router creation, then the router will be created as local router in that region.

  • For example, local router could be created as follows:

    neutron --os-region-name=CentralRegion router-create --availability-zone-hint RegionOne R1
Cross Neutron L2 Network

  • Cross Neutron L2 Network is a network which can be stretched into more than one Neutron servers, these Neutron servers may work in one OpenStack cloud or multiple OpenStack clouds.

  • Network type could be VLAN, VxLAN, Flat.

  • During the network creation, if availability-zone-hint is not specified, or specified with availability zone name, or more than one region name, or more than one availability zone name, then the network will be created as cross Neutron L2 network.

  • If the default network type to be created is not configured to “local” in central Neutron, then the network will be cross Neutron L2 network if the network was created without specified provider network type and single region name in availability-zone-hint.

  • For example, cross Neutron L2 network could be created as follows:

    neutron --os-region-name=CentralRegion net-create --provider:network_type vxlan --availability-zone-hint RegionOne --availability-zone-hint RegionTwo net1
Non-Local Router

  • Non-Local Router will be able to reside in more than one OpenStack cloud, and internally inter-connected with bridge network.

  • Bridge network used internally for non-local router is a special cross Neutron L2 network.

  • Local networks or cross Neutron L2 networks can be attached to local router or non-local routers if the network can be presented in the region where the router can reside.

  • During the router creation, if availability-zone-hint is not specified, or specified with availability zone name, or more than one region name, or more than one availability zone name, then the router will be created as non-local router.

  • For example, non-local router could be created as follows:

    neutron --os-region-name=CentralRegion router-create --availability-zone-hint RegionOne --availability-zone-hint RegionTwo R3

It’s also important to understand that cross Neutron L2 network, local router and non-local router can be created for different north-south/east-west networking purpose.

North-South and East-West Networking
  • Instances in different OpenStack clouds can be attached to a cross Neutron L2 network directly, so that they can communicate with each other no matter in which OpenStack cloud.
  • If L3 networking across OpenStack clouds is preferred, local network attached to non-local router can be created for instances to attach.
  • Local router can be set gateway with external networks to support north-south traffic handled locally.
  • Non-local router can work only for cross Neutron east-west networking purpose if no external network is set to the router.
  • Non-local router can serve as the centralized north-south traffic gateway if external network is attached to the router, and support east-west traffic at the same time.

Prerequisites

One CentralRegion in which central Neutron and Tricircle services are started, and central Neutron is configured with Tricircle Central Neutron plugin properly. And at least two regions(RegionOne, RegionTwo) in which Tricircle Local Neutron plugin is configured properly in local Neutron.

RegionOne is mapped to az1, and RegionTwo is mapped to az2 by pod management through Tricircle Admin API.

You can use az1 or RegionOne as the value of availability-zone-hint when creating a network. Although in this document only one region in one availability zone, one availability zone can include more than one region in Tricircle pod management, so if you specify az1 as the value, then it means the network will reside in az1, and az1 is mapped to RegionOne, if you add more regions into az1, then the network can spread into these regions too.

Please refer to the installation guide and configuration guide how to setup multi-region environment with Tricircle service enabled.

If you setup the environment through devstack, you can get these settings which are used in this document as follows:

Suppose that each node has 3 interfaces, and eth1 for tenant vlan network, eth2 for external vlan network. If you want to verify the data plane connectivity, please make sure the bridges “br-vlan” and “br-ext” are connected to regarding interface. Using following command to connect the bridge to physical ethernet interface, as shown below, “br-vlan” is wired to eth1, and “br-ext” to eth2:

sudo ovs-vsctl add-br br-vlan
sudo ovs-vsctl add-port br-vlan eth1
sudo ovs-vsctl add-br br-ext
sudo ovs-vsctl add-port br-ext eth2

Suppose the vlan range for tenant network is 101~150, external network is 151~200, in the node which will run central Neutron and Tricircle services, configure the local.conf like this:

Q_ML2_PLUGIN_VLAN_TYPE_OPTIONS=(network_vlan_ranges=bridge:101:150,extern:151:200)
OVS_BRIDGE_MAPPINGS=bridge:br-vlan,extern:br-ext

TRICIRCLE_START_SERVICES=True
enable_plugin tricircle https://github.com/openstack/tricircle/

If you also want to configure vxlan network, suppose the vxlan range for tenant network is 1001~2000, add the following configuration to the above local.conf:

Q_ML2_PLUGIN_VXLAN_TYPE_OPTIONS=(vni_ranges=1001:2000)

In the node which will run local Neutron without Tricircle services, configure the local.conf like this:

Q_ML2_PLUGIN_VLAN_TYPE_OPTIONS=(network_vlan_ranges=bridge:101:150,extern:151:200)
OVS_BRIDGE_MAPPINGS=bridge:br-vlan,extern:br-ext

TRICIRCLE_START_SERVICES=False
enable_plugin tricircle https://github.com/openstack/tricircle/

You may have noticed that the only difference is TRICIRCLE_START_SERVICES is True or False. All examples given in this document will be based on these settings.

If you also want to configure vxlan network, suppose the vxlan range for tenant network is 1001~2000, add the following configuration to the above local.conf:

Q_ML2_PLUGIN_VXLAN_TYPE_OPTIONS=(vni_ranges=1001:2000)

If you also want to configure flat network, suppose you use the same physical network as the vlan network, configure the local.conf like this:

Q_ML2_PLUGIN_FLAT_TYPE_OPTIONS=(flat_networks=bridge,extern)

In both RegionOne and RegionTwo, external network is able to be provisioned, the settings will look like this in /etc/neutron/plugins/ml2/ml2_conf.ini:

network_vlan_ranges = bridge:101:150,extern:151:200

vni_ranges = 1001:2000(or the range that you configure)

flat_networks = bridge,extern

bridge_mappings = bridge:br-vlan,extern:br-ext

Please be aware that the physical network name for tenant VLAN network is “bridge”, and the external network physical network name is “extern”.

In central Neutron’s configuration file, the default settings look like as follows:

bridge_network_type = vxlan
network_vlan_ranges = bridge:101:150,extern:151:200
vni_ranges = 1001:2000
flat_networks = bridge,extern
tenant_network_types = local,vlan,vxlan,flat
type_drivers = local,vlan,vxlan,flat

The default network type in central Neutron is local network, i.e, one network can only be presented in one local Neutron. In which region the local network will be located, it’s up to in which region the first instance will be booted in this network. After that, it’ll fail if you want to boot instance in another region to this network. The local network could be VLAN or VxLAN or GRE network by default, it’s up to your local Neutron’s configuration.

If you want to create a L2 network across multiple Neutron servers, then you have to speficy –provider-network-type vlan in network creation command for vlan network type, or –provider-network-type vxlan for vxlan network type. Both vlan and vxlan network type could work as the bridge network. The default bridge network type is vxlan.

If you want to create a flat network, which is usually used as the external network type, then you have to specify –provider-network-type flat in network creation command.

You can create L2 network for different purposes, and the supported network types for different purposes are summarized as follows.

Networking purpose Supported
Local L2 network for instances FLAT, VLAN, VxLAN
Cross Neutron L2 network for instances FLAT, VLAN, VxLAN
Bridge network for routers FLAT, VLAN, VxLAN
External network FLAT, VLAN