Neutron Metering system¶

The Neutron metering service enables operators to account the traffic in/out of the OpenStack environment. The concept is quite simple, operators can create metering labels, and decide if the labels are applied to all projects (tenants) or if they are applied to a specific one. Then, the operator needs to create traffic rules in the metering labels. The traffic rules are used to match traffic in/out of the OpenStack environment, and the accounting of packets and bytes is sent to the notification queue for further processing by Ceilometer (or some other system that is consuming that queue). The message sent in the queue is of type event. Therefore, it requires an event processing configuration to be added/enabled in Ceilometer.

The metering agent has the following configurations:

driver: the driver used to implement the metering rules. The default is neutron.services.metering.drivers.noop, which means, we do not execute anything in the networking host. The only driver implemented so far is neutron.services.metering.drivers.iptables.iptables_driver.IptablesMeteringDriver. Therefore, only iptables is supported so far;
measure_interval: the interval in seconds used to gather the bytes and packets information from the network plane. The default value is 30 seconds;
report_interval: the interval in seconds used to generated the report (message) of the data that is gathered. The default value is 300 seconds.
granular_traffic_data: Defines if the metering agent driver should present traffic data in a granular fashion, instead of grouping all of the traffic data for all projects and routers where the labels were assigned to. The default value is False for backward compatibility.

Non-granular traffic messages¶

The non-granular (granular_traffic_data = False) traffic messages (here also called as legacy) have the following format; bear in mind that if labels are shared, then the counters are for all routers of all projects where the labels were applied.

{
 "pkts": "<the number of packets that matched the rules of the labels>",
 "bytes": "<the number of bytes that matched the rules of the labels>",
 "time": "<seconds between the first data collection and the last one>",
 "first_update": "timeutils.utcnow_ts() of the first collection",
 "last_update": "timeutils.utcnow_ts() of the last collection",
 "host": "<neutron metering agent host name>",
 "label_id": "<the label id>",
 "tenant_id": "<the tenant id>"
 }

The first_update and last_update timestamps represent the moment when the first and last data collection happened within the report interval. On the other hand, the time represents the difference between those two timestamp.

The tenant_id is only consistent when labels are not shared. Otherwise, they will contain the project id of the last router of the last project processed when the agent is started up. In other words, it is better not use it when dealing with shared labels.

All of the messages generated in this configuration mode are sent to the message bus as l3.meter events.

Granular traffic messages¶

The granular (granular_traffic_data = True) traffic messages allow operators to obtain granular information for shared metering labels. Therefore, a single label, when configured as shared=True and applied in all projects/routers of the environment, it will generate data in a granular fashion.

It (the metering agent) will account the traffic counter data in the following granularities.

label – all of the traffic counter for a given label. One must bear in mind that a label can be assigned to multiple routers. Therefore, this granularity represents all aggregation for all data for all routers of all projects where the label has been applied.
router – all of the traffic counter for all labels that are assigned to the router.
project – all of the traffic counters for all labels of all routers that a project has.
router-label – all of the traffic counters for a router and the given label.
project-label – all of the traffic counters for all routers of a project that have a given label.

Each granularity presented here is sent to the message bus with different events types that vary according to the granularity. The mapping between granularity and event type is presented as follows.

label – event type l3.meter.label.
router – event type l3.meter.router.
project – event type l3.meter.project..
router-label – event type l3.meter.label_router.
project-label – event type l3.meter.label_project.

Furthermore, we have metadata that is appended to the messages depending on the granularity. As follows we present the mapping between the granularities and the metadata that will be available.

label, router-label, and project-label granularities – have the metadata label_id, label_name, label_shared, project_id (if shared, this value will come with all for the label granularity), and router_id (only for router-label granularity).
The router granularity – has the router_id and project_id metadata.
The project granularity only has the project_id metadata.

The message will also contain some attributes that can be found in the legacy mode such as bytes, pkts, time, first_update, last_update, and host. As follows we present an example of JSON message with all of the possible attributes.

{
    "resource_id": "router-f0f745d9a59c47fdbbdd187d718f9e41-label-00c714f1-49c8-462c-8f5d-f05f21e035c7",
    "project_id": "f0f745d9a59c47fdbbdd187d718f9e41",
    "first_update": 1591058790,
    "bytes": 0,
    "label_id": "00c714f1-49c8-462c-8f5d-f05f21e035c7",
    "label_name": "test1",
    "last_update": 1591059037,
    "host": "<hostname>",
    "time": 247,
    "pkts": 0,
    "label_shared": true
}

The resource_id is a unique identified for the “resource” being monitored. Here we consider a resource to be any of the granularities that we handle.

Sample of metering_agent.ini¶

As follows we present all of the possible configuration one can use in the metering agent init file.

DEFAULT¶

debug¶

Type:: boolean
Default:: False
Mutable:: This option can be changed without restarting.

If set to true, the logging level will be set to DEBUG instead of the default INFO level.

log_config_append¶

Type:: string
Default:: <None>
Mutable:: This option can be changed without restarting.

The name of a logging configuration file. This file is appended to any existing logging configuration files. For details about logging configuration files, see the Python logging module documentation. Note that when logging configuration files are used then all logging configuration is set in the configuration file and other logging configuration options are ignored (for example, log-date-format).

Deprecated Variations¶
Group	Name
DEFAULT	log-config
DEFAULT	log_config

log_date_format¶

Type:: string
Default:: %Y-%m-%d %H:%M:%S

Defines the format string for %(asctime)s in log records. Default: the value above . This option is ignored if log_config_append is set.

log_file¶

Type:: string
Default:: <None>

(Optional) Name of log file to send logging output to. If no default is set, logging will go to stderr as defined by use_stderr. This option is ignored if log_config_append is set.

Deprecated Variations¶
Group	Name
DEFAULT	logfile

log_dir¶

Type:: string
Default:: <None>

(Optional) The base directory used for relative log_file paths. This option is ignored if log_config_append is set.

Deprecated Variations¶
Group	Name
DEFAULT	logdir

watch_log_file¶

Type:: boolean
Default:: False

Uses logging handler designed to watch file system. When log file is moved or removed this handler will open a new log file with specified path instantaneously. It makes sense only if log_file option is specified and Linux platform is used. This option is ignored if log_config_append is set.

Warning

This option is deprecated for removal. Its value may be silently ignored in the future.

Reason:: This function is known to have bene broken for long time, and depends on the unmaintained library

use_syslog¶

Type:: boolean
Default:: False

Use syslog for logging. Existing syslog format is DEPRECATED and will be changed later to honor RFC5424. This option is ignored if log_config_append is set.

use_journal¶

Type:: boolean
Default:: False

Enable journald for logging. If running in a systemd environment you may wish to enable journal support. Doing so will use the journal native protocol which includes structured metadata in addition to log messages.This option is ignored if log_config_append is set.

syslog_log_facility¶

Type:: string
Default:: LOG_USER

Syslog facility to receive log lines. This option is ignored if log_config_append is set.

use_json¶

Type:: boolean
Default:: False

Use JSON formatting for logging. This option is ignored if log_config_append is set.

use_stderr¶

Type:: boolean
Default:: False

Log output to standard error. This option is ignored if log_config_append is set.

use_eventlog¶

Type:: boolean
Default:: False

Log output to Windows Event Log.

Warning

This option is deprecated for removal. Its value may be silently ignored in the future.

Reason:: Windows support is no longer maintained.

log_color¶

Type:: boolean
Default:: False

(Optional) Set the ‘color’ key according to log levels. This option takes effect only when logging to stderr or stdout is used. This option is ignored if log_config_append is set.

log_rotate_interval¶

Type:: integer
Default:: 1

The amount of time before the log files are rotated. This option is ignored unless log_rotation_type is set to “interval”.

log_rotate_interval_type¶

Type:: string
Default:: days
Valid Values:: Seconds, Minutes, Hours, Days, Weekday, Midnight

Rotation interval type. The time of the last file change (or the time when the service was started) is used when scheduling the next rotation.

max_logfile_count¶

Type:: integer
Default:: 30

Maximum number of rotated log files.

max_logfile_size_mb¶

Type:: integer
Default:: 200

Log file maximum size in MB. This option is ignored if “log_rotation_type” is not set to “size”.

log_rotation_type¶

Type:: string
Default:: none
Valid Values:: interval, size, none

Log rotation type.

Possible values

interval: Rotate logs at predefined time intervals.
size: Rotate logs once they reach a predefined size.
none: Do not rotate log files.

logging_context_format_string¶

Type:: string
Default:: %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(user_identity)s] %(instance)s%(message)s

Format string to use for log messages with context. Used by oslo_log.formatters.ContextFormatter

logging_default_format_string¶

Type:: string
Default:: %(asctime)s.%(msecs)03d %(process)d %(levelname)s %(name)s [-] %(instance)s%(message)s

Format string to use for log messages when context is undefined. Used by oslo_log.formatters.ContextFormatter

logging_debug_format_suffix¶

Type:: string
Default:: %(funcName)s %(pathname)s:%(lineno)d

Additional data to append to log message when logging level for the message is DEBUG. Used by oslo_log.formatters.ContextFormatter

logging_exception_prefix¶

Type:: string
Default:: %(asctime)s.%(msecs)03d %(process)d ERROR %(name)s %(instance)s

Prefix each line of exception output with this format. Used by oslo_log.formatters.ContextFormatter

logging_user_identity_format¶

Type:: string
Default:: %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s

Defines the format string for %(user_identity)s that is used in logging_context_format_string. Used by oslo_log.formatters.ContextFormatter

default_log_levels¶

Type:: list
Default:: ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO']

List of package logging levels in logger=LEVEL pairs. This option is ignored if log_config_append is set.

publish_errors¶

Type:: boolean
Default:: False

Enables or disables publication of error events.

instance_format¶

Type:: string
Default:: "[instance: %(uuid)s] "

The format for an instance that is passed with the log message.

instance_uuid_format¶

Type:: string
Default:: "[instance: %(uuid)s] "

The format for an instance UUID that is passed with the log message.

rate_limit_interval¶

Type:: integer
Default:: 0

Interval, number of seconds, of log rate limiting.

rate_limit_burst¶

Type:: integer
Default:: 0

Maximum number of logged messages per rate_limit_interval.

rate_limit_except_level¶

Type:: string
Default:: CRITICAL
Valid Values:: CRITICAL, ERROR, INFO, WARNING, DEBUG, ‘’

Log level name used by rate limiting. Logs with level greater or equal to rate_limit_except_level are not filtered. An empty string means that all levels are filtered.

fatal_deprecations¶

Type:: boolean
Default:: False

Enables or disables fatal status of deprecations.

ovs_use_veth¶

Type:: boolean
Default:: False

Uses veth for an OVS interface or not. Support kernels with limited namespace support (e.g. RHEL 6.5) and rate limiting on router’s gateway port so long as ovs_use_veth is set to True.

interface_driver¶

Type:: string
Default:: <None>

The driver used to manage virtual interfaces.

rpc_response_max_timeout¶

Type:: integer
Default:: 600

Maximum seconds to wait for a response from an RPC call.

driver¶

Type:: string
Default:: neutron.services.metering.drivers.noop.noop_driver.NoopMeteringDriver

Metering driver

measure_interval¶

Type:: integer
Default:: 30

Interval between two metering measures

report_interval¶

Type:: integer
Default:: 300

Interval between two metering reports

granular_traffic_data¶

Type:: boolean
Default:: False

Defines if the metering agent driver should present traffic data in a granular fashion, instead of grouping all of the traffic data for all projects and routers where the labels were assigned to. The default value is False for backward compatibility.

agent¶

report_interval¶

Type:: floating point
Default:: 30

Seconds between nodes reporting state to server; should be less than agent_down_time, best if it is half or less than agent_down_time.

log_agent_heartbeats¶

Type:: boolean
Default:: False

Log agent heartbeats

ovs¶

ovsdb_connection¶

Type:: string
Default:: tcp:127.0.0.1:6640

The connection string for the OVSDB backend. Will be used for all OVSDB commands and by ovsdb-client when monitoring

ssl_key_file¶

Type:: string
Default:: <None>

The SSL private key file to use when interacting with OVSDB. Required when using an “ssl:” prefixed ovsdb_connection

ssl_cert_file¶

Type:: string
Default:: <None>

The SSL certificate file to use when interacting with OVSDB. Required when using an “ssl:” prefixed ovsdb_connection

ssl_ca_cert_file¶

Type:: string
Default:: <None>

The Certificate Authority (CA) certificate to use when interacting with OVSDB. Required when using an “ssl:” prefixed ovsdb_connection

ovsdb_debug¶

Type:: boolean
Default:: False

Enable OVSDB debug logs

ovsdb_timeout¶

Type:: integer
Default:: 10

Timeout in seconds for OVSDB commands. If the timeout expires, OVSDB commands will fail with ALARMCLOCK error.

bridge_mac_table_size¶

Type:: integer
Default:: 50000

The maximum number of MAC addresses to learn on a bridge managed by the Neutron OVS agent. Values outside a reasonable range (10 to 1,000,000) might be overridden by Open vSwitch according to the documentation.

igmp_snooping_enable¶

Type:: boolean
Default:: False

Enable IGMP snooping for integration bridge. If this option is set to True, support for Internet Group Management Protocol (IGMP) is enabled in integration bridge.

igmp_flood¶

Type:: boolean
Default:: False

Multicast packets (except reports) are unconditionally forwarded to the ports bridging a logical network to a physical network.

igmp_flood_reports¶

Type:: boolean
Default:: True

Multicast reports are unconditionally forwarded to the ports bridging a logical network to a physical network.

igmp_flood_unregistered¶

Type:: boolean
Default:: False

This option enables or disables flooding of unregistered multicast packets to all ports. If False, The switch will send unregistered multicast packets only to ports connected to multicast routers.

Neutron Metering system

Neutron Metering system¶

Non-granular traffic messages¶

Granular traffic messages¶

Sample of metering_agent.ini¶

DEFAULT¶

agent¶

ovs¶

Neutron 25.0.1.dev27

Page Contents