6. Test Results

• 6.1. 1000 Compute nodes resource scalability testing
  • 6.1.1. Environment setup
  • 6.1.2. Additional information
  • 6.1.3. Methodology
• 6.2. Ceph RBD performance report
  • 6.2.1. Environment description
    • 6.2.1.1. Hardware configuration of each server
    • 6.2.1.2. Network configuration of each server
    • 6.2.1.3. Software configuration on servers with controller, compute and compute-osd roles
  • 6.2.2. Testing process
  • 6.2.3. Test results
• Container cluster systems performance
  • 1. Results of measuring performance of Mesos Marathon
    • 1.1. Environment description
      • 1.1.1. Cluster description
      • 1.1.2. Software versions
      • 1.1.3. Running options
      • 1.1.4. Hardware configuration of each server
      • 1.1.5. Network scheme and part of configuration of hardware network switches
    • 1.2. Results
      • 1.2.1. Test create
      • 1.2.2. Test delete
      • 1.2.3. Test restart
      • 1.2.4. Test update_cpu
      • 1.2.5. Test update_disk
      • 1.2.6. Test update_instances
      • 1.2.7. Test update_mem
      • 1.2.8. Test health check
  • 2. Kubernetes system performance
    • 2.1. Results of measuring of API performance of Kubernetes
      • 2.1.1. Environment description
        • 2.1.1.1. Hardware configuration of each server
        • 2.1.1.2. Network scheme and part of configuration of hardware network switches
        • 2.1.1.3. Software configuration of Kubernetes service
          • 2.1.1.3.1. Setting up Kubernetes
          • 2.1.1.3.2. Operating system configuration
        • 2.1.1.4. Software configuration of Test tool:
          • 2.1.1.4.1. Test tool preparation
          • 2.1.1.4.2. Test tool description
          • 2.1.1.4.3. Operating system configuration:
      • 2.1.2. Testing process
        • 2.1.2.1. Preparation
      • 2.1.3. Results
        • 2.1.3.1. 10-nodes cluster (all values are presented in milliseconds)
          • 2.1.3.1.1. resourcequotas
          • 2.1.3.1.2. secrets
          • 2.1.3.1.3. replicationcontrollers
          • 2.1.3.1.4. namespaces
          • 2.1.3.1.5. nodes
          • 2.1.3.1.6. endpoints
          • 2.1.3.1.7. pods
        • 2.1.3.2. 50-nodes cluster (all values are presented in milliseconds)
          • 2.1.3.2.1. resourcequotas
          • 2.1.3.2.2. jobs
          • 2.1.3.2.3. secrets
          • 2.1.3.2.4. replicasets
          • 2.1.3.2.5. replicationcontrollers
          • 2.1.3.2.6. daemonsets
          • 2.1.3.2.7. deployments
          • 2.1.3.2.8. petsets
          • 2.1.3.2.9. namespaces
          • 2.1.3.2.10. services
          • 2.1.3.2.11. bindings
          • 2.1.3.2.12. endpoints
          • 2.1.3.2.13. pods
          • 2.1.3.2.14. nodes
        • 2.1.3.3. 355-nodes cluster (all values are presented in milliseconds)
          • 2.1.3.3.1. resourcequotas
          • 2.1.3.3.2. jobs
          • 2.1.3.3.3. secrets
          • 2.1.3.3.4. replicasets
          • 2.1.3.3.5. replicationcontrollers
          • 2.1.3.3.6. daemonsets
          • 2.1.3.3.7. deployments
          • 2.1.3.3.8. petsets
          • 2.1.3.3.9. namespaces
          • 2.1.3.3.10. services
          • 2.1.3.3.11. nodes
          • 2.1.3.3.12. endpoints
          • 2.1.3.3.13. pods
        • 2.1.3.4. Comparation
        • 2.1.3.5. Kubernetes pod startup latency measurement
      • 2.1.4. Applications
        • 2.1.4.1. Files and scripts to build Docker container with e2e-test tool
        • 2.1.4.2. Files and scripts to run Docker container with e2e-test tool
        • 2.1.4.3. Script to convert JSON from log file to RST table
        • 2.1.4.4. Pod startup time measurement toolkit
    • 2.2. Kubernetes density test report
      • 2.2.1. Environment description
        • 2.2.1.1. Software
      • 2.2.2. Reports
        • 2.2.2.1. Test Case #1: Maximum pods per node
          • 2.2.2.1.1. Detailed Stats
            • 2.2.2.1.1.1. 50 pods
            • 2.2.2.1.1.2. 100 pods
            • 2.2.2.1.1.3. 200 pods
            • 2.2.2.1.1.4. 400 pods
            • 2.2.2.1.1.5. Scale by 100 pods steps
        • 2.2.2.2. Test Case #2: Measure Kubelet capacity
          • 2.2.2.2.1. Detailed Stats
            • 2.2.2.2.1.1. 50 pods (~1 pod per core) on 50 nodes
            • 2.2.2.2.1.2. 100 pods (~2 pod per core) on 50 nodes
            • 2.2.2.2.1.3. 200 pods (~4 pod per core) on 50 nodes
            • 2.2.2.2.1.4. 50 pods (~1 pod per core) on 100 nodes
            • 2.2.2.2.1.5. 100 pods (~2 pod per core) on 100 nodes
            • 2.2.2.2.1.6. 200 pods (~4 pod per core) on 100 nodes
            • 2.2.2.2.1.7. 50 pods (~1 pod per core) on 200 nodes
            • 2.2.2.2.1.8. 100 pods (~2 pod per core) on 200 nodes
            • 2.2.2.2.1.9. 200 pods (~4 pod per core) on 200 nodes
            • 2.2.2.2.1.10. 400 pods (~8 pod per core) on 50 nodes
  • 3. Results of measuring performance of Kargo
    • 3.1. Environment description
      • 3.1.1. Hardware configuration of each server
      • 3.1.2. Network scheme and part of configuration of hardware network switches
      • 3.1.3. Software configuration of Kargo
        • 3.1.3.1. Setting up Kargo:
        • 3.1.3.2. Test tool:
        • 3.1.3.3. Operating system configuration:
    • 3.2. Testing process
    • 3.3. Results
    • 3.4. Issues that have been found during the tests
    • 3.5. Applications
      • 3.5.1. Launcher script
      • 3.5.2. Script for parsing collected metrics
  • 4. Results of the ETCD health tests
    • 4.1. Environment description
      • 4.1.1. Hardware configuration of each server
      • 4.1.2. Network scheme and part of configuration of hardware network switches
      • 4.1.3. Software configuration
        • 4.1.3.1. Setting up Kubernetes cluster:
        • 4.1.3.2. Test tools
    • 4.2. Testing process
    • 4.3. Results
      • 4.3.1. Case 1: “ETCD vs PODS”
        • 4.3.1.1. ETCD performance(time series graphs)
        • 4.3.1.2. ETCD performance(number of pods dependency)
      • 4.3.2. Case 2: “ETCD vs SERVICES”
        • 4.3.2.1. ETCD performance(time series graphs)
        • 4.3.2.2. ETCD performance(number of services dependency)
      • 4.3.3. Analysis
        • 4.3.3.1. Case 1 (ETCD vs PODS)
        • 4.3.3.2. Case 2 (ETCD vs SERVICES)
    • 4.4. Applications
      • 4.4.1. prepare-minion.py
• 6.3. Container repositories performance
  • 6.3.1. Results of measuring performance of Docker Registry
    • 6.3.1.1. Environment description
      • 6.3.1.1.1. Hardware configuration of each server
      • 6.3.1.1.2. Network scheme and part of configuration of hardware network switches
      • 6.3.1.1.3. Software configuration of the DockerRegistry service
        • 6.3.1.1.3.1. Installation of Registry2:
        • 6.3.1.1.3.2. Operating system configuration:
      • 6.3.1.1.4. Software configuration of the node with test tool
        • 6.3.1.1.4.1. Test tool:
        • 6.3.1.1.4.2. Operating system:
    • 6.3.1.2. Testing process
    • 6.3.1.3. Results
      • 6.3.1.3.1. Push action results
        • 6.3.1.3.1.1. PUSH_TIME(ITERATION)
        • 6.3.1.3.1.2. PUSH_TIME(CONCURRENCY)
      • 6.3.1.3.2. Pull action results
        • 6.3.1.3.2.1. PULL_TIME(ITERATION)
        • 6.3.1.3.2.2. PULL_TIME(CONCURRENCY)
    • 6.3.1.4. Issues which have been found during the tests
  • 6.3.2. Results of measuring performance of JFrog Artifactory Pro
    • 6.3.2.1. Environment description
      • 6.3.2.1.1. Hardware configuration of each server
      • 6.3.2.1.2. Network scheme and part of configuration of hardware network switches
      • 6.3.2.1.3. Software configuration of the Artifactory service
        • 6.3.2.1.3.1. Installation of Artifactory:
        • 6.3.2.1.3.2. Operating system configuration:
      • 6.3.2.1.4. Software configuration of the node with test tool
        • 6.3.2.1.4.1. Test tool:
        • 6.3.2.1.4.2. Operating system:
    • 6.3.2.2. Testing process
    • 6.3.2.3. Results
      • 6.3.2.3.1. Push action results
        • 6.3.2.3.1.1. PUSH_TIME(ITERATION)
        • 6.3.2.3.1.2. PUSH_TIME(CONCURRENCY)
      • 6.3.2.3.2. Pull action results
        • 6.3.2.3.2.1. PULL_TIME(ITERATION)
        • 6.3.2.3.2.2. PULL_TIME(CONCURRENCY)
    • 6.3.2.4. Issues which have been found during the tests
  • 6.3.3. Results of measuring performance of Sonatype Nexus
    • 6.3.3.1. Environment description
      • 6.3.3.1.1. Hardware configuration of each server
      • 6.3.3.1.2. Network scheme and part of configuration of hardware network switches
      • 6.3.3.1.3. Software configuration of the Nexus service
        • 6.3.3.1.3.1. Installation of Nexus:
        • 6.3.3.1.3.2. Operating system configuration:
      • 6.3.3.1.4. Software configuration of the node with test tool
        • 6.3.3.1.4.1. Test tool:
        • 6.3.3.1.4.2. Operating system:
    • 6.3.3.2. Testing process
    • 6.3.3.3. Results
      • 6.3.3.3.1. Push action results
        • 6.3.3.3.1.1. PUSH_TIME(ITERATION)
        • 6.3.3.3.1.2. PUSH_TIME(CONCURRENCY)
      • 6.3.3.3.2. Pull action results
        • 6.3.3.3.2.1. PULL_TIME(ITERATION)
        • 6.3.3.3.2.2. PULL_TIME(CONCURRENCY)
    • 6.3.3.4. Issues which have been found during the tests
• 6.4. Results of OpenStack Services density testing
  • 6.4.1. Environment description
    • 6.4.1.1. Hardware configuration of each server
    • 6.4.1.2. Network configuration of each server
    • 6.4.1.3. Software configuration on servers with controller and compute roles
    • 6.4.1.4. Software configuration on servers with Rally role
  • 6.4.2. Test results
    • 6.4.2.1. Cinder
    • 6.4.2.2. Neutron
    • 6.4.2.3. Nova
• 6.5. Openstack control plane performance tests results
  • 6.5.1. OpenStack control plane performance report for 1000 nodes
    • 6.5.1.1. Environment description
      • 6.5.1.1.1. Hardware configuration of each server
      • 6.5.1.1.2. Network configuration of each server
      • 6.5.1.1.3. Software configuration on servers with controller, osd and compute roles
      • 6.5.1.1.4. Software configuration on servers with rally role
    • 6.5.1.2. Testing process
    • 6.5.1.3. Test results
      • 6.5.1.3.1. Cinder
      • 6.5.1.3.2. Glance
      • 6.5.1.3.3. Keystone
      • 6.5.1.3.4. Neutron
      • 6.5.1.3.5. Nova
    • 6.5.1.4. Issues which have been found during the tests
  • 6.5.2. OpenStack control plane performance report for 200 nodes
    • 6.5.2.1. Environment description
      • 6.5.2.1.1. Hardware configuration of each server
      • 6.5.2.1.2. Network configuration of each server
      • 6.5.2.1.3. Software configuration on servers with controller, compute and compute-osd roles
      • 6.5.2.1.4. Software configuration on servers with rally role
    • 6.5.2.2. Testing process
    • 6.5.2.3. Test results
      • 6.5.2.3.1. Cinder
      • 6.5.2.3.2. Glance
      • 6.5.2.3.3. Keystone
      • 6.5.2.3.4. Neutron
      • 6.5.2.3.5. Nova
  • 6.5.3. OpenStack control plane performance report for 400 nodes
    • 6.5.3.1. Environment description
      • 6.5.3.1.1. Hardware configuration of each server
      • 6.5.3.1.2. Network configuration of each server
      • 6.5.3.1.3. Software configuration on servers with controller, compute and compute-osd roles
      • 6.5.3.1.4. Software configuration on servers with rally role
    • 6.5.3.2. Testing process
    • 6.5.3.3. Test results
      • 6.5.3.3.1. Cinder
      • 6.5.3.3.2. Glance
      • 6.5.3.3.3. Keystone
      • 6.5.3.3.4. Neutron
      • 6.5.3.3.5. Nova
  • 6.5.4. OpenStack control plane performance report for 6 nodes
    • 6.5.4.1. Environment description
      • 6.5.4.1.1. Hardware configuration of each server
      • 6.5.4.1.2. Network configuration of each server
      • 6.5.4.1.3. Software configuration on servers with controller, compute and compute-osd roles
      • 6.5.4.1.4. Software configuration of test suite
    • 6.5.4.2. Test results
      • 6.5.4.2.1. Keystone with uuid tokens
      • 6.5.4.2.2. Keysotne with fernet tokens
      • 6.5.4.2.3. Glance with filesystem backend
      • 6.5.4.2.4. Glance with Ceph backend
• 6.6. SQL Database performance report
  • 6.6.1. MySQL Database performance report
    • 6.6.1.1. Environment description
      • 6.6.1.1.1. Software
    • 6.6.1.2. Reports
      • 6.6.1.2.1. MySQL + Galera performance report
        • 6.6.1.2.1.1. Throughput
        • 6.6.1.2.1.2. Throughput and server CPU consumption
        • 6.6.1.2.1.3. Operation latency
      • 6.6.1.2.2. MySQL + Galera + HAproxy performance report
        • 6.6.1.2.2.1. Throughput
        • 6.6.1.2.2.2. Throughput and server CPU consumption
        • 6.6.1.2.2.3. Operation latency
  • 6.6.2. MariaDB Database performance report
    • 6.6.2.1. Environment description
      • 6.6.2.1.1. Software
    • 6.6.2.2. Reports
      • 6.6.2.2.1. MariaDB + Galera performance report
        • 6.6.2.2.1.1. Throughput
        • 6.6.2.2.1.2. Throughput and server CPU consumption
        • 6.6.2.2.1.3. Operation latency
• 6.7. SQL Database on Kubernetes performance report
  • 6.7.1. SQL Database on kubernetes performance report
    • 6.7.1.1. Environment description
      • 6.7.1.1.1. Software
    • 6.7.1.2. Reports
      • 6.7.1.2.1. MySQL on Kubernetes with Ceph performance report
        • 6.7.1.2.1.1. YAML for Kubernetes pod
        • 6.7.1.2.1.2. CEPH status
        • 6.7.1.2.1.3. /etc/ceph/ceph.conf
        • 6.7.1.2.1.4. Throughput
      • 6.7.1.2.2. MySQL on Kubernetes with volume on host
        • 6.7.1.2.2.1. YAML for Kubernetes pod
        • 6.7.1.2.2.2. Throughput
      • 6.7.1.2.3. MySQL on Kubernetes without volume
        • 6.7.1.2.3.1. YAML for Kubernetes pod
        • 6.7.1.2.3.2. Throughput
      • 6.7.1.2.4. Common graphs
• 6.8. Fuel Containerized Control Plane performance report
  • 6.8.1. Environment description
    • 6.8.1.1. Hardware configuration of each server
    • 6.8.1.2. Network configuration of each server
  • 6.8.2. Test results
    • 6.8.2.1. Test Case 1: Boot and delete server
      • 6.8.2.1.1. 200 nodes OpenStack cluster, concurrency 5, 910 iterations
      • 6.8.2.1.2. 350 nodes OpenStack cluster, concurrency 5, 1750 iterations
    • 6.8.2.2. Test Case 2: Boot and list servers
      • 6.8.2.2.1. 150 nodes OpenStack cluster, concurrency 20, 5520 iterations
      • 6.8.2.2.2. 150 nodes OpenStack cluster, concurrency 40, 5520 iterations
      • 6.8.2.2.3. 200 nodes OpenStack cluster, concurrency 5, 910 iterations
      • 6.8.2.2.4. 200 nodes OpenStack cluster, concurrency 40, 7280 iterations
    • 6.8.2.3. Test case 3: Keystone authentication
      • 6.8.2.3.1. 50 nodes OpenStack cluster, 30 RPS, 12000 iterations
      • 6.8.2.3.2. 50 nodes OpenStack cluster, 60 RPS, 12000 iterations
      • 6.8.2.3.3. 50 nodes OpenStack cluster, 90 RPS, 12000 iterations
      • 6.8.2.3.4. 50 nodes OpenStack cluster, 120 RPS, 12000 iterations
      • 6.8.2.3.5. 50 nodes OpenStack cluster, 150 RPS, 12000 iterations
• 6.9. Fuel Containerized Control Plane performance report during upgrade
  • 6.9.1. Environment description
    • 6.9.1.1. Hardware configuration of each server
    • 6.9.1.2. Network configuration of each server
    • 6.9.1.3. Fuel CCP configuration
    • 6.9.1.4. Rally configuration and scenario
    • 6.9.1.5. Upgrade and tests procedure
  • 6.9.2. Test results
    • 6.9.2.1. Test Case 1: Boot and delete server
      • 6.9.2.1.1. 149 nodes OpenStack cluster, concurrency 5, 1530 iterations
• 6.10. Hardware features testing
  • 6.10.1. Hardware Offloads - Test results
    • 6.10.1.1. Baseline physical scenario
    • 6.10.1.2. Baseline physical over VxLAN scenario
    • 6.10.1.3. VM-to-VM on different nodes scenario
    • 6.10.1.4. VM-to-VM on different nodes over VxLAN scenario
    • 6.10.1.5. Summary
• 6.11. Results of measuring performance of Cinder with Ceph backend
  • 6.11.1. Environment description
    • 6.11.1.1. Hardware configuration of each server
    • 6.11.1.2. Configuration of physical network
    • 6.11.1.3. Ceph cluster configuration
    • 6.11.1.4. Kubernetes + Calico configuration
    • 6.11.1.5. OpenStack deployment configuration
      • 6.11.1.5.1. Test tools:
  • 6.11.2. Testing process
    • 6.11.2.1. Preparation
    • 6.11.2.2. Test Case 1
    • 6.11.2.3. Test Case 2
  • 6.11.3. Results
    • 6.11.3.1. Test Case 1
      • 6.11.3.1.1. HTML reports generated by Rally
      • 6.11.3.1.2. Graphs which shows time dependecy from amount of instances
    • 6.11.3.2. Test Case 2
      • 6.11.3.2.1. Average data
      • 6.11.3.2.2. Graphs for average results
      • 6.11.3.2.3. Summary data
      • 6.11.3.2.4. Graphs for summary results
  • 6.11.4. Issues which have been found during the tests
    • 6.11.4.1. Ceph performance degradation due to use of non optimal configuration
  • 6.11.5. Applications
    • 6.11.5.1. Sofware versions used during tests
• 6.12. Keystone Performance Test Reports
  • 6.12.1. Reports
    • 6.12.1.1. Keystone DB / cache operations analysis
      • 6.12.1.1.1. Environment description
        • 6.12.1.1.1.1. Hardware
        • 6.12.1.1.1.2. Software
      • 6.12.1.1.2. Execution
      • 6.12.1.1.3. Short summary
      • 6.12.1.1.4. Reports
        • 6.12.1.1.4.1. Running profiling against Liberty Keystone (Fernet tokens, cache turned off)
          • 6.12.1.1.4.1.1. Parameters
          • 6.12.1.1.4.1.2. Endpoint list request stats
          • 6.12.1.1.4.1.3. Server create request stats
          • 6.12.1.1.4.1.4. Service list request stats
          • 6.12.1.1.4.1.5. Token issue request stats
          • 6.12.1.1.4.1.6. User list request stats
        • 6.12.1.1.4.2. Running profiling against Liberty Keystone (Fernet tokens, cache turned on)
          • 6.12.1.1.4.2.1. Parameters
          • 6.12.1.1.4.2.2. Endpoint list request stats
          • 6.12.1.1.4.2.3. Server create request stats
          • 6.12.1.1.4.2.4. Service list request stats
          • 6.12.1.1.4.2.5. Token issue request stats
          • 6.12.1.1.4.2.6. User list request stats
        • 6.12.1.1.4.3. Running profiling against Liberty Keystone (UUID tokens, cache turned off)
          • 6.12.1.1.4.3.1. Parameters
          • 6.12.1.1.4.3.2. Endpoint list request stats
          • 6.12.1.1.4.3.3. Server create request stats
          • 6.12.1.1.4.3.4. Service list request stats
          • 6.12.1.1.4.3.5. Token issue request stats
          • 6.12.1.1.4.3.6. User list request stats
        • 6.12.1.1.4.4. Running profiling against Liberty Keystone (UUID tokens, cache turned on)
          • 6.12.1.1.4.4.1. Parameters
          • 6.12.1.1.4.4.2. Endpoint list request stats
          • 6.12.1.1.4.4.3. Server create request stats
          • 6.12.1.1.4.4.4. Service list request stats
          • 6.12.1.1.4.4.5. Token issue request stats
          • 6.12.1.1.4.4.6. User list request stats
        • 6.12.1.1.4.5. Running profiling against Mitaka Keystone (Fernet tokens, cache turned off)
          • 6.12.1.1.4.5.1. Parameters
          • 6.12.1.1.4.5.2. Endpoint list request stats
          • 6.12.1.1.4.5.3. Server create request stats
          • 6.12.1.1.4.5.4. Service list request stats
          • 6.12.1.1.4.5.5. Token issue request stats
          • 6.12.1.1.4.5.6. User list request stats
        • 6.12.1.1.4.6. Running profiling against Mitaka Keystone (Fernet tokens, cache turned on)
          • 6.12.1.1.4.6.1. Parameters
          • 6.12.1.1.4.6.2. Endpoint list request stats
          • 6.12.1.1.4.6.3. Server create request stats
          • 6.12.1.1.4.6.4. Service list request stats
          • 6.12.1.1.4.6.5. Token issue request stats
          • 6.12.1.1.4.6.6. User list request stats
        • 6.12.1.1.4.7. Running profiling against Mitaka Keystone (UUID tokens, cache turned off)
          • 6.12.1.1.4.7.1. Parameters
          • 6.12.1.1.4.7.2. Endpoint list request stats
          • 6.12.1.1.4.7.3. Server create request stats
          • 6.12.1.1.4.7.4. Service list request stats
          • 6.12.1.1.4.7.5. Token issue request stats
          • 6.12.1.1.4.7.6. User list request stats
        • 6.12.1.1.4.8. Running profiling against Mitaka Keystone (UUID tokens, cache turned on)
          • 6.12.1.1.4.8.1. Parameters
          • 6.12.1.1.4.8.2. Endpoint list request stats
          • 6.12.1.1.4.8.3. Server create request stats
          • 6.12.1.1.4.8.4. Service list request stats
          • 6.12.1.1.4.8.5. Token issue request stats
          • 6.12.1.1.4.8.6. User list request stats
• 6.13. Monitoring systems test results
  • 6.13.1. LMA Test Results
    • 6.13.1.1. Environment description
      • 6.13.1.1.1. Hardware configuration of each server
      • 6.13.1.1.2. Software configuration of the services
        • 6.13.1.1.2.1. Installation of OpenStack and LMA plugins:
    • 6.13.1.2. Testing process
    • 6.13.1.3. Usage Results
      • 6.13.1.3.1. Collector: Hekad / collectd
      • 6.13.1.3.2. Influxdb
      • 6.13.1.3.3. Elasticsearch
      • 6.13.1.3.4. All RabbitMQ queues collection impact
    • 6.13.1.4. Reliability Results
      • 6.13.1.4.1. Backends outage for 2 hours
        • 6.13.1.4.1.1. Data loss
          • 6.13.1.4.1.1.1. Elasticsearch
        • 6.13.1.4.1.2. Data lost
          • 6.13.1.4.1.2.1. Apache2/Nagios3
      • 6.13.1.4.2. Elasticsearch failover/recovery
      • 6.13.1.4.3. InfluxDB failover/recovery
      • 6.13.1.4.4. Apache2 overloaded
    • 6.13.1.5. Outcomes
      • 6.13.1.5.1. InfluxDB
      • 6.13.1.5.2. Elasticsearch
      • 6.13.1.5.3. Sizing guide
    • 6.13.1.6. Issues which have been found during the tests
• 6.14. Message Queue Performance Reports
  • 6.14.1. RabbitMQ Performance Report
    • 6.14.1.1. Environment description
    • 6.14.1.2. Reports
      • 6.14.1.2.1. RabbitMQ performance (Client and Server connected to Master)
        • 6.14.1.2.1.1. Test Case 1: RPC CALL Throughput Test
          • 6.14.1.2.1.1.1. Message processing
          • 6.14.1.2.1.1.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.1.1.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.1.2. Test Case 2: RPC CAST Throughput Test
          • 6.14.1.2.1.2.1. Message processing
          • 6.14.1.2.1.2.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.1.2.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.1.3. Test Case 3: Notification Throughput Test
          • 6.14.1.2.1.3.1. Message processing
          • 6.14.1.2.1.3.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.1.3.3. Detailed RabbitMQ CPU consumption
      • 6.14.1.2.2. RabbitMQ performance (Client -> Slave-1, Server -> Slave-2)
        • 6.14.1.2.2.1. Test Case 1: RPC CALL Throughput Test
          • 6.14.1.2.2.1.1. Message processing
          • 6.14.1.2.2.1.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.2.1.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.2.2. Test Case 2: RPC CAST Throughput Test
          • 6.14.1.2.2.2.1. Message processing
          • 6.14.1.2.2.2.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.2.2.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.2.3. Test Case 3: Notification Throughput Test
          • 6.14.1.2.2.3.1. Message processing
          • 6.14.1.2.2.3.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.2.3.3. Detailed RabbitMQ CPU consumption
      • 6.14.1.2.3. RabbitMQ performance (Client and Server connected to Master), HA queues enabled
        • 6.14.1.2.3.1. Test Case 1: RPC CALL Throughput Test
          • 6.14.1.2.3.1.1. Message processing
          • 6.14.1.2.3.1.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.3.1.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.3.2. Test Case 2: RPC CAST Throughput Test
          • 6.14.1.2.3.2.1. Message processing
          • 6.14.1.2.3.2.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.3.2.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.3.3. Test Case 3: Notification Throughput Test
          • 6.14.1.2.3.3.1. Message processing
          • 6.14.1.2.3.3.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.3.3.3. Detailed RabbitMQ CPU consumption
      • 6.14.1.2.4. RabbitMQ performance (Client -> Slave-1, Server -> Slave-2), HA queues enabled
        • 6.14.1.2.4.1. Test Case 1: RPC CALL Throughput Test
          • 6.14.1.2.4.1.1. Message processing
          • 6.14.1.2.4.1.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.4.1.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.4.2. Test Case 2: RPC CAST Throughput Test
          • 6.14.1.2.4.2.1. Message processing
          • 6.14.1.2.4.2.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.4.2.3. Detailed RabbitMQ CPU consumption
        • 6.14.1.2.4.3. Test Case 3: Notification Throughput Test
          • 6.14.1.2.4.3.1. Message processing
          • 6.14.1.2.4.3.2. The throughput, latency and RabbitMQ CPU utilization
          • 6.14.1.2.4.3.3. Detailed RabbitMQ CPU consumption
      • 6.14.1.2.5. High-level performance overview
  • 6.14.2. ZeroMQ performance
    • 6.14.2.1. Environment description
      • 6.14.2.1.1. Software
    • 6.14.2.2. Test Case 1: RPC CALL Throughput Test
      • 6.14.2.2.1. Message processing
      • 6.14.2.2.2. The throughput and latency
    • 6.14.2.3. Test Case 2: RPC CAST Throughput Test
      • 6.14.2.3.1. Message processing
      • 6.14.2.3.2. The throughput and latency
    • 6.14.2.4. Test Case 3: Notification Throughput Test
• 6.15. RabbitMQ HA test reports
  • 6.15.1. Environment description
  • 6.15.2. Reports
    • 6.15.2.1. RabbitMQ HA Test Reports: HA queues disabled
      • 6.15.2.1.1. Environment description
        • 6.15.2.1.1.1. Software
      • 6.15.2.1.2. Reports
        • 6.15.2.1.2.1. Client and Server connected to Master, Master fails
          • 6.15.2.1.2.1.1. RPC CALL fail-over test report
            • 6.15.2.1.2.1.1.1. Execution Summary
            • 6.15.2.1.2.1.1.2. Message flow
            • 6.15.2.1.2.1.1.3. Messages sent by the client
            • 6.15.2.1.2.1.1.4. Messages received by the server
            • 6.15.2.1.2.1.1.5. Replies received by the client
          • 6.15.2.1.2.1.2. RPC CAST fail-over test report
            • 6.15.2.1.2.1.2.1. Execution Summary
            • 6.15.2.1.2.1.2.2. Message flow
            • 6.15.2.1.2.1.2.3. Messages sent by the client
            • 6.15.2.1.2.1.2.4. Messages received by the server
          • 6.15.2.1.2.1.3. NOTIFY fail-over test report
            • 6.15.2.1.2.1.3.1. Execution Summary
            • 6.15.2.1.2.1.3.2. Message flow
            • 6.15.2.1.2.1.3.3. Messages sent by the client
            • 6.15.2.1.2.1.3.4. Messages received by the server
        • 6.15.2.1.2.2. Client and Server connected to Slave 1, Slave 1 fails
          • 6.15.2.1.2.2.1. RPC CALL fail-over test report
            • 6.15.2.1.2.2.1.1. Execution Summary
            • 6.15.2.1.2.2.1.2. Message flow
            • 6.15.2.1.2.2.1.3. Messages sent by the client
            • 6.15.2.1.2.2.1.4. Messages received by the server
            • 6.15.2.1.2.2.1.5. Replies received by the client
          • 6.15.2.1.2.2.2. RPC CAST fail-over test report
            • 6.15.2.1.2.2.2.1. Execution Summary
            • 6.15.2.1.2.2.2.2. Message flow
            • 6.15.2.1.2.2.2.3. Messages sent by the client
            • 6.15.2.1.2.2.2.4. Messages received by the server
          • 6.15.2.1.2.2.3. NOTIFY fail-over test report
            • 6.15.2.1.2.2.3.1. Execution Summary
            • 6.15.2.1.2.2.3.2. Message flow
            • 6.15.2.1.2.2.3.3. Messages sent by the client
            • 6.15.2.1.2.2.3.4. Messages received by the server
        • 6.15.2.1.2.3. Client connected to Slave 1 and Server connected to Slave 2, Slave 2 fails
          • 6.15.2.1.2.3.1. RPC CALL fail-over test report
            • 6.15.2.1.2.3.1.1. Execution Summary
            • 6.15.2.1.2.3.1.2. Message flow
            • 6.15.2.1.2.3.1.3. Messages sent by the client
            • 6.15.2.1.2.3.1.4. Messages received by the server
            • 6.15.2.1.2.3.1.5. Replies received by the client
          • 6.15.2.1.2.3.2. RPC CAST fail-over test report
            • 6.15.2.1.2.3.2.1. Execution Summary
            • 6.15.2.1.2.3.2.2. Message flow
            • 6.15.2.1.2.3.2.3. Messages sent by the client
            • 6.15.2.1.2.3.2.4. Messages received by the server
          • 6.15.2.1.2.3.3. NOTIFY fail-over test report
            • 6.15.2.1.2.3.3.1. Execution Summary
            • 6.15.2.1.2.3.3.2. Message flow
            • 6.15.2.1.2.3.3.3. Messages sent by the client
            • 6.15.2.1.2.3.3.4. Messages received by the server
        • 6.15.2.1.2.4. Client connected to Master and Server connected to Slave, Master fails
          • 6.15.2.1.2.4.1. RPC CALL fail-over test report
            • 6.15.2.1.2.4.1.1. Execution Summary
            • 6.15.2.1.2.4.1.2. Message flow
            • 6.15.2.1.2.4.1.3. Messages sent by the client
            • 6.15.2.1.2.4.1.4. Messages received by the server
            • 6.15.2.1.2.4.1.5. Replies received by the client
          • 6.15.2.1.2.4.2. RPC CAST fail-over test report
            • 6.15.2.1.2.4.2.1. Execution Summary
            • 6.15.2.1.2.4.2.2. Message flow
            • 6.15.2.1.2.4.2.3. Messages sent by the client
            • 6.15.2.1.2.4.2.4. Messages received by the server
          • 6.15.2.1.2.4.3. NOTIFY fail-over test report
            • 6.15.2.1.2.4.3.1. Execution Summary
            • 6.15.2.1.2.4.3.2. Message flow
            • 6.15.2.1.2.4.3.3. Messages sent by the client
            • 6.15.2.1.2.4.3.4. Messages received by the server
    • 6.15.2.2. RabbitMQ HA Test Reports: HA queues enabled
      • 6.15.2.2.1. Environment description
        • 6.15.2.2.1.1. Software
      • 6.15.2.2.2. Reports
        • 6.15.2.2.2.1. Client connected to Slave 1 and Server connected to Slave 2, Slave 2 fails
          • 6.15.2.2.2.1.1. RPC CALL fail-over test report
            • 6.15.2.2.2.1.1.1. Execution Summary
            • 6.15.2.2.2.1.1.2. Message flow
            • 6.15.2.2.2.1.1.3. Messages sent by the client
            • 6.15.2.2.2.1.1.4. Messages received by the server
            • 6.15.2.2.2.1.1.5. Replies received by the client
          • 6.15.2.2.2.1.2. RPC CAST fail-over test report
            • 6.15.2.2.2.1.2.1. Execution Summary
            • 6.15.2.2.2.1.2.2. Message flow
            • 6.15.2.2.2.1.2.3. Messages sent by the client
            • 6.15.2.2.2.1.2.4. Messages received by the server
          • 6.15.2.2.2.1.3. NOTIFY fail-over test report
            • 6.15.2.2.2.1.3.1. Execution Summary
            • 6.15.2.2.2.1.3.2. Message flow
            • 6.15.2.2.2.1.3.3. Messages sent by the client
            • 6.15.2.2.2.1.3.4. Messages received by the server
        • 6.15.2.2.2.2. Client connected to Master and Server connected to Slave, Master fails
          • 6.15.2.2.2.2.1. RPC CALL fail-over test report
            • 6.15.2.2.2.2.1.1. Execution Summary
            • 6.15.2.2.2.2.1.2. Message flow
            • 6.15.2.2.2.2.1.3. Messages sent by the client
            • 6.15.2.2.2.2.1.4. Messages received by the server
            • 6.15.2.2.2.2.1.5. Replies received by the client
          • 6.15.2.2.2.2.2. RPC CAST fail-over test report
            • 6.15.2.2.2.2.2.1. Execution Summary
            • 6.15.2.2.2.2.2.2. Message flow
            • 6.15.2.2.2.2.2.3. Messages sent by the client
            • 6.15.2.2.2.2.2.4. Messages received by the server
          • 6.15.2.2.2.2.3. NOTIFY fail-over test report
            • 6.15.2.2.2.2.3.1. Execution Summary
            • 6.15.2.2.2.2.3.2. Message flow
            • 6.15.2.2.2.2.3.3. Messages sent by the client
            • 6.15.2.2.2.2.3.4. Messages received by the server
• 6.16. OpenStack Networking (Neutron) control plane performance report for 400 nodes
  • 6.16.1. Environment description
    • 6.16.1.1. Lab A (200 nodes)
      • 6.16.1.1.1. Hardware configuration of each server
    • 6.16.1.2. Lab B (378 nodes)
      • 6.16.1.2.1. Hardware configuration of each server
      • 6.16.1.2.2. Network configuration of each server
      • 6.16.1.2.3. Software configuration on servers with controller, compute and compute-osd roles
      • 6.16.1.2.4. Software configuration on servers with Rally role
  • 6.16.2. Test results
    • 6.16.2.1. Test Case 1: Basic Neutron test suite
    • 6.16.2.2. Test Case 2: Stressful Neutron test suite
      • 6.16.2.2.1. Observed trends
    • 6.16.2.3. Test case 3: Neutron scalability test with many networks
    • 6.16.2.4. Test case 4: Neutron scalability test with many servers
• 6.17. Neutron features scale testing
  • 6.17.1. Neutron L3 HA test results Liberty
    • 6.17.1.1. Environment description
      • 6.17.1.1.1. Cluster description
      • 6.17.1.1.2. Software versions
      • 6.17.1.1.3. Hardware configuration of each server
    • 6.17.1.2. Rally test results
      • 6.17.1.2.1. Summary:
    • 6.17.1.3. Shaker test results
      • 6.17.1.3.1. Summary:
    • 6.17.1.4. Manual tests execution
      • 6.17.1.4.1. Ping to external network from VM during reset of primary(non-primary) controller
      • 6.17.1.4.2. Ping from one VM to another VM in different network during ban L3 agent
      • 6.17.1.4.3. Iperf UPD testing between VMs in different networks ban L3 agent
      • 6.17.1.4.4. Summary:
  • 6.17.2. Neutron L3 HA test results Mitaka
    • 6.17.2.1. Environment description
      • 6.17.2.1.1. Cluster description
      • 6.17.2.1.2. Software versions
      • 6.17.2.1.3. Hardware configuration of each server
    • 6.17.2.2. Rally test results
      • 6.17.2.2.1. Summary:
    • 6.17.2.3. Shaker test results
      • 6.17.2.3.1. Summary:
    • 6.17.2.4. Manual tests execution
      • 6.17.2.4.1. Ping to external network from VM during reset of primary(non-primary) controller
      • 6.17.2.4.2. Ping from one VM to another VM in different network during ban L3 agent
      • 6.17.2.4.3. Iperf UPD testing between VMs in different networks ban L3 agent
      • 6.17.2.4.4. Summary:
  • 6.17.3. OpenStack Neutron Resource Density Test Report
    • 6.17.3.1. Environment description
      • 6.17.3.1.1. Cluster description
      • 6.17.3.1.2. Software versions
      • 6.17.3.1.3. Hardware configuration of each server
    • 6.17.3.2. Report
      • 6.17.3.2.1. Preparation
      • 6.17.3.2.2. Measurements
        • 6.17.3.2.2.1. Baseline
        • 6.17.3.2.2.2. 10 instances per compute in separate network
        • 6.17.3.2.2.3. 30 instances per compute in separate network
        • 6.17.3.2.2.4. 10 instances per compute in tester network
        • 6.17.3.2.2.5. 30 instances per compute in tester network
        • 6.17.3.2.2.6. 10 instances and 10 routers per compute
        • 6.17.3.2.2.7. 30 instances and 30 routers per compute
        • 6.17.3.2.2.8. 30 instances and 30 routers per compute, QEMU paused
        • 6.17.3.2.2.9. 30 instances, 30 routers and 500 SG rules per instance, QEMU paused
        • 6.17.3.2.2.10. 1000 security group rules
        • 6.17.3.2.2.11. Conclusion
  • 6.17.4. OpenStack Neutron Control Plane Performance and Agent Restart Test Report
    • 6.17.4.1. Environment description
      • 6.17.4.1.1. Cluster description
      • 6.17.4.1.2. Software versions
      • 6.17.4.1.3. Hardware configuration of each server
    • 6.17.4.2. Reports
      • 6.17.4.2.1. Networks operations and L3-agent restart
        • 6.17.4.2.1.1. Summary
        • 6.17.4.2.1.2. Details
          • 6.17.4.2.1.2.1. Run #1
            • 6.17.4.2.1.2.1.1. Baseline
      • 6.17.4.2.2. Networks operations and OVS agent restart
        • 6.17.4.2.2.1. Summary
        • 6.17.4.2.2.2. Details
          • 6.17.4.2.2.2.1. Run #1
            • 6.17.4.2.2.2.1.1. Baseline
      • 6.17.4.2.3. Ports operations and L3-agent restart
        • 6.17.4.2.3.1. Summary
        • 6.17.4.2.3.2. Details
          • 6.17.4.2.3.2.1. Run #1
            • 6.17.4.2.3.2.1.1. Baseline
      • 6.17.4.2.4. Ports operations and OVS agent restart
        • 6.17.4.2.4.1. Summary
        • 6.17.4.2.4.2. Details
          • 6.17.4.2.4.2.1. Run #1
            • 6.17.4.2.4.2.1.1. Baseline
      • 6.17.4.2.5. Subnets operations and L3-agent restart
        • 6.17.4.2.5.1. Summary
        • 6.17.4.2.5.2. Details
          • 6.17.4.2.5.2.1. Run #1
            • 6.17.4.2.5.2.1.1. Baseline
      • 6.17.4.2.6. Subnets operations and OVS-agent restart
        • 6.17.4.2.6.1. Summary
        • 6.17.4.2.6.2. Details
          • 6.17.4.2.6.2.1. Run #1
            • 6.17.4.2.6.2.1.1. Baseline
  • 6.17.5. OpenStack Neutron Density Testing report
    • 6.17.5.1. Environment description
      • 6.17.5.1.1. Lab A (200 nodes)
        • 6.17.5.1.1.1. Hardware configuration of each server
    • 6.17.5.2. Test results
      • 6.17.5.2.1. Test Case: VM density check
      • 6.17.5.2.2. Observed issues
• 6.18. Fuel Containerized Control Plane performance report during upgrade
  • 6.18.1. Environment description
    • 6.18.1.1. Hardware configuration of each server
    • 6.18.1.2. Network configuration of each server
    • 6.18.1.3. Fuel CCP configuration
    • 6.18.1.4. Upgrade and tests procedure
  • 6.18.2. Test results
    • 6.18.2.1. Test Case 1: Analysis of L2 metrics during Open Stack upgrade
    • 6.18.2.2. Test Case 2: Analysis of L3 metrics during Open Stack upgrade
• 6.19. OpenStack load testing report
  • 6.19.1. Environment description
    • 6.19.1.1. Hardware configuration of each server
    • 6.19.1.2. Network configuration of each server
    • 6.19.1.3. Software configuration on environment servers
    • 6.19.1.4. Software configuration on servers with rally role
  • 6.19.2. Test results
    • 6.19.2.1. Create Neutron networks
    • 6.19.2.2. Create Neutron network with port
    • 6.19.2.3. Create and list Neutron ports
    • 6.19.2.4. Create Cinder volumes
    • 6.19.2.5. Boot Nova Server
• Provisioning systems test results
  • 1. Results of measuring performance of Cobbler
    • 1.1. Environment description
      • 1.1.1. Hardware configuration of each server
      • 1.1.2. Network scheme and part of configuration of hardware network switches
      • 1.1.3. Software configuration of Cobbler service
        • 1.1.3.1. Installation of Cobbler:
        • 1.1.3.2. Test tool:
        • 1.1.3.3. Operating system configuration:
      • 1.1.4. Software configuration of provisioned nodes
        • 1.1.4.1. Operating system:
    • 1.2. Testing process
      • 1.2.1. Preparation
      • 1.2.2. Testing. Case when pxlinux downloads kernel via HTTP
      • 1.2.3. Testing. Case when pxlinux downloads kernel via TFTP
    • 1.3. Results
      • 1.3.1. Case when pxlinux downloads kernel via HTTP
        • 1.3.1.1. CPU(TIME), RAM(TIME)
        • 1.3.1.2. NET(TIME), DISK(TIME)
        • 1.3.1.3. PROVISIONING_TIME(NODES)
        • 1.3.1.4. CPU(NODES), RAM(NODES)
        • 1.3.1.5. NET(NODES), DISK(NODES)
      • 1.3.2. Case when pxlinux downloads kernel via TFTP
        • 1.3.2.1. CPU(TIME), RAM(TIME)
        • 1.3.2.2. NET(TIME), DISK(TIME)
        • 1.3.2.3. PROVISIONING_TIME(NODES)
        • 1.3.2.4. CPU(NODES), RAM(NODES)
        • 1.3.2.5. NET(NODES), DISK(NODES)
    • 1.4. Issues which have been found during the tests
      • 1.4.1. Linux kernel can’t be downloaded by pxelinux via HTTP
    • 1.5. Applications
      • 1.5.1. Installation script and config files
      • 1.5.2. Script to start provisioning and collect metrics
      • 1.5.3. Script to add cobbler systems
• 6.20. OpenStack reliability testing
  • 6.20.1. Containerized OpenStack reliability testing
    • 6.20.1.1. Test results
      • 6.20.1.1.1. Environment description
        • 6.20.1.1.1.1. Software
        • 6.20.1.1.1.2. Configs
      • 6.20.1.1.2. Reports
        • 6.20.1.1.2.1. Keystone authentication with kill of Keystone on one node
          • 6.20.1.1.2.1.1. Summary
          • 6.20.1.1.2.1.2. Details
            • 6.20.1.1.2.1.2.1. Run #1
              • 6.20.1.1.2.1.2.1.1. Baseline
              • 6.20.1.1.2.1.2.1.2. Service downtime
              • 6.20.1.1.2.1.2.1.3. Service performance degradation
            • 6.20.1.1.2.1.2.2. Run #2
              • 6.20.1.1.2.1.2.2.1. Baseline
              • 6.20.1.1.2.1.2.2.2. Service downtime
              • 6.20.1.1.2.1.2.2.3. Service performance degradation
            • 6.20.1.1.2.1.2.3. Run #3
              • 6.20.1.1.2.1.2.3.1. Baseline
              • 6.20.1.1.2.1.2.3.2. Service downtime
              • 6.20.1.1.2.1.2.3.3. Service performance degradation
            • 6.20.1.1.2.1.2.4. Run #4
              • 6.20.1.1.2.1.2.4.1. Baseline
              • 6.20.1.1.2.1.2.4.2. Service downtime
              • 6.20.1.1.2.1.2.4.3. Service performance degradation
            • 6.20.1.1.2.1.2.5. Run #5
              • 6.20.1.1.2.1.2.5.1. Baseline
              • 6.20.1.1.2.1.2.5.2. Service downtime
              • 6.20.1.1.2.1.2.5.3. Service performance degradation
        • 6.20.1.1.2.2. Keystone authentication with kill of MySQL on one node
          • 6.20.1.1.2.2.1. Summary
          • 6.20.1.1.2.2.2. Details
            • 6.20.1.1.2.2.2.1. Run #1
              • 6.20.1.1.2.2.2.1.1. Baseline
        • 6.20.1.1.2.3. Keystone authentication with Keystone API restart on one node
          • 6.20.1.1.2.3.1. Summary
          • 6.20.1.1.2.3.2. Details
            • 6.20.1.1.2.3.2.1. Run #1
              • 6.20.1.1.2.3.2.1.1. Baseline
              • 6.20.1.1.2.3.2.1.2. Service downtime
              • 6.20.1.1.2.3.2.1.3. Service performance degradation
            • 6.20.1.1.2.3.2.2. Run #2
              • 6.20.1.1.2.3.2.2.1. Baseline
              • 6.20.1.1.2.3.2.2.2. Service downtime
              • 6.20.1.1.2.3.2.2.3. Service performance degradation
            • 6.20.1.1.2.3.2.3. Run #3
              • 6.20.1.1.2.3.2.3.1. Baseline
              • 6.20.1.1.2.3.2.3.2. Service downtime
              • 6.20.1.1.2.3.2.3.3. Service performance degradation
            • 6.20.1.1.2.3.2.4. Run #4
              • 6.20.1.1.2.3.2.4.1. Baseline
              • 6.20.1.1.2.3.2.4.2. Service downtime
              • 6.20.1.1.2.3.2.4.3. Service performance degradation
            • 6.20.1.1.2.3.2.5. Run #5
              • 6.20.1.1.2.3.2.5.1. Baseline
              • 6.20.1.1.2.3.2.5.2. Service downtime
              • 6.20.1.1.2.3.2.5.3. Service performance degradation
        • 6.20.1.1.2.4. Keystone authentication with memcached restart on one node
          • 6.20.1.1.2.4.1. Summary
          • 6.20.1.1.2.4.2. Details
            • 6.20.1.1.2.4.2.1. Run #1
              • 6.20.1.1.2.4.2.1.1. Baseline
              • 6.20.1.1.2.4.2.1.2. Service performance degradation
            • 6.20.1.1.2.4.2.2. Run #2
              • 6.20.1.1.2.4.2.2.1. Baseline
              • 6.20.1.1.2.4.2.2.2. Service performance degradation
            • 6.20.1.1.2.4.2.3. Run #3
              • 6.20.1.1.2.4.2.3.1. Baseline
              • 6.20.1.1.2.4.2.3.2. Service performance degradation
            • 6.20.1.1.2.4.2.4. Run #4
              • 6.20.1.1.2.4.2.4.1. Baseline
              • 6.20.1.1.2.4.2.4.2. Service performance degradation
            • 6.20.1.1.2.4.2.5. Run #5
              • 6.20.1.1.2.4.2.5.1. Baseline
              • 6.20.1.1.2.4.2.5.2. Service performance degradation
        • 6.20.1.1.2.5. Create and list networks with kill of one of MySQL servers
          • 6.20.1.1.2.5.1. Summary
          • 6.20.1.1.2.5.2. Details
            • 6.20.1.1.2.5.2.1. Run #1
              • 6.20.1.1.2.5.2.1.1. Baseline
              • 6.20.1.1.2.5.2.1.2. Service performance degradation
            • 6.20.1.1.2.5.2.2. Run #2
              • 6.20.1.1.2.5.2.2.1. Baseline
              • 6.20.1.1.2.5.2.2.2. Service performance degradation
            • 6.20.1.1.2.5.2.3. Run #3
              • 6.20.1.1.2.5.2.3.1. Baseline
              • 6.20.1.1.2.5.2.3.2. Service performance degradation
        • 6.20.1.1.2.6. Boot and delete VM with kill of RabbitMQ on one of nodes
          • 6.20.1.1.2.6.1. Summary
          • 6.20.1.1.2.6.2. Details
            • 6.20.1.1.2.6.2.1. Run #1
              • 6.20.1.1.2.6.2.1.1. Baseline
        • 6.20.1.1.2.7. Boot and delete VM with reboot of one of controllers
          • 6.20.1.1.2.7.1. Summary
          • 6.20.1.1.2.7.2. Details
            • 6.20.1.1.2.7.2.1. Run #1
              • 6.20.1.1.2.7.2.1.1. Baseline
              • 6.20.1.1.2.7.2.1.2. Service downtime
              • 6.20.1.1.2.7.2.1.3. Service performance degradation
  • 6.20.2. OpenStack reliability testing
    • 6.20.2.1. Test results
      • 6.20.2.1.1. Test environment
        • 6.20.2.1.1.1. Software configuration on servers with OpenStack
        • 6.20.2.1.1.2. Software configuration on servers with Rally role
          • 6.20.2.1.1.2.1. Environment description
        • 6.20.2.1.1.3. Hardware
        • 6.20.2.1.1.4. Software
        • 6.20.2.1.1.5. High availability cluster architecture
        • 6.20.2.1.1.6. Networking
      • 6.20.2.1.2. Factors description
    • 6.20.2.2. Testing process
      • 6.20.2.2.1. Test case 1: NovaServers.boot_and_delete_server
      • 6.20.2.2.2. Test case 1 results
        • 6.20.2.2.2.1. reboot-random-controller
        • 6.20.2.2.2.2. sigkill-random-rabbitmq
        • 6.20.2.2.2.3. sigkill-random-mysql
        • 6.20.2.2.2.4. freeze-random-nova-api
        • 6.20.2.2.2.5. freeze-random-memcached
        • 6.20.2.2.2.6. freeze-random-keystone
  • 6.20.3. OpenStack reliability testing. Version 2
    • 6.20.3.1. Test results
      • 6.20.3.1.1. Environment description
        • 6.20.3.1.1.1. Software
      • 6.20.3.1.2. Reports
        • 6.20.3.1.2.1. Keystone authentication with kill of Keystone on one node
          • 6.20.3.1.2.1.1. Summary
          • 6.20.3.1.2.1.2. Details
            • 6.20.3.1.2.1.2.1. Run #1
              • 6.20.3.1.2.1.2.1.1. Baseline
              • 6.20.3.1.2.1.2.1.2. Service downtime
              • 6.20.3.1.2.1.2.1.3. Service performance degradation
            • 6.20.3.1.2.1.2.2. Run #2
              • 6.20.3.1.2.1.2.2.1. Baseline
              • 6.20.3.1.2.1.2.2.2. Service downtime
              • 6.20.3.1.2.1.2.2.3. Service performance degradation
            • 6.20.3.1.2.1.2.3. Run #3
              • 6.20.3.1.2.1.2.3.1. Baseline
              • 6.20.3.1.2.1.2.3.2. Service downtime
              • 6.20.3.1.2.1.2.3.3. Service performance degradation
            • 6.20.3.1.2.1.2.4. Run #4
              • 6.20.3.1.2.1.2.4.1. Baseline
              • 6.20.3.1.2.1.2.4.2. Service downtime
              • 6.20.3.1.2.1.2.4.3. Service performance degradation
            • 6.20.3.1.2.1.2.5. Run #5
              • 6.20.3.1.2.1.2.5.1. Baseline
              • 6.20.3.1.2.1.2.5.2. Service downtime
              • 6.20.3.1.2.1.2.5.3. Service performance degradation
        • 6.20.3.1.2.2. Keystone authentication with kill of MySQL on one node
          • 6.20.3.1.2.2.1. Summary
          • 6.20.3.1.2.2.2. Details
            • 6.20.3.1.2.2.2.1. Run #1
              • 6.20.3.1.2.2.2.1.1. Baseline
              • 6.20.3.1.2.2.2.1.2. Service downtime
        • 6.20.3.1.2.3. Keystone authentication with Keystone API restart on one node
          • 6.20.3.1.2.3.1. Summary
          • 6.20.3.1.2.3.2. Details
            • 6.20.3.1.2.3.2.1. Run #1
              • 6.20.3.1.2.3.2.1.1. Baseline
              • 6.20.3.1.2.3.2.1.2. Service downtime
              • 6.20.3.1.2.3.2.1.3. Service performance degradation
            • 6.20.3.1.2.3.2.2. Run #2
              • 6.20.3.1.2.3.2.2.1. Baseline
              • 6.20.3.1.2.3.2.2.2. Service downtime
              • 6.20.3.1.2.3.2.2.3. Service performance degradation
            • 6.20.3.1.2.3.2.3. Run #3
              • 6.20.3.1.2.3.2.3.1. Baseline
              • 6.20.3.1.2.3.2.3.2. Service downtime
              • 6.20.3.1.2.3.2.3.3. Service performance degradation
            • 6.20.3.1.2.3.2.4. Run #4
              • 6.20.3.1.2.3.2.4.1. Baseline
              • 6.20.3.1.2.3.2.4.2. Service downtime
              • 6.20.3.1.2.3.2.4.3. Service performance degradation
            • 6.20.3.1.2.3.2.5. Run #5
              • 6.20.3.1.2.3.2.5.1. Baseline
              • 6.20.3.1.2.3.2.5.2. Service downtime
              • 6.20.3.1.2.3.2.5.3. Service performance degradation
        • 6.20.3.1.2.4. Keystone authentication with memcached restart on one node
          • 6.20.3.1.2.4.1. Summary
          • 6.20.3.1.2.4.2. Details
            • 6.20.3.1.2.4.2.1. Run #1
              • 6.20.3.1.2.4.2.1.1. Baseline
            • 6.20.3.1.2.4.2.2. Run #2
              • 6.20.3.1.2.4.2.2.1. Baseline
              • 6.20.3.1.2.4.2.2.2. Service performance degradation
            • 6.20.3.1.2.4.2.3. Run #3
              • 6.20.3.1.2.4.2.3.1. Baseline
            • 6.20.3.1.2.4.2.4. Run #4
              • 6.20.3.1.2.4.2.4.1. Baseline
            • 6.20.3.1.2.4.2.5. Run #5
              • 6.20.3.1.2.4.2.5.1. Baseline
              • 6.20.3.1.2.4.2.5.2. Service performance degradation
        • 6.20.3.1.2.5. Create and list networks with kill of one of MySQL servers
          • 6.20.3.1.2.5.1. Summary
          • 6.20.3.1.2.5.2. Details
            • 6.20.3.1.2.5.2.1. Run #1
              • 6.20.3.1.2.5.2.1.1. Baseline
            • 6.20.3.1.2.5.2.2. Run #2
              • 6.20.3.1.2.5.2.2.1. Baseline
              • 6.20.3.1.2.5.2.2.2. Service performance degradation
            • 6.20.3.1.2.5.2.3. Run #3
              • 6.20.3.1.2.5.2.3.1. Baseline
              • 6.20.3.1.2.5.2.3.2. Service performance degradation
        • 6.20.3.1.2.6. Boot and delete VM with disabling management network on one of controllers
          • 6.20.3.1.2.6.1. Summary
          • 6.20.3.1.2.6.2. Details
            • 6.20.3.1.2.6.2.1. Run #1
              • 6.20.3.1.2.6.2.1.1. Baseline
              • 6.20.3.1.2.6.2.1.2. Service downtime
              • 6.20.3.1.2.6.2.1.3. Service performance degradation
        • 6.20.3.1.2.7. Boot and delete VM with kill of RabbitMQ on one of nodes
          • 6.20.3.1.2.7.1. Summary
          • 6.20.3.1.2.7.2. Details
            • 6.20.3.1.2.7.2.1. Run #1
              • 6.20.3.1.2.7.2.1.1. Baseline
        • 6.20.3.1.2.8. Boot and delete VM with reboot of one of controllers
          • 6.20.3.1.2.8.1. Summary
          • 6.20.3.1.2.8.2. Details
            • 6.20.3.1.2.8.2.1. Run #1
              • 6.20.3.1.2.8.2.1.1. Baseline
              • 6.20.3.1.2.8.2.1.2. Service downtime
              • 6.20.3.1.2.8.2.1.3. Service performance degradation
• 6.21. Telemetry Services resource consumption/scalability test results on Gnocchi with CephStorage driver
  • 6.21.1. Test Environment
    • 6.21.1.1. Environment description
      • 6.21.1.1.1. Hardware
      • 6.21.1.1.2. Software
      • 6.21.1.1.3. Tuning/Configuration
      • 6.21.1.1.4. System Performance Monitoring
      • 6.21.1.1.5. Test Diagram
  • 6.21.2. Test Case 1
    • 6.21.2.1. Description
    • 6.21.2.2. Setup
    • 6.21.2.3. Analysis/Results
• 6.22. OpenStack tenant networking
  • 6.22.1. Tenant Networking: Neutron DVR + VXLAN at 200 nodes
    • 6.22.1.1. Environment description
      • 6.22.1.1.1. Software
    • 6.22.1.2. Reports
      • 6.22.1.2.1. OpenStack L2 Performance within single compute node
        • 6.22.1.2.1.1. Ping
        • 6.22.1.2.1.2. TCP
        • 6.22.1.2.1.3. UDP
      • 6.22.1.2.2. OpenStack L2 Performance 2 nodes
        • 6.22.1.2.2.1. Ping
        • 6.22.1.2.2.2. TCP
        • 6.22.1.2.2.3. UDP
      • 6.22.1.2.3. OpenStack L2 Full
        • 6.22.1.2.3.1. Bi-directional
          • 6.22.1.2.3.1.1. Concurrency 1
          • 6.22.1.2.3.1.2. Concurrency 3
          • 6.22.1.2.3.1.3. Concurrency 6
          • 6.22.1.2.3.1.4. Concurrency 12
          • 6.22.1.2.3.1.5. Concurrency 24
          • 6.22.1.2.3.1.6. Concurrency 48
          • 6.22.1.2.3.1.7. Concurrency 97
        • 6.22.1.2.3.2. Download
          • 6.22.1.2.3.2.1. Concurrency 1
          • 6.22.1.2.3.2.2. Concurrency 3
          • 6.22.1.2.3.2.3. Concurrency 6
          • 6.22.1.2.3.2.4. Concurrency 12
          • 6.22.1.2.3.2.5. Concurrency 24
          • 6.22.1.2.3.2.6. Concurrency 48
          • 6.22.1.2.3.2.7. Concurrency 97
        • 6.22.1.2.3.3. Upload
          • 6.22.1.2.3.3.1. Concurrency 1
          • 6.22.1.2.3.3.2. Concurrency 3
          • 6.22.1.2.3.3.3. Concurrency 6
          • 6.22.1.2.3.3.4. Concurrency 12
          • 6.22.1.2.3.3.5. Concurrency 24
          • 6.22.1.2.3.3.6. Concurrency 48
          • 6.22.1.2.3.3.7. Concurrency 97
      • 6.22.1.2.4. OpenStack L3 East-West Performance within single compute node
        • 6.22.1.2.4.1. Ping
        • 6.22.1.2.4.2. TCP
        • 6.22.1.2.4.3. UDP
      • 6.22.1.2.5. OpenStack L3 East-West Performance 2 nodes
        • 6.22.1.2.5.1. Ping
        • 6.22.1.2.5.2. TCP
        • 6.22.1.2.5.3. UDP
      • 6.22.1.2.6. OpenStack L3 East-West Full
        • 6.22.1.2.6.1. Bi-directional
          • 6.22.1.2.6.1.1. Concurrency 1
          • 6.22.1.2.6.1.2. Concurrency 3
          • 6.22.1.2.6.1.3. Concurrency 6
          • 6.22.1.2.6.1.4. Concurrency 12
          • 6.22.1.2.6.1.5. Concurrency 24
          • 6.22.1.2.6.1.6. Concurrency 48
          • 6.22.1.2.6.1.7. Concurrency 97
        • 6.22.1.2.6.2. Download
          • 6.22.1.2.6.2.1. Concurrency 1
          • 6.22.1.2.6.2.2. Concurrency 3
          • 6.22.1.2.6.2.3. Concurrency 6
          • 6.22.1.2.6.2.4. Concurrency 12
          • 6.22.1.2.6.2.5. Concurrency 24
          • 6.22.1.2.6.2.6. Concurrency 48
          • 6.22.1.2.6.2.7. Concurrency 97
        • 6.22.1.2.6.3. Upload
          • 6.22.1.2.6.3.1. Concurrency 1
          • 6.22.1.2.6.3.2. Concurrency 3
          • 6.22.1.2.6.3.3. Concurrency 6
          • 6.22.1.2.6.3.4. Concurrency 12
          • 6.22.1.2.6.3.5. Concurrency 24
          • 6.22.1.2.6.3.6. Concurrency 48
          • 6.22.1.2.6.3.7. Concurrency 97
      • 6.22.1.2.7. OpenStack L3 North-South Performance 2 nodes
        • 6.22.1.2.7.1. Ping
        • 6.22.1.2.7.2. TCP
        • 6.22.1.2.7.3. UDP
  • 6.22.2. Tenant Networking: Neutron DVR + VXLAN at 378 nodes
    • 6.22.2.1. Environment description
      • 6.22.2.1.1. Software
    • 6.22.2.2. Reports
      • 6.22.2.2.1. OpenStack L2 Performance within single compute node
        • 6.22.2.2.1.1. Bi-directional
          • 6.22.2.2.1.1.1. Concurrency 1
          • 6.22.2.2.1.1.2. Concurrency 2
          • 6.22.2.2.1.1.3. Concurrency 3
          • 6.22.2.2.1.1.4. Concurrency 4
          • 6.22.2.2.1.1.5. Concurrency 5
          • 6.22.2.2.1.1.6. Concurrency 6
          • 6.22.2.2.1.1.7. Concurrency 7
          • 6.22.2.2.1.1.8. Concurrency 8
        • 6.22.2.2.1.2. Download
          • 6.22.2.2.1.2.1. Concurrency 1
          • 6.22.2.2.1.2.2. Concurrency 2
          • 6.22.2.2.1.2.3. Concurrency 3
          • 6.22.2.2.1.2.4. Concurrency 4
          • 6.22.2.2.1.2.5. Concurrency 5
          • 6.22.2.2.1.2.6. Concurrency 6
          • 6.22.2.2.1.2.7. Concurrency 7
          • 6.22.2.2.1.2.8. Concurrency 8
        • 6.22.2.2.1.3. Upload
          • 6.22.2.2.1.3.1. Concurrency 1
          • 6.22.2.2.1.3.2. Concurrency 2
          • 6.22.2.2.1.3.3. Concurrency 3
          • 6.22.2.2.1.3.4. Concurrency 4
          • 6.22.2.2.1.3.5. Concurrency 5
          • 6.22.2.2.1.3.6. Concurrency 6
          • 6.22.2.2.1.3.7. Concurrency 7
          • 6.22.2.2.1.3.8. Concurrency 8
      • 6.22.2.2.2. OpenStack L3 East-West Performance within single compute node
        • 6.22.2.2.2.1. Bi-directional
          • 6.22.2.2.2.1.1. Concurrency 1
          • 6.22.2.2.2.1.2. Concurrency 2
          • 6.22.2.2.2.1.3. Concurrency 3
          • 6.22.2.2.2.1.4. Concurrency 4
          • 6.22.2.2.2.1.5. Concurrency 5
          • 6.22.2.2.2.1.6. Concurrency 6
          • 6.22.2.2.2.1.7. Concurrency 7
          • 6.22.2.2.2.1.8. Concurrency 8
        • 6.22.2.2.2.2. Download
          • 6.22.2.2.2.2.1. Concurrency 1
          • 6.22.2.2.2.2.2. Concurrency 2
          • 6.22.2.2.2.2.3. Concurrency 3
          • 6.22.2.2.2.2.4. Concurrency 4
          • 6.22.2.2.2.2.5. Concurrency 5
          • 6.22.2.2.2.2.6. Concurrency 6
          • 6.22.2.2.2.2.7. Concurrency 7
          • 6.22.2.2.2.2.8. Concurrency 8
        • 6.22.2.2.2.3. Upload
          • 6.22.2.2.2.3.1. Concurrency 1
          • 6.22.2.2.2.3.2. Concurrency 2
          • 6.22.2.2.2.3.3. Concurrency 3
          • 6.22.2.2.2.3.4. Concurrency 4
          • 6.22.2.2.2.3.5. Concurrency 5
          • 6.22.2.2.2.3.6. Concurrency 6
          • 6.22.2.2.2.3.7. Concurrency 7
          • 6.22.2.2.2.3.8. Concurrency 8
      • 6.22.2.2.3. OpenStack L3 North-South Performance within single compute node
        • 6.22.2.2.3.1. Bi-directional
          • 6.22.2.2.3.1.1. Concurrency 1
          • 6.22.2.2.3.1.2. Concurrency 2
          • 6.22.2.2.3.1.3. Concurrency 3
          • 6.22.2.2.3.1.4. Concurrency 4
          • 6.22.2.2.3.1.5. Concurrency 5
          • 6.22.2.2.3.1.6. Concurrency 6
          • 6.22.2.2.3.1.7. Concurrency 7
          • 6.22.2.2.3.1.8. Concurrency 8
        • 6.22.2.2.3.2. Download
          • 6.22.2.2.3.2.1. Concurrency 1
          • 6.22.2.2.3.2.2. Concurrency 2
          • 6.22.2.2.3.2.3. Concurrency 3
          • 6.22.2.2.3.2.4. Concurrency 4
          • 6.22.2.2.3.2.5. Concurrency 5
          • 6.22.2.2.3.2.6. Concurrency 6
          • 6.22.2.2.3.2.7. Concurrency 7
          • 6.22.2.2.3.2.8. Concurrency 8
        • 6.22.2.2.3.3. Upload
          • 6.22.2.2.3.3.1. Concurrency 1
          • 6.22.2.2.3.3.2. Concurrency 2
          • 6.22.2.2.3.3.3. Concurrency 3
          • 6.22.2.2.3.3.4. Concurrency 4
          • 6.22.2.2.3.3.5. Concurrency 5
          • 6.22.2.2.3.3.6. Concurrency 6
          • 6.22.2.2.3.3.7. Concurrency 7
          • 6.22.2.2.3.3.8. Concurrency 8
      • 6.22.2.2.4. OpenStack L2 Full
        • 6.22.2.2.4.1. Bi-directional
          • 6.22.2.2.4.1.1. Concurrency 1
          • 6.22.2.2.4.1.2. Concurrency 2
          • 6.22.2.2.4.1.3. Concurrency 5
          • 6.22.2.2.4.1.4. Concurrency 11
          • 6.22.2.2.4.1.5. Concurrency 23
          • 6.22.2.2.4.1.6. Concurrency 46
          • 6.22.2.2.4.1.7. Concurrency 92
          • 6.22.2.2.4.1.8. Concurrency 185
        • 6.22.2.2.4.2. Download
          • 6.22.2.2.4.2.1. Concurrency 1
          • 6.22.2.2.4.2.2. Concurrency 2
          • 6.22.2.2.4.2.3. Concurrency 5
          • 6.22.2.2.4.2.4. Concurrency 11
          • 6.22.2.2.4.2.5. Concurrency 23
          • 6.22.2.2.4.2.6. Concurrency 46
          • 6.22.2.2.4.2.7. Concurrency 92
          • 6.22.2.2.4.2.8. Concurrency 185
        • 6.22.2.2.4.3. Upload
          • 6.22.2.2.4.3.1. Concurrency 1
          • 6.22.2.2.4.3.2. Concurrency 2
          • 6.22.2.2.4.3.3. Concurrency 5
          • 6.22.2.2.4.3.4. Concurrency 11
          • 6.22.2.2.4.3.5. Concurrency 23
          • 6.22.2.2.4.3.6. Concurrency 46
          • 6.22.2.2.4.3.7. Concurrency 92
          • 6.22.2.2.4.3.8. Concurrency 185
      • 6.22.2.2.5. OpenStack L3 East-West Full
        • 6.22.2.2.5.1. Bi-directional
          • 6.22.2.2.5.1.1. Concurrency 1
          • 6.22.2.2.5.1.2. Concurrency 2
          • 6.22.2.2.5.1.3. Concurrency 5
          • 6.22.2.2.5.1.4. Concurrency 11
          • 6.22.2.2.5.1.5. Concurrency 23
          • 6.22.2.2.5.1.6. Concurrency 46
          • 6.22.2.2.5.1.7. Concurrency 92
          • 6.22.2.2.5.1.8. Concurrency 185
        • 6.22.2.2.5.2. Download
          • 6.22.2.2.5.2.1. Concurrency 1
          • 6.22.2.2.5.2.2. Concurrency 2
          • 6.22.2.2.5.2.3. Concurrency 5
          • 6.22.2.2.5.2.4. Concurrency 11
          • 6.22.2.2.5.2.5. Concurrency 23
          • 6.22.2.2.5.2.6. Concurrency 46
          • 6.22.2.2.5.2.7. Concurrency 92
          • 6.22.2.2.5.2.8. Concurrency 185
        • 6.22.2.2.5.3. Upload
          • 6.22.2.2.5.3.1. Concurrency 1
          • 6.22.2.2.5.3.2. Concurrency 2
          • 6.22.2.2.5.3.3. Concurrency 5
          • 6.22.2.2.5.3.4. Concurrency 11
          • 6.22.2.2.5.3.5. Concurrency 23
          • 6.22.2.2.5.3.6. Concurrency 46
          • 6.22.2.2.5.3.7. Concurrency 92
          • 6.22.2.2.5.3.8. Concurrency 185
      • 6.22.2.2.6. OpenStack L3 North-South Full
        • 6.22.2.2.6.1. Bi-directional
          • 6.22.2.2.6.1.1. Concurrency 1
          • 6.22.2.2.6.1.2. Concurrency 2
          • 6.22.2.2.6.1.3. Concurrency 5
          • 6.22.2.2.6.1.4. Concurrency 11
          • 6.22.2.2.6.1.5. Concurrency 23
          • 6.22.2.2.6.1.6. Concurrency 46
          • 6.22.2.2.6.1.7. Concurrency 92
          • 6.22.2.2.6.1.8. Concurrency 185
        • 6.22.2.2.6.2. Download
          • 6.22.2.2.6.2.1. Concurrency 1
          • 6.22.2.2.6.2.2. Concurrency 2
          • 6.22.2.2.6.2.3. Concurrency 5
          • 6.22.2.2.6.2.4. Concurrency 11
          • 6.22.2.2.6.2.5. Concurrency 23
          • 6.22.2.2.6.2.6. Concurrency 46
          • 6.22.2.2.6.2.7. Concurrency 92
          • 6.22.2.2.6.2.8. Concurrency 185
        • 6.22.2.2.6.3. Upload
          • 6.22.2.2.6.3.1. Concurrency 1
          • 6.22.2.2.6.3.2. Concurrency 2
          • 6.22.2.2.6.3.3. Concurrency 5
          • 6.22.2.2.6.3.4. Concurrency 11
          • 6.22.2.2.6.3.5. Concurrency 23
          • 6.22.2.2.6.3.6. Concurrency 46
          • 6.22.2.2.6.3.7. Concurrency 92
          • 6.22.2.2.6.3.8. Concurrency 185
      • 6.22.2.2.7. OpenStack L2 Performance 2 nodes
        • 6.22.2.2.7.1. Ping
        • 6.22.2.2.7.2. TCP
        • 6.22.2.2.7.3. UDP
      • 6.22.2.2.8. OpenStack L3 East-West Performance 2 nodes
        • 6.22.2.2.8.1. Ping
        • 6.22.2.2.8.2. TCP
        • 6.22.2.2.8.3. UDP