6.2. Ceph RBD performance report

Abstract

This document includes Ceph RBD performance test results for 40 OSD nodes. Test cluster contain 40 OSD servers and forms 581TiB ceph cluster.

6.2.1. Environment description

Environment contains 3 types of servers:

• ceph-mon node

• ceph-osd node

• compute node

Amount of servers each role

Role	Servers count	Type
compute	1	1
ceph-mon	3	1
ceph-osd	40	2

6.2.1.1. Hardware configuration of each server

All servers have 2 types of configuration describing in table below

Description of servers hardware type 1

server	vendor,model	Dell PowerEdge R630
CPU	vendor,model	Intel,E5-2680 v3
	processor_count	2
	core_count	12
	frequency_MHz	2500
RAM	vendor,model	Samsung, M393A2G40DB0-CPB
	amount_MB	262144
NETWORK	interface_name s	eno1, eno2
	vendor,model	Intel,X710 Dual Port
	bandwidth	10G
	interface_names	enp3s0f0, enp3s0f1
	vendor,model	Intel,X710 Dual Port
	bandwidth	10G
STORAGE	dev_name	/dev/sda
	vendor,model	raid1 - Dell, PERC H730P Mini 2 disks Intel S3610
	SSD/HDD	SSD
	size	3,6TB

Description of servers hardware type 2

server	vendor,model	Lenovo ThinkServer RD650
CPU	vendor,model	Intel,E5-2670 v3
	processor_count	2
	core_count	12
	frequency_MHz	2500
RAM	vendor,model	Samsung, M393A2G40DB0-CPB
	amount_MB	131916
NETWORK	interface_names	enp3s0f0, enp3s0f1
	vendor,model	Intel,X710 Dual Port
	bandwidth	10G
	interface_names	ens2f0, ens2f1
	vendor,model	Intel,X710 Dual Port
	bandwidth	10G
STORAGE	vendor,model	2 disks Intel S3610
	SSD/HDD	SSD
	size	799GB
	vendor,model	10 disks 2T
	SSD/HDD	HDD
	size	2TB

6.2.1.2. Network configuration of each server

All servers have same network configuration:

6.2.1.3. Software configuration on servers with controller, compute and compute-osd roles

Ceph was deployed by Decapod tool. Cluster config for decapod: ceph_config.yaml

Software version on servers

Software	Version
Ceph	Jewel
Ubuntu	Ubuntu 16.04 LTS

You can find outputs of some commands and /etc folder in the following archives:

ceph-osd-1.tar.gz

ceph-mon-1.tar.gz

compute-1.tar.gz

6.2.2. Testing process

1. Run virtual machine on compute node with attached RBD disk.

2. SSH into VM operation system

3. Clone Wally repository.

4. Create ceph_raw.yaml file in cloned repository

5. Run command python -m wally test ceph_rbd_2 ./ceph_raw.yaml

As a result we got the following HTML file:

Report.html

6.2.3. Test results