2. Results of measuring performance of JFrog Artifactory Pro¶

Abstract: This document includes performance test results of JFrog Artifactory Pro service as a repository of docker images. All test have been performed regarding Measuring performance of container repositories

2.1. Environment description¶

2.1.1. Hardware configuration of each server¶

Description of servers hardware¶
server	name	728998-comp-disk-228	728998-comp-disk-227
	role	test_tool	artifactory
	vendor,model	HP,DL380 Gen9	HP,DL380 Gen9
	operating_system	3.13.0-76-generic Ubuntu-trusty x86_64	3.13.0-76-generic Ubuntu-trusty x86_64
CPU	vendor,model	Intel,E5-2680 v3	Intel,E5-2680 v3
	processor_count	2	2
	core_count	12	12
	frequency_MHz	2500	2500
RAM	vendor,model	HP,752369-081	HP,752369-081
RAM	amount_MB	262144	262144
NETWORK	interface_name	p1p1	p1p1
	vendor,model	Intel,X710 Dual Port	Intel,X710 Dual Port
	bandwidth	10G	10G
STORAGE	dev_name	/dev/sda	/dev/sda
	vendor,model	raid10 - HP P840 12 disks EH0600JEDHE	raid10 - HP P840 12 disks EH0600JEDHE
	SSD/HDD	HDD	HDD
	size	3,6TB	3,6TB

2.1.2. Network scheme and part of configuration of hardware network switches¶

Network scheme of the environment:

Here is the part of switch configuration for each switch port which connected to p1p1 interface of a server:

switchport mode trunk
switchport trunk native vlan 600
switchport trunk allowed vlan 600-602,630-649
spanning-tree port type edge trunk
spanning-tree bpduguard enable
no snmp trap link-status

2.1.3. Software configuration of the Artifactory service¶

2.1.3.1. Installation of Artifactory:¶

# Install Java
apt-get install software-properties-common
sudo add-apt-repository ppa:webupd8team/java
apt-get update
sudo apt-get install oracle-java8-installer # Interactive java installer
# Install ArtifactoryPro
echo "deb https://jfrog.bintray.com/artifactory-pro-debs trusty main" > /etc/apt/sources.list.d/artifactory.list
apt-get update
apt-get install jfrog-artifactory-pro
# Install Nginx as a reverse proxy
# https://www.jfrog.com/confluence/display/RTF/Configuring+NGINX
apt-get install nginx

You can find nginx configuration file here nginx.conf

Here is the all config files of Artifactory artifactory_etc.tar.gz

After installation all default repositories was removed from Artifactory and “docker-local” hosted repository was created. You can find full configuration of the Artifactory installation here: config_descriptor.xml security_descriptor.xml

Versions of some software¶
Software	Version
Ubuntu	Ubuntu 14.04.3 LTS
Artifactory	4.5.2 rev 40121

2.1.3.2. Operating system configuration:¶

You can find outputs of some commands and /etc folder in the following archive: server_description_of_728998-comp-disk-227.tar.gz

2.1.4. Software configuration of the node with test tool¶

2.1.4.1. Test tool:¶

Firstly we need to install docker-engine:

echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" > /etc/apt/sources.list.d/docker.list
apt-get update && apt-get -y install docker-engine
echo DOCKER_OPTS=\"--insecure-registry 172.20.9.16:5000\" >> /etc/default/docker
service docker restart

We use Python2.7 and Script for collecting performance metrics with Proposed docker file to perform the tests. The image size is a sum of layers:

IMAGE               CREATED              CREATED BY                                      SIZE                COMMENT
93333b8ed564        About a minute ago   /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/usr/s   0 B
35d8142196c0        About a minute ago   /bin/sh -c #(nop) EXPOSE 80/tcp                 0 B
3a63f30ab247        About a minute ago   /bin/sh -c apt-get install -y nginx             18.14 MB
97434d46f197        2 days ago           /bin/sh -c #(nop) CMD ["/bin/bash"]             0 B
<missing>           2 days ago           /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/   1.895 kB
<missing>           2 days ago           /bin/sh -c set -xe   && echo '#!/bin/sh' > /u   194.5 kB
<missing>           2 days ago           /bin/sh -c #(nop) ADD file:e01d51d39ea04c8efb   187.8 MB

It means that DATA_SIZE=206.13 MB

Versions of some software¶
Software	Version
Ubuntu	Ubuntu 14.04.3 LTS
Docker	1.10

2.1.4.2. Operating system:¶

You can find outputs of some commands and /etc folder in the following archive: server_description_of_728997-comp-disk-228.tar.gz

2.2. Testing process¶

Artifactory was installed on top of 728998-comp-disk-227 server as described in Installation of Artifactory: section.
The values of the variables in test-repo.py script was changed: iterations = 1000 concurrency = 1 repo_address = “172.20.9.16:5000”
The following command was executed to perform the tests:
```
sudo python test-repo.py
```
push_results.csv and pull_results.csv was saved in persistent folder.
Clean up docker repository in Artifactory via Web UI
The steps from 2 to 5 was repeated with the following values of the concurrency parameters: 1,10,30,50,100

As a result of this part we got the following CSV files:

PUSH_TIME(CONCURRENCY=1) PUSH_TIME(CONCURRENCY=10) PUSH_TIME(CONCURRENCY=30) PUSH_TIME(CONCURRENCY=50) PUSH_TIME(CONCURRENCY=100) PULL_TIME(CONCURRENCY=1) PULL_TIME(CONCURRENCY=10) PULL_TIME(CONCURRENCY=30) PULL_TIME(CONCURRENCY=50) PULL_TIME(CONCURRENCY=100)

2.3. Results¶

2.3.1. Push action results¶

2.3.1.1. PUSH_TIME(ITERATION)¶

After simple processing results the following plots for push action in depend on iteration number created (click to expand an image):

Function	Plot
PUSH_TIME (ITERATION, CONCURRENCY=1)
PUSH_TIME (ITERATION, CONCURRENCY=10)
PUSH_TIME (ITERATION, CONCURRENCY=30)
PUSH_TIME (ITERATION, CONCURRENCY=50)
PUSH_TIME (ITERATION, CONCURRENCY=100)
PUSH_TIME (ITERATION, CONCURRENCY={1, 10,30,50,100}	$PUSH_TIME(ITERATION, CONCURRENCY={1,10,30,50,100}$

2.3.1.2. PUSH_TIME(CONCURRENCY)¶

The following table and graph show how PUSH_TIME parameter depend on CONCURRENCY parameter.

Maximum, Minimum, Average and Percentile 90% of PUSH_TIME values in depend on CONCURRENCY parameter.¶
Concurrency	Maximum	Minimum	Average	Percentile 90%

1	20.45173717	18.13562512	18.45595757	18.70360937
10	52.81749606	21.4737699	47.39192888	47.92230422
30	147.2520959	21.35128784	140.8133277	143.3592351
50	242.0436599	21.53936005	232.1077632	238.6817375
100	474.1983588	21.41779709	447.7350837	471.5060248

2.3.2. Pull action results¶

2.3.2.1. PULL_TIME(ITERATION)¶

After simple processing results the following plots for pull action in depend on iteration number created (click to expand an image):

Function	Plot
PULL_TIME (ITERATION, CONCURRENCY=1)
PULL_TIME (ITERATION, CONCURRENCY=10)
PULL_TIME (ITERATION, CONCURRENCY=30)
PULL_TIME (ITERATION, CONCURRENCY=50)
PULL_TIME (ITERATION, CONCURRENCY=100)
PULL_TIME (ITERATION, CONCURRENCY={1, 10,30,50,100}	$PULL_TIME(ITERATION, CONCURRENCY={1,10,30,50,100}$

2.3.2.2. PULL_TIME(CONCURRENCY)¶