6.3.2. Results of measuring performance of JFrog Artifactory Pro¶
| Abstract: | This document includes performance test results of JFrog Artifactory Pro service as a repository of docker images. All test have been performed regarding Measuring performance of container repositories |
|---|
6.3.2.1. Environment description¶
6.3.2.1.1. Hardware configuration of each server¶
| server | name | 728998-comp-disk-228 | 728998-comp-disk-227 |
| role | test_tool | artifactory | |
| vendor,model | HP,DL380 Gen9 | HP,DL380 Gen9 | |
| operating_system | 3.13.0-76-generic
Ubuntu-trusty
x86_64
|
3.13.0-76-generic
Ubuntu-trusty
x86_64
|
|
| CPU | vendor,model | Intel,E5-2680 v3 | Intel,E5-2680 v3 |
| processor_count | 2 | 2 | |
| core_count | 12 | 12 | |
| frequency_MHz | 2500 | 2500 | |
| RAM | vendor,model | HP,752369-081 | HP,752369-081 |
| amount_MB | 262144 | 262144 | |
| NETWORK | interface_name | p1p1 | p1p1 |
| vendor,model | Intel,X710 Dual Port | Intel,X710 Dual Port | |
| bandwidth | 10G | 10G | |
| STORAGE | dev_name | /dev/sda | /dev/sda |
| vendor,model | raid10 - HP P840
12 disks EH0600JEDHE
|
raid10 - HP P840
12 disks EH0600JEDHE
|
|
| SSD/HDD | HDD | HDD | |
| size | 3,6TB | 3,6TB |
6.3.2.1.2. Network scheme and part of configuration of hardware network switches¶
Network scheme of the environment:
Here is the part of switch configuration for each switch port which connected to p1p1 interface of a server:
switchport mode trunk
switchport trunk native vlan 600
switchport trunk allowed vlan 600-602,630-649
spanning-tree port type edge trunk
spanning-tree bpduguard enable
no snmp trap link-status
6.3.2.1.3. Software configuration of the Artifactory service¶
6.3.2.1.3.1. Installation of Artifactory:¶
# Install Java
apt-get install software-properties-common
sudo add-apt-repository ppa:webupd8team/java
apt-get update
sudo apt-get install oracle-java8-installer # Interactive java installer
# Install ArtifactoryPro
echo "deb https://jfrog.bintray.com/artifactory-pro-debs trusty main" > /etc/apt/sources.list.d/artifactory.list
apt-get update
apt-get install jfrog-artifactory-pro
# Install Nginx as a reverse proxy
# https://www.jfrog.com/confluence/display/RTF/Configuring+NGINX
apt-get install nginx
You can find nginx configuration file here
nginx.conf
Here is the all config files of Artifactory
artifactory_etc.tar.gz
After installation all default repositories was removed from Artifactory and
“docker-local” hosted repository was created. You can find full configuration of
the Artifactory installation here:
config_descriptor.xml
security_descriptor.xml
| Software | Version |
|---|---|
| Ubuntu | Ubuntu 14.04.3 LTS |
| Artifactory | 4.5.2 rev 40121 |
6.3.2.1.3.2. Operating system configuration:¶
You can find outputs of some commands and /etc folder in the following archive:
server_description_of_728998-comp-disk-227.tar.gz
6.3.2.1.4. Software configuration of the node with test tool¶
6.3.2.1.4.1. Test tool:¶
Firstly we need to install docker-engine:
echo "deb https://apt.dockerproject.org/repo ubuntu-trusty main" > /etc/apt/sources.list.d/docker.list
apt-get update && apt-get -y install docker-engine
echo DOCKER_OPTS=\"--insecure-registry 172.20.9.16:5000\" >> /etc/default/docker
service docker restart
We use Python2.7 and Script for collecting performance metrics with Proposed docker file to perform the tests. The image size is a sum of layers:
IMAGE CREATED CREATED BY SIZE COMMENT
93333b8ed564 About a minute ago /bin/sh -c #(nop) CMD ["/bin/sh" "-c" "/usr/s 0 B
35d8142196c0 About a minute ago /bin/sh -c #(nop) EXPOSE 80/tcp 0 B
3a63f30ab247 About a minute ago /bin/sh -c apt-get install -y nginx 18.14 MB
97434d46f197 2 days ago /bin/sh -c #(nop) CMD ["/bin/bash"] 0 B
<missing> 2 days ago /bin/sh -c sed -i 's/^#\s*\(deb.*universe\)$/ 1.895 kB
<missing> 2 days ago /bin/sh -c set -xe && echo '#!/bin/sh' > /u 194.5 kB
<missing> 2 days ago /bin/sh -c #(nop) ADD file:e01d51d39ea04c8efb 187.8 MB
It means that DATA_SIZE=206.13 MB
| Software | Version |
|---|---|
| Ubuntu | Ubuntu 14.04.3 LTS |
| Docker | 1.10 |
6.3.2.1.4.2. Operating system:¶
You can find outputs of some commands and /etc folder in the following archive:
server_description_of_728997-comp-disk-228.tar.gz
6.3.2.2. Testing process¶
Artifactory was installed on top of 728998-comp-disk-227 server as described in Installation of Artifactory: section.
The values of the variables in test-repo.py script was changed: iterations = 1000 concurrency = 1 repo_address = “172.20.9.16:5000”
The following command was executed to perform the tests:
sudo python test-repo.py
push_results.csv and pull_results.csv was saved in persistent folder.
Clean up docker repository in Artifactory via Web UI
The steps from 2 to 5 was repeated with the following values of the concurrency parameters: 1,10,30,50,100
As a result of this part we got the following CSV files:
PUSH_TIME(CONCURRENCY=1)
PUSH_TIME(CONCURRENCY=10)
PUSH_TIME(CONCURRENCY=30)
PUSH_TIME(CONCURRENCY=50)
PUSH_TIME(CONCURRENCY=100)
PULL_TIME(CONCURRENCY=1)
PULL_TIME(CONCURRENCY=10)
PULL_TIME(CONCURRENCY=30)
PULL_TIME(CONCURRENCY=50)
PULL_TIME(CONCURRENCY=100)
6.3.2.3. Results¶
6.3.2.3.1. Push action results¶
6.3.2.3.1.1. PUSH_TIME(ITERATION)¶
After simple processing results the following plots for push action in depend on iteration number created (click to expand an image):
| Function | Plot |
|---|---|
| PUSH_TIME (ITERATION, CONCURRENCY=1) | 
|
| PUSH_TIME (ITERATION, CONCURRENCY=10) | 
|
| PUSH_TIME (ITERATION, CONCURRENCY=30) | 
|
| PUSH_TIME (ITERATION, CONCURRENCY=50) | 
|
| PUSH_TIME (ITERATION, CONCURRENCY=100) | 
|
| PUSH_TIME (ITERATION, CONCURRENCY={1, 10,30,50,100} | 
|
6.3.2.3.1.2. PUSH_TIME(CONCURRENCY)¶
The following table and graph show how PUSH_TIME parameter depend on CONCURRENCY parameter.
| Concurrency | Maximum | Minimum | Average | Percentile 90% |
|---|---|---|---|---|
| 1 | 20.45173717 | 18.13562512 | 18.45595757 | 18.70360937 |
| 10 | 52.81749606 | 21.4737699 | 47.39192888 | 47.92230422 |
| 30 | 147.2520959 | 21.35128784 | 140.8133277 | 143.3592351 |
| 50 | 242.0436599 | 21.53936005 | 232.1077632 | 238.6817375 |
| 100 | 474.1983588 | 21.41779709 | 447.7350837 | 471.5060248 |
6.3.2.3.2. Pull action results¶
6.3.2.3.2.1. PULL_TIME(ITERATION)¶
After simple processing results the following plots for pull action in depend on iteration number created (click to expand an image):
| Function | Plot |
|---|---|
| PULL_TIME (ITERATION, CONCURRENCY=1) | 
|
| PULL_TIME (ITERATION, CONCURRENCY=10) | 
|
| PULL_TIME (ITERATION, CONCURRENCY=30) | 
|
| PULL_TIME (ITERATION, CONCURRENCY=50) | 
|
| PULL_TIME (ITERATION, CONCURRENCY=100) | 
|
| PULL_TIME (ITERATION, CONCURRENCY={1, 10,30,50,100} | 
|
6.3.2.3.2.2. PULL_TIME(CONCURRENCY)¶
The following table and graph show how PUSH_TIME parameter depend on CONCURRENCY parameter.
| Concurrency | Maximum | Minimum | Average | Percentile 90% |
|---|---|---|---|---|
| 1 | 1.023168087 | 0.09103393555 | 0.7067174957 | 0.7760614157 |
| 10 | 1.87360692 | 0.08845591545 | 1.127708118 | 1.361271906 |
| 30 | 4.27976203 | 0.1920921803 | 2.644113734 | 2.870757174 |
| 50 | 6.287320852 | 0.2292528152 | 4.416184656 | 4.711911416 |
| 100 | 10.30138803 | 0.167137146 | 8.50843117 | 9.09850688 |
6.3.2.4. Issues which have been found during the tests¶
| Issue description | Root cause, Link |
|---|---|
Artifactory allows re-pushing
existed docker layers instead
of answer with
“already exists” message
|
https://www.jfrog.com/jira/browse/RTFACT-9591 |
only 5 images can be uploaded
to Artifactory at the
same time
|
Root cause of the issue haven’t found yet
|