For the most part we try to follow PEP 8 guidelines which can be viewed here: http://www.python.org/dev/peps/pep-0008/
Swift has a comprehensive suite of tests and pep8 checks that are run on all submitted code, and it is recommended that developers execute the tests themselves to catch regressions early. Developers are also expected to keep the test suite up-to-date with any submitted code changes.
Swift’s tests and pep8 checks can be executed in an isolated environment
with tox
: http://tox.testrun.org/
To execute the tests:
Ensure pip
and virtualenv
are upgraded to satisfy the version
requirements listed in the OpenStack global requirements:
pip install pip -U
pip install virtualenv -U
Install tox
:
pip install tox
Generate list of distribution packages to install for testing:
tox -e bindep
Now install these packages using your distribution package manager like apt-get, dnf, yum, or zypper.
Run tox
from the root of the swift repo:
tox
Note
If you installed using cd ~/swift; sudo python setup.py develop
, you may
need to do cd ~/swift; sudo chown -R ${USER}:${USER} swift.egg-info
prior
to running tox
.
By default tox
will run all of the unit test and pep8 checks listed in
the tox.ini
file envlist
option. A subset of the test environments
can be specified on the tox
command line or by setting the TOXENV
environment variable. For example, to run only the pep8 checks and python2.7
unit tests use:
tox -e pep8,py27
or:
TOXENV=py27,pep8 tox
Note
As of tox
version 2.0.0, most environment variables are not automatically
passed to the test environment. Swift’s tox.ini
overrides this default
behavior so that variable names matching SWIFT_*
and *_proxy
will be
passed, but you may need to run tox --recreate
for this to take effect
after upgrading from tox
<2.0.0.
Conversely, if you do not want those environment variables to be passed to
the test environment then you will need to unset them before calling tox
.
Also, if you ever encounter DistributionNotFound, try to use tox
--recreate
or remove the .tox
directory to force tox
to recreate the
dependency list.
Swift’s tests require having an XFS directory available in /tmp
or
in the TMPDIR
environment variable.
Swift’s functional tests may be executed against a SAIO - Swift All In One or other running Swift cluster using the command:
tox -e func
The endpoint and authorization credentials to be used by functional tests
should be configured in the test.conf
file as described in the section
Setting up scripts for running Swift.
The environment variable SWIFT_TEST_POLICY
may be set to specify a
particular storage policy name that will be used for testing. When set, tests
that would otherwise not specify a policy or choose a random policy from
those available will instead use the policy specified. Tests that use more than
one policy will include the specified policy in the set of policies used. The
specified policy must be available on the cluster under test.
For example, this command would run the functional tests using policy ‘silver’:
SWIFT_TEST_POLICY=silver tox -e func
To run a single functional test, use the --no-discover
option together with
a path to a specific test method, for example:
tox -e func -- --no-discover test.functional.tests.TestFile.testCopy
If the test.conf
file is not found then the functional test framework will
instantiate a set of Swift servers in the same process that executes the
functional tests. This ‘in-process test’ mode may also be enabled (or disabled)
by setting the environment variable SWIFT_TEST_IN_PROCESS
to a true (or
false) value prior to executing tox -e func
.
When using the ‘in-process test’ mode some server configuration options may be set using environment variables:
SWIFT_TEST_IN_MEMORY_OBJ
to a true value.SWIFT_TEST_IN_PROCESS_CONF_LOADER
to
encryption
.SWIFT_TEST_IN_PROCESS_CONF_LOADER
to
ec
.SWIFT_TEST_DEBUG_LOGS
.For example, this command would run the in-process mode functional tests with encryption enabled in the proxy-server:
SWIFT_TEST_IN_PROCESS=1 SWIFT_TEST_IN_PROCESS_CONF_LOADER=encryption \
tox -e func
This particular example may also be run using the func-encryption
tox environment:
tox -e func-encryption
The tox.ini
file also specifies test environments for running other
in-process functional test configurations, e.g.:
tox -e func-ec
To debug the functional tests, use the ‘in-process test’ mode and pass the
--pdb
flag to tox
:
SWIFT_TEST_IN_PROCESS=1 tox -e func -- --pdb \
test.functional.tests.TestFile.testCopy
The ‘in-process test’ mode searches for proxy-server.conf
and
swift.conf
config files from which it copies config options and overrides
some options to suit in process testing. The search will first look for config
files in a <custom_conf_source_dir>
that may optionally be specified using
the environment variable:
SWIFT_TEST_IN_PROCESS_CONF_DIR=<custom_conf_source_dir>
If SWIFT_TEST_IN_PROCESS_CONF_DIR
is not set, or if a config file is not
found in <custom_conf_source_dir>
, the search will then look in the
etc/
directory in the source tree. If the config file is still not found,
the corresponding sample config file from etc/
is used (e.g.
proxy-server.conf-sample
or swift.conf-sample
).
When using the ‘in-process test’ mode SWIFT_TEST_POLICY
may be set to
specify a particular storage policy name that will be used for testing as
described above. When set, this policy must exist in the swift.conf
file
and its corresponding ring file must exist in <custom_conf_source_dir>
(if
specified) or etc/
. The test setup will set the specified policy to be the
default and use its ring file properties for constructing the test object ring.
This allows in-process testing to be run against various policy types and ring
files.
For example, this command would run the in-process mode functional tests
using config files found in $HOME/my_tests
and policy ‘silver’:
SWIFT_TEST_IN_PROCESS=1 SWIFT_TEST_IN_PROCESS_CONF_DIR=$HOME/my_tests \
SWIFT_TEST_POLICY=silver tox -e func
Swift uses flake8 with the OpenStack hacking module to enforce coding style.
Install flake8 and hacking with pip or by the packages of your Operating System.
It is advised to integrate flake8+hacking with your editor to get it automated and not get caught by Jenkins.
For example for Vim the syntastic plugin can do this for you.
The documentation in docstrings should follow the PEP 257 conventions (as mentioned in the PEP 8 guidelines).
More specifically:
Installing Sphinx:
sudo apt-get install python-sphinx
)python setup.py build_sphinx
For sanity check of your change in manpage, use this command in the root of your Swift repo:
./.manpages
You can have the following copyright and license statement at the top of each source file. Copyright assignment is optional.
New files should contain the current year. Substantial updates can have another year added, and date ranges are not needed.:
# Copyright (c) 2013 OpenStack Foundation.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
# implied.
# See the License for the specific language governing permissions and
# limitations under the License.
Except where otherwise noted, this document is licensed under Creative Commons Attribution 3.0 License. See all OpenStack Legal Documents.