pbr - Python Build Reasonableness¶
A library for managing setuptools packaging needs in a consistent manner.
pbr reads and then filters the setup.cfg data through a setup hook to fill in default values and provide more sensible behaviors, and then feeds the results in as the arguments to a call to setup.py - so the heavy lifting of handling python packaging needs is still being done by setuptools.
Note that we don’t support the easy_install aspects of setuptools: while we depend on setup_requires, for any install_requires we recommend that they be installed prior to running setup.py install - either by hand, or by using an install tool such as pip.
What It Does¶
PBR can and does do a bunch of things for you:
- Version: Manage version number based on git revisions and tags
- AUTHORS: Generate AUTHORS file from git log
- ChangeLog: Generate ChangeLog from git log
- Manifest: Generate a sensible manifest from git files and some standard files
- Sphinx Autodoc: Generate autodoc stub files for your whole module
- Requirements: Store your dependencies in a pip requirements file
- long_description: Use your README file as a long_description
- Smart find_packages: Smartly find packages under your root package
Version¶
Versions can be managed two ways - postversioning and preversioning.
Postversioning is the default, and preversioning is enabled by setting
version in the setup.cfg metadata section. In both cases version
strings are inferred from git.
If the currently checked out revision is tagged, that tag is used as the version.
If the currently checked out revision is not tagged, then we take the last tagged version number and increment it to get a minimum target version.
We then walk git history back to the last release. Within each commit we look
for a Sem-Ver: pseudo header, and if found parse it looking for keywords.
Unknown symbols are not an error (so that folk can’t wedge pbr or break their
tree), but we will emit an info level warning message. Known symbols:
feature, api-break, deprecation, bugfix. A missing
Sem-Ver line is equivalent to Sem-Ver: bugfix. The bugfix symbol causes
a patch level increment to the version. The feature and deprecation
symbols cause a minor version increment. The api-break symbol causes a
major version increment.
If postversioning is in use, we use the resulting version number as the target version.
If preversioning is in use we check that the version set in the metadata section of setup.cfg is greater than the version we infer using the above method. If the inferred version is greater than the preversioning value we raise an error, otherwise we use the version from setup.cfg as the target.
We then generate dev version strings based on the commits since the last release and include the current git sha to disambiguate multiple dev versions with the same number of commits since the release.
Note
pbr expects git tags to be signed for use in calculating versions
The versions are expected to be compliant with Linux/Python Compatible Semantic Versioning 3.0.0.
The version.SemanticVersion class can be used to query versions of a
package and present it in various forms - debian_version(),
release_string(), rpm_string(), version_string(), or
version_tuple().
AUTHORS and ChangeLog¶
Why keep an AUTHORS or a ChangeLog file when git already has all of the information you need? AUTHORS generation supports filtering/combining based on a standard .mailmap file.
Manifest¶
Just like AUTHORS and ChangeLog, why keep a list of files you wish to include when you can find many of these in git. MANIFEST.in generation ensures almost all files stored in git, with the exception of .gitignore, .gitreview and .pyc files, are automatically included in your distribution. In addition, the generated AUTHORS and ChangeLog files are also included. In many cases, this removes the need for an explicit ‘MANIFEST.in’ file
Sphinx Autodoc¶
Sphinx can produce auto documentation indexes based on signatures and docstrings of your project but you have to give it index files to tell it to autodoc each module: that’s kind of repetitive and boring. PBR will scan your project, find all of your modules, and generate all of the stub files for you.
Sphinx documentation setups are altered to generate man pages by default. They also have several pieces of information that are known to setup.py injected into the sphinx config.
See the pbr section for details on configuring your project for autodoc.
Requirements¶
You may not have noticed, but there are differences in how pip requirements.txt files work and how distutils wants to be told about requirements. The pip way is nicer because it sure does make it easier to populate a virtualenv for testing or to just install everything you need. Duplicating the information, though, is super lame. To solve this issue, pbr will let you use requirements.txt-format files to describe the requirements for your project and will then parse these files, split them up appropriately, and inject them into the install_requires, tests_require and/or dependency_links arguments to setup. Voila!
You can also have a requirement file for each specific major version of Python. If you want to have a different package list for Python 3 then just drop a requirements-py3.txt and it will be used instead.
Finally, it is possible to specify groups of optional dependencies, or “extra” requirements, in your setup.cfg rather than setup.py.
long_description¶
There is no need to maintain two long descriptions- and your README file is probably a good long_description. So we’ll just inject the contents of your README.rst, README.txt or README file into your empty long_description. Yay for you.
Usage¶
pbr is a setuptools plugin and so to use it you must use setuptools and call
setuptools.setup(). While the normal setuptools facilities are available,
pbr makes it possible to express them through static data files.
setup.py¶
pbr only requires a minimal setup.py file compared to a standard setuptools project. This is because most configuration is located in static configuration files. This recommended minimal setup.py file should look something like this:
#!/usr/bin/env python
from setuptools import setup
setup(
setup_requires=['pbr>=1.9', 'setuptools>=17.1'],
pbr=True,
)
Note
It is necessary to specify pbr=True to enabled pbr functionality.
Note
While one can pass any arguments supported by setuptools to setup(), any conflicting arguments supplied in setup.cfg will take precedence.
setup.cfg¶
The setup.cfg file is an ini-like file that can mostly replace the setup.py file. It is based on the distutils2 setup.cfg file. A simple sample can be found in pbr‘s own setup.cfg (it uses its own machinery to install itself):
[metadata]
name = pbr
author = OpenStack Foundation
author-email = openstack-dev@lists.openstack.org
summary = OpenStack's setup automation in a reusable form
description-file = README
home-page = https://launchpad.net/pbr
license = Apache-2
classifier =
Development Status :: 4 - Beta
Environment :: Console
Environment :: OpenStack
Intended Audience :: Developers
Intended Audience :: Information Technology
License :: OSI Approved :: Apache Software License
Operating System :: OS Independent
Programming Language :: Python
keywords =
setup
distutils
[files]
packages =
pbr
data_files =
etc/pbr = etc/*
etc/init =
pbr.packaging.conf
pbr.version.conf
[entry_points]
console_scripts =
pbr = pbr.cmd:main
pbr.config.drivers =
plain = pbr.cfg.driver:Plain
pbr provides its own section in these documents, ostensibly called pbr. Most other sections are provided by setuptools and may influence either the build itself or the output of various setuptools commands. The remaining sections are provided by libraries that provide setuptools extensions, such as extract_mesages (provided by Babel) or sphinx_build (provided by Sphinx). Some of these are described below.
Note
Comments may be used in setup.cfg, however all comments should start with a # and may be on a single line, or in line, with at least one white space character immediately preceding the #. Semicolons are not a supported comment delimiter. For instance:
[section]
# A comment at the start of a dedicated line
key =
value1 # An in line comment
value2
# A comment on a dedicated line
value3
files¶
The files section defines the install location of files in the package
using three fundamental keys: packages, namespace_packages, and
data_files.
packages is a list of top-level packages that should be installed. The
behavior of packages is similar to setuptools.find_packages in that it
recurses the python package hierarchy below the given top level and installs
all of it. If packages is not specified, it defaults to the value of the
name field given in the [metadata] section.
namespace_packages is the same, but is a list of packages that provide
namespace packages.
data_files lists files to be installed. The format is an indented block
that contains key value pairs which specify target directory and source file
to install there. More than one source file for a directory may be indicated
with a further indented list. Source files are stripped of leading directories.
Additionally, pbr supports a simple file globbing syntax for installing
entire directory structures, thus:
[files]
data_files =
etc/pbr = etc/pbr/*
etc/neutron =
etc/api-paste.ini
etc/dhcp-agent.ini
etc/init.d = neutron.init
will result in /etc/neutron containing api-paste.ini and dhcp-agent.ini, both of which pbr will expect to find in the etc directory in the root of the source tree. Additionally, neutron.init from that dir will be installed in /etc/init.d. All of the files and directories located under etc/pbr in the source tree will be installed into /etc/pbr.
Note that this behavior is relative to the effective root of the environment into which the packages are installed, so depending on available permissions this could be the actual system-wide /etc directory or just a top-level etc subdirectory of a virtualenv.
pbr¶
The pbr section controls pbr specific options and behaviours.
The autodoc_tree_index_modules is a boolean option controlling whether pbr
should generate an index of modules using sphinx-apidoc. By default,
setup.py is excluded. The list of excluded modules can be specified with the
autodoc_tree_excludes option. See the sphinx-apidoc man page for more
information.
The autodoc_index_modules is a boolean option controlling whether pbr
should itself generates documentation for Python modules of the project. By
default, all found Python modules are included; some of them can be excluded
by listing them in autodoc_exclude_modules. This list of modules can
contains fnmatch style pattern (e.g. myapp.tests.*) to exclude some modules.
Note
When using autodoc_tree_excludes or autodoc_index_modules you may
also need to set exclude_patterns in your Sphinx configuration file
(generally found at doc/source/conf.py in most OpenStack projects)
otherwise Sphinx may complain about documents that are not in a toctree.
This is especially true if the [sphinx_build] warning-is-error option is
set. See the Sphinx build configuration file documentation for more
information on configuring Sphinx.
entry_points¶
The entry_points section defines entry points for generated console scripts
and python libraries. This is actually provided by setuptools but is
documented here owing to its importance.
The general syntax of specifying entry points is a top level name indicating the entry point group name, followed by one or more key value pairs naming the entry point to be installed. For instance:
[entry_points]
console_scripts =
pbr = pbr.cmd:main
pbr.config.drivers =
plain = pbr.cfg.driver:Plain
fancy = pbr.cfg.driver:Fancy
Will cause a console script called pbr to be installed that executes the main function found in pbr.cmd. Additionally, two entry points will be installed for pbr.config.drivers, one called plain which maps to the Plain class in pbr.cfg.driver and one called fancy which maps to the Fancy class in pbr.cfg.driver.
Requirements¶
Requirement files should be given one of the below names. This order is also the order that the requirements are tried in (where N is the Python major version number used to install the package):
- requirements-pyN.txt
- tools/pip-requires-py3
- requirements.txt
- tools/pip-requires
Only the first file found is used to install the list of packages it contains.
Note
The ‘requirements-pyN.txt’ file is deprecated - ‘requirements.txt’ should be universal. You can use Environment markers for this purpose.
Extra requirements¶
Groups of optional dependencies, or “extra” requirements, can be described in your setup.cfg, rather than needing to be added to setup.py. An example (which also demonstrates the use of environment markers) is shown below.
Environment markers¶
Environment markers are conditional dependencies which can be added to the requirements (or to a group of extra requirements) automatically, depending on the environment the installer is running in. They can be added to requirements in the requirements file, or to extras defined in setup.cfg, but the format is slightly different for each.
For requirements.txt:
argparse; python_version=='2.6'
This will result in the package depending on argparse only if it’s being
installed into Python 2.6
For extras specified in setup.cfg, add an extras section. For instance,
to create two groups of extra requirements with additional constraints on the
environment, you can use:
[extras]
security =
aleph
bet:python_version=='3.2'
gimel:python_version=='2.7'
testing =
quux:python_version=='2.7'