oslo_concurrency.processutils

System-level utilities and helper functions.

exception oslo_concurrency.processutils.InvalidArgumentError(message=None)

Bases: Exception

class oslo_concurrency.processutils.LogErrors(value)

Bases: IntEnum

Enumerations that affect if stdout and stderr are logged on error.

New in version 2.7.

ALL = 1

Log an error on each occurence of an error.

DEFAULT = 0

No logging on errors.

FINAL = 2

Log an error on the last attempt that errored only.

exception oslo_concurrency.processutils.NoRootWrapSpecified(message=None)

Bases: Exception

exception oslo_concurrency.processutils.ProcessExecutionError(stdout=None, stderr=None, exit_code=None, cmd=None, description=None)

Bases: Exception

class oslo_concurrency.processutils.ProcessLimits(**kw)

Bases: object

Resource limits on a process.

Attributes:

  • address_space: Address space limit in bytes

  • core_file_size: Core file size limit in bytes

  • cpu_time: CPU time limit in seconds

  • data_size: Data size limit in bytes

  • file_size: File size limit in bytes

  • memory_locked: Locked memory limit in bytes

  • number_files: Maximum number of open files

  • number_processes: Maximum number of processes

  • resident_set_size: Maximum Resident Set Size (RSS) in bytes

  • stack_size: Stack size limit in bytes

This object can be used for the prlimit parameter of execute().

prlimit_args()

Create a list of arguments for the prlimit command line.

exception oslo_concurrency.processutils.UnknownArgumentError(message=None)

Bases: Exception

oslo_concurrency.processutils.execute(*cmd, **kwargs)

Helper method to shell out and execute a command through subprocess.

Allows optional retry.

Parameters:

  • cmd (string) – Passed to subprocess.Popen.

  • cwd (string) – Set the current working directory

  • process_input (string or bytes) – Send to opened process.

  • env_variables (dict) – Environment variables and their values that will be set for the process.

  • check_exit_code (boolean, int, or [int]) – Single bool, int, or list of allowed exit codes. Defaults to [0]. Raise ProcessExecutionError unless program exits with one of these code.

  • delay_on_retry (boolean) – True | False. Defaults to True. If set to True, wait a short amount of time before retrying.

  • attempts (int) – How many times to retry cmd.

  • run_as_root (boolean) – True | False. Defaults to False. If set to True, the command is prefixed by the command specified in the root_helper kwarg.

  • root_helper (string) – command to prefix to commands called with run_as_root=True

  • shell (boolean) – whether or not there should be a shell used to execute this command. Defaults to false.

  • loglevel (int. (Should be logging.DEBUG or logging.INFO)) – log level for execute commands.

  • log_errors (LogErrors) – Should stdout and stderr be logged on error? Possible values are DEFAULT, FINAL, or ALL. Note that the values FINAL and ALL are only relevant when multiple attempts of command execution are requested using the attempts parameter.

  • binary (boolean) – On Python 3, return stdout and stderr as bytes if binary is True, as Unicode otherwise.

  • on_execute (function(subprocess.Popen)) – This function will be called upon process creation with the object as a argument. The Purpose of this is to allow the caller of processutils.execute to track process creation asynchronously.

  • on_completion (function(subprocess.Popen)) – This function will be called upon process completion with the object as a argument. The Purpose of this is to allow the caller of processutils.execute to track process completion asynchronously.

  • preexec_fn (function()) – This function will be called in the child process just before the child is executed. WARNING: On windows, we silently drop this preexec_fn as it is not supported by subprocess.Popen on windows (throws a ValueError)

  • prlimit (ProcessLimits) – Set resource limits on the child process. See below for a detailed description.

  • python_exec (string) – The python executable to use for enforcing prlimits. If this is not set or is None, it will default to use sys.executable.

  • timeout (int) – Timeout (in seconds) to wait for the process termination. If timeout is reached, subprocess.TimeoutExpired is raised.

Returns:

(stdout, stderr) from process execution

Raises:

UnknownArgumentError on receiving unknown arguments

Raises:

ProcessExecutionError

Raises:

OSError

Raises:

subprocess.TimeoutExpired

The prlimit parameter can be used to set resource limits on the child process. If this parameter is used, the child process will be spawned by a wrapper process which will set limits before spawning the command.

Changed in version 3.17: process_input can now be either bytes or string on python3.

Changed in version 3.4: Added prlimit optional parameter.

Changed in version 1.5: Added cwd optional parameter.

Changed in version 1.9: Added binary optional parameter. On Python 3, stdout and stderr are now returned as Unicode strings by default, or bytes if binary is true.

Changed in version 2.1: Added on_execute and on_completion optional parameters.

Changed in version 2.3: Added preexec_fn optional parameter.

oslo_concurrency.processutils.get_worker_count()

Utility to get the default worker count.

Returns:

The number of CPUs if that can be determined, else a default worker count of 1 is returned.

oslo_concurrency.processutils.ssh_execute(ssh, cmd, process_input=None, addl_env=None, check_exit_code=True, binary=False, timeout=None, sanitize_stdout=True)

Run a command through SSH.

Parameters:

  • ssh – An SSH Connection object.

  • cmd – The command string to run.

  • check_exit_code – If an exception should be raised for non-zero exit.

  • timeout – Max time in secs to wait for command execution.

  • sanitize_stdout – Defaults to True. If set to True, stdout is sanitized i.e. any sensitive information like password in command output will be masked.

Returns:

(stdout, stderr) from command execution through SSH.

Changed in version 1.9: Added binary optional parameter.

oslo_concurrency.processutils.trycmd(*args, **kwargs)

A wrapper around execute() to more easily handle warnings and errors.

Returns an (out, err) tuple of strings containing the output of the command’s stdout and stderr. If ‘err’ is not empty then the command can be considered to have failed.

Parameters:

discard_warnings (boolean) – True | False. Defaults to False. If set to True, then for succeeding commands, stderr is cleared

Returns:

(out, err) from process execution