oslo_cache.backends package

Submodules

oslo_cache.backends.dictionary module

dogpile.cache backend that uses dictionary for storage

class oslo_cache.backends.dictionary.DictCacheBackend(arguments)

Bases: dogpile.cache.api.CacheBackend

A DictCacheBackend based on dictionary.

Arguments accepted in the arguments dictionary:

Parameters

expiration_time (real) – interval in seconds to indicate maximum time-to-live value for each key in DictCacheBackend. Default expiration_time value is 0, that means that all keys have infinite time-to-live value.

delete(key)

Deletes the value associated with the key if it exists.

Parameters

key – dictionary key

delete_multi(keys)

Deletes the value associated with each key in list if it exists.

Parameters

keys – list of dictionary keys

get(key)

Retrieves the value for a key.

Parameters

key – dictionary key

Returns

value for a key or oslo_cache.core.NO_VALUE for nonexistent or expired keys.

get_multi(keys)

Retrieves the value for a list of keys.

set(key, value)

Sets the value for a key.

Expunges expired keys during each set.

Parameters

  • key – dictionary key

  • value – value associated with the key

set_multi(mapping)

Set multiple values in the cache. Expunges expired keys during each set.

Parameters

mapping – dictionary with key/value pairs

oslo_cache.backends.etcd3gw module

dogpile.cache backend that uses etcd 3.x for storage

class oslo_cache.backends.etcd3gw.Etcd3gwCacheBackend(arguments)

Bases: dogpile.cache.api.CacheBackend

DEFAULT_HOST = 'localhost'

Default hostname used when none is provided.

DEFAULT_PORT = 2379

Default port used if none provided (4001 or 2379 are the common ones).

DEFAULT_TIMEOUT = 30

Default socket/lock/member/leader timeout used when none is provided.

delete(key)

Delete a value from the cache.

Parameters

key – String key that was passed to the CacheRegion.delete() method, which will also be processed by the “key mangling” function if one was present.

The behavior here should be idempotent, that is, can be called any number of times regardless of whether or not the key exists.

delete_multi(keys)

Delete multiple values from the cache.

Parameters

keys – sequence of string keys that was passed to the CacheRegion.delete_multi() method, which will also be processed by the “key mangling” function if one was present.

The behavior here should be idempotent, that is, can be called any number of times regardless of whether or not the key exists.

New in version 0.5.0.

get(key)

Retrieve an optionally serialized value from the cache.

Parameters

key – String key that was passed to the CacheRegion.get() method, which will also be processed by the “key mangling” function if one was present.

Returns

the Python object that corresponds to what was established via the CacheBackend.set() method, or the NO_VALUE constant if not present.

If a serializer is in use, this method will only be called if the CacheBackend.get_serialized() method is not overridden.

get_multi(keys)

Retrieves the value for a list of keys.

set(key, value)

Set an optionally serialized value in the cache.

Parameters

  • key – String key that was passed to the CacheRegion.set() method, which will also be processed by the “key mangling” function if one was present.

  • value – The optionally serialized CachedValue object. May be an instance of CachedValue or a bytes object depending on if a serializer is in use with the region and if the CacheBackend.set_serialized() method is not overridden.

See also

CacheBackend.set_serialized()

set_multi(mapping)

Set multiple values in the cache.

Parameters

mapping – a dict in which the key will be whatever was passed to the CacheRegion.set_multi() method, processed by the “key mangling” function, if any.

When implementing a new CacheBackend or cutomizing via ProxyBackend, be aware that when this method is invoked by Region.get_or_create_multi(), the mapping values are the same ones returned to the upstream caller. If the subclass alters the values in any way, it must not do so ‘in-place’ on the mapping dict – that will have the undesirable effect of modifying the returned values as well.

If a serializer is in use, this method will only be called if the CacheBackend.set_serialized_multi() method is not overridden.

New in version 0.5.0.

oslo_cache.backends.memcache_pool module

dogpile.cache backend that uses Memcached connection pool

class oslo_cache.backends.memcache_pool.ClientProxy(client_pool)

Bases: object

class oslo_cache.backends.memcache_pool.PooledMemcachedBackend(arguments)

Bases: dogpile.cache.backends.memcached.MemcachedBackend

Memcached backend that does connection pooling.

This memcached backend only allows for reuse of a client object, prevents too many client object from being instantiated, and maintains proper tracking of dead servers so as to limit delays when a server (or all servers) become unavailable.

This backend doesn’t allow to load balance things between servers.

Memcached isn’t HA. Values aren’t automatically replicated between servers unless the client went out and wrote the value multiple time.

The memcache server to use is determined by python-memcached itself by picking the host to use (from the given server list) based on a key hash.

property client

Return the memcached client.

This uses a threading.local by default as it appears most modern memcached libs aren’t inherently threadsafe.

oslo_cache.backends.mongo module

class oslo_cache.backends.mongo.MongoCacheBackend(arguments)

Bases: dogpile.cache.api.CacheBackend

A MongoDB based caching backend implementing dogpile backend APIs.

Arguments accepted in the arguments dictionary:

Parameters

  • db_hosts – string (required), hostname or IP address of the MongoDB server instance. This can be a single MongoDB connection URI, or a list of MongoDB connection URIs.

  • db_name – string (required), the name of the database to be used.

  • cache_collection – string (required), the name of collection to store cached data. Note: Different collection name can be provided if there is need to create separate container (i.e. collection) for cache data. So region configuration is done per collection.

Following are optional parameters for MongoDB backend configuration,

Parameters

  • username – string, the name of the user to authenticate.

  • password – string, the password of the user to authenticate.

  • max_pool_size – integer, the maximum number of connections that the pool will open simultaneously. By default the pool size is 10.

  • w –

    integer, write acknowledgement for MongoDB client

    If not provided, then no default is set on MongoDB and then write acknowledgement behavior occurs as per MongoDB default. This parameter name is same as what is used in MongoDB docs. This value is specified at collection level so its applicable to cache_collection db write operations.

    If this is a replica set, write operations will block until they have been replicated to the specified number or tagged set of servers. Setting w=0 disables write acknowledgement and all other write concern options.

  • read_preference – string, the read preference mode for MongoDB client Expected value is primary, primaryPreferred, secondary, secondaryPreferred, or nearest. This read_preference is specified at collection level so its applicable to cache_collection db read operations.

  • use_replica – boolean, flag to indicate if replica client to be used. Default is False. replicaset_name value is required if True.

  • replicaset_name – string, name of replica set. Becomes required if use_replica is True

  • son_manipulator –

    string, name of class with module name which implements MongoDB SONManipulator. Default manipulator used is BaseTransform.

    This manipulator is added per database. In multiple cache configurations, the manipulator name should be same if same database name db_name is used in those configurations.

    SONManipulator is used to manipulate custom data types as they are saved or retrieved from MongoDB. Custom impl is only needed if cached data is custom class and needs transformations when saving or reading from db. If dogpile cached value contains built-in data types, then BaseTransform class is sufficient as it already handles dogpile CachedValue class transformation.

  • mongo_ttl_seconds –

    integer, interval in seconds to indicate maximum time-to-live value. If value is greater than 0, then its assumed that cache_collection needs to be TTL type (has index at ‘doc_date’ field). By default, the value is -1 and its disabled. Reference: <http://docs.mongodb.org/manual/tutorial/expire-data/>

    Note

    This parameter is different from Dogpile own expiration_time, which is the number of seconds after which Dogpile will consider the value to be expired. When Dogpile considers a value to be expired, it continues to use the value until generation of a new value is complete, when using CacheRegion.get_or_create(). Therefore, if you are setting mongo_ttl_seconds, you will want to make sure it is greater than expiration_time by at least enough seconds for new values to be generated, else the value would not be available during a regeneration, forcing all threads to wait for a regeneration each time a value expires.

  • ssl – boolean, If True, create the connection to the server using SSL. Default is False. Client SSL connection parameters depends on server side SSL setup. For further reference on SSL configuration: <http://docs.mongodb.org/manual/tutorial/configure-ssl/>

  • ssl_keyfile – string, the private keyfile used to identify the local connection against mongod. If included with the certfile then only the ssl_certfile is needed. Used only when ssl is True.

  • ssl_certfile – string, the certificate file used to identify the local connection against mongod. Used only when ssl is True.

  • ssl_ca_certs – string, the ca_certs file contains a set of concatenated ‘certification authority’ certificates, which are used to validate certificates passed from the other end of the connection. Used only when ssl is True.

  • ssl_cert_reqs – string, the parameter cert_reqs specifies whether a certificate is required from the other side of the connection, and whether it will be validated if provided. It must be one of the three values ssl.CERT_NONE (certificates ignored), ssl.CERT_OPTIONAL (not required, but validated if provided), or ssl.CERT_REQUIRED (required and validated). If the value of this parameter is not ssl.CERT_NONE, then the ssl_ca_certs parameter must point to a file of CA certificates. Used only when ssl is True.

Rest of arguments are passed to mongo calls for read, write and remove. So related options can be specified to pass to these operations.

Further details of various supported arguments can be referred from <http://api.mongodb.org/python/current/api/pymongo/>

client

Initializes MongoDB connection and collection defaults.

This initialization is done only once and performed as part of lazy inclusion of MongoDB dependency i.e. add imports only if related backend is used.

Returns

MongoApi instance

delete(key)

Delete a value from the cache.

Parameters

key – String key that was passed to the CacheRegion.delete() method, which will also be processed by the “key mangling” function if one was present.

The behavior here should be idempotent, that is, can be called any number of times regardless of whether or not the key exists.

delete_multi(keys)

Delete multiple values from the cache.

Parameters

keys – sequence of string keys that was passed to the CacheRegion.delete_multi() method, which will also be processed by the “key mangling” function if one was present.

The behavior here should be idempotent, that is, can be called any number of times regardless of whether or not the key exists.

New in version 0.5.0.

get(key)

Retrieves the value for a key.

Parameters

key – key to be retrieved.

Returns

value for a key or oslo_cache.core.NO_VALUE for nonexistent or expired keys.

get_multi(keys)

Return multiple values from the cache, based on the given keys.

Parameters

keys – sequence of keys to be retrieved.

Returns

returns values (or oslo_cache.core.NO_VALUE) as a list matching the keys given.

set(key, value)

Set an optionally serialized value in the cache.

Parameters

  • key – String key that was passed to the CacheRegion.set() method, which will also be processed by the “key mangling” function if one was present.

  • value – The optionally serialized CachedValue object. May be an instance of CachedValue or a bytes object depending on if a serializer is in use with the region and if the CacheBackend.set_serialized() method is not overridden.

See also

CacheBackend.set_serialized()

set_multi(mapping)

Set multiple values in the cache.

Parameters

mapping – a dict in which the key will be whatever was passed to the CacheRegion.set_multi() method, processed by the “key mangling” function, if any.

When implementing a new CacheBackend or cutomizing via ProxyBackend, be aware that when this method is invoked by Region.get_or_create_multi(), the mapping values are the same ones returned to the upstream caller. If the subclass alters the values in any way, it must not do so ‘in-place’ on the mapping dict – that will have the undesirable effect of modifying the returned values as well.

If a serializer is in use, this method will only be called if the CacheBackend.set_serialized_multi() method is not overridden.

New in version 0.5.0.

Module contents