Account¶
Account Auditor¶
- class swift.account.auditor.AccountAuditor(conf, logger=None)¶
Bases:
DatabaseAuditor
Audit accounts.
- broker_class¶
alias of
AccountBroker
- server_type = 'account'¶
- swift.account.auditor.main()¶
Account Backend¶
Pluggable Back-end for Account Server
- class swift.account.backend.AccountBroker(db_file, timeout=25, logger=None, account=None, container=None, pending_timeout=None, stale_reads_ok=False, skip_commits=False)¶
Bases:
DatabaseBroker
Encapsulates working with an account database.
- create_account_stat_table(conn, put_timestamp)¶
Create account_stat table which is specific to the account DB. Not a part of Pluggable Back-ends, internal to the baseline code.
- Parameters:
conn – DB connection object
put_timestamp – put timestamp
- create_container_table(conn)¶
Create container table which is specific to the account DB.
- Parameters:
conn – DB connection object
- create_policy_stat_table(conn)¶
Create policy_stat table which is specific to the account DB. Not a part of Pluggable Back-ends, internal to the baseline code.
- Parameters:
conn – DB connection object
- db_contains_type = 'container'¶
- db_reclaim_timestamp = 'delete_timestamp'¶
- db_type = 'account'¶
- empty()¶
Check if the account DB is empty.
- Returns:
True if the database has no active containers.
- get_db_version(conn)¶
- get_info()¶
Get global data for the account.
- Returns:
dict with keys: account, created_at, put_timestamp, delete_timestamp, status_changed_at, container_count, object_count, bytes_used, hash, id
- get_policy_stats(do_migrations=False)¶
Get global policy stats for the account.
- Parameters:
do_migrations – boolean, if True the policy stat dicts will always include the ‘container_count’ key; otherwise it may be omitted on legacy databases until they are migrated.
- Returns:
dict of policy stats where the key is the policy index and the value is a dictionary like {‘object_count’: M, ‘bytes_used’: N, ‘container_count’: L}
- is_status_deleted()¶
Only returns true if the status field is set to DELETED.
- list_containers_iter(limit, marker, end_marker, prefix, delimiter, reverse=False, allow_reserved=False)¶
Get a list of containers sorted by name starting at marker onward, up to limit entries. Entries will begin with the prefix and will not have the delimiter after the prefix.
- Parameters:
limit – maximum number of entries to get
marker – marker query
end_marker – end marker query
prefix – prefix query
delimiter – delimiter for query
reverse – reverse the result order.
allow_reserved – exclude names with reserved-byte by default
- Returns:
list of tuples of (name, object_count, bytes_used, put_timestamp, 0)
- make_tuple_for_pickle(record)¶
Turn this db record dict into the format this service uses for pending pickles.
- merge_items(item_list, source=None)¶
Merge items into the container table.
- Parameters:
item_list – list of dictionaries of {‘name’, ‘put_timestamp’, ‘delete_timestamp’, ‘object_count’, ‘bytes_used’, ‘deleted’, ‘storage_policy_index’}
source – if defined, update incoming_sync with the source
- put_container(name, put_timestamp, delete_timestamp, object_count, bytes_used, storage_policy_index)¶
Create a container with the given attributes.
- Parameters:
name – name of the container to create (a native string)
put_timestamp – put_timestamp of the container to create
delete_timestamp – delete_timestamp of the container to create
object_count – number of objects in the container
bytes_used – number of bytes used by the container
storage_policy_index – the storage policy for this container
Account Reaper¶
- class swift.account.reaper.AccountReaper(conf, logger=None)¶
Bases:
Daemon
Removes data from status=DELETED accounts. These are accounts that have been asked to be removed by the reseller via services remove_storage_account XMLRPC call.
The account is not deleted immediately by the services call, but instead the account is simply marked for deletion by setting the status column in the account_stat table of the account database. This account reaper scans for such accounts and removes the data in the background. The background deletion process will occur on the primary account server for the account.
- Parameters:
server_conf – The [account-server] dictionary of the account server configuration file
reaper_conf – The [account-reaper] dictionary of the account server configuration file
See the etc/account-server.conf-sample for information on the possible configuration parameters.
- get_account_ring()¶
The account
swift.common.ring.Ring
for the cluster.
- get_container_ring()¶
The container
swift.common.ring.Ring
for the cluster.
- get_object_ring(policy_idx)¶
Get the ring identified by the policy index
- Parameters:
policy_idx – Storage policy index
- Returns:
A ring matching the storage policy
- reap_account(broker, partition, nodes, container_shard=None)¶
Called once per pass for each account this server is the primary for and attempts to delete the data for the given account. The reaper will only delete one account at any given time. It will call
reap_container()
up to sqrt(self.concurrency) times concurrently while reaping the account.If there is any exception while deleting a single container, the process will continue for any other containers and the failed containers will be tried again the next time this function is called with the same parameters.
If there is any exception while listing the containers for deletion, the process will stop (but will obviously be tried again the next time this function is called with the same parameters). This isn’t likely since the listing comes from the local database.
After the process completes (successfully or not) statistics about what was accomplished will be logged.
This function returns nothing and should raise no exception but only update various self.stats_* values for what occurs.
- Parameters:
broker – The AccountBroker for the account to delete.
partition – The partition in the account ring the account is on.
nodes – The primary node dicts for the account to delete.
container_shard – int used to shard containers reaped. If None, will reap all containers.
See also
swift.account.backend.AccountBroker
for the broker class.See also
swift.common.ring.Ring.get_nodes()
for a description of the node dicts.
- reap_container(account, account_partition, account_nodes, container)¶
Deletes the data and the container itself for the given container. This will call
reap_object()
up to sqrt(self.concurrency) times concurrently for the objects in the container.If there is any exception while deleting a single object, the process will continue for any other objects in the container and the failed objects will be tried again the next time this function is called with the same parameters.
If there is any exception while listing the objects for deletion, the process will stop (but will obviously be tried again the next time this function is called with the same parameters). This is a possibility since the listing comes from querying just the primary remote container server.
Once all objects have been attempted to be deleted, the container itself will be attempted to be deleted by sending a delete request to all container nodes. The format of the delete request is such that each container server will update a corresponding account server, removing the container from the account’s listing.
This function returns nothing and should raise no exception but only update various self.stats_* values for what occurs.
- Parameters:
account – The name of the account for the container.
account_partition – The partition for the account on the account ring.
account_nodes – The primary node dicts for the account.
container – The name of the container to delete.
See also:
swift.common.ring.Ring.get_nodes()
for a description of the account node dicts.
- reap_device(device)¶
Called once per pass for each device on the server. This will scan the accounts directory for the device, looking for partitions this device is the primary for, then looking for account databases that are marked status=DELETED and still have containers and calling
reap_account()
. Account databases marked status=DELETED that no longer have containers will eventually be permanently removed by the reclaim process within the account replicator (seeswift.db_replicator
).- Parameters:
device – The device to look for accounts to be deleted.
- reap_object(account, container, container_partition, container_nodes, obj, policy_index)¶
Deletes the given object by issuing a delete request to each node for the object. The format of the delete request is such that each object server will update a corresponding container server, removing the object from the container’s listing.
This function returns nothing and should raise no exception but only update various self.stats_* values for what occurs.
- Parameters:
account – The name of the account for the object.
container – The name of the container for the object.
container_partition – The partition for the container on the container ring.
container_nodes – The primary node dicts for the container.
obj – The name of the object to delete.
policy_index – The storage policy index of the object’s container
See also:
swift.common.ring.Ring.get_nodes()
for a description of the container node dicts.
- reset_stats()¶
- run_forever(*args, **kwargs)¶
Main entry point when running the reaper in normal daemon mode.
This repeatedly calls
run_once()
no quicker than the configuration interval.
- run_once(*args, **kwargs)¶
Main entry point when running the reaper in ‘once’ mode, where it will do a single pass over all accounts on the server. This is called repeatedly by
run_forever()
. This will callreap_device()
once for each device on the server.
- swift.account.reaper.main()¶
Account Server¶
- class swift.account.server.AccountController(conf, logger=None)¶
Bases:
BaseStorageServer
WSGI controller for the account server.
- DELETE(req)¶
Handle HTTP DELETE request.
- GET(req)¶
Handle HTTP GET request.
- HEAD(req)¶
Handle HTTP HEAD request.
- POST(req)¶
Handle HTTP POST request.
- PUT(req)¶
Handle HTTP PUT request.
- REPLICATE(req)¶
Handle HTTP REPLICATE request. Handler for RPC calls for account replication.
- check_free_space(drive)¶
- server_type = 'account-server'¶
- swift.account.server.app_factory(global_conf, **local_conf)¶
paste.deploy app factory for creating WSGI account server apps
- swift.account.server.get_account_name_and_placement(req)¶
Split and validate path for an account.
- Parameters:
req – a swob request
- Returns:
a tuple of path parts as strings
- swift.account.server.get_container_name_and_placement(req)¶
Split and validate path for a container.
- Parameters:
req – a swob request
- Returns:
a tuple of path parts as strings
- swift.account.server.main()¶