Bases: object
Base WSGI controller class for the proxy
Handler for HTTP GET requests.
Parameters: | req – The client request |
---|---|
Returns: | the response to the client |
Base handler for HTTP GET or HEAD requests.
Parameters: |
|
---|---|
Returns: | swob.Response object |
Handler for HTTP HEAD requests.
Parameters: | req – The client request |
---|---|
Returns: | the response to the client |
Base handler for OPTIONS requests
Parameters: | req – swob.Request object |
---|---|
Returns: | swob.Response object |
Get account information, and also verify that the account exists.
Parameters: |
|
---|---|
Returns: | tuple of (account partition, account nodes, container_count) or (None, None, None) if it does not exist |
Autocreate an account
Parameters: |
|
---|
Given a list of responses from several servers, choose the best to return to the API.
Parameters: |
|
---|---|
Returns: | swob.Response object with the correct status, body, etc. set |
Get container information and thusly verify container existence. This will also verify account existence.
Parameters: |
|
---|---|
Returns: | dict containing at least container partition (‘partition’), container nodes (‘containers’), container read acl (‘read_acl’), container write acl (‘write_acl’), and container sync key (‘sync_key’). Values are set to None if the container does not exist. |
Create a list of headers to be used in backend requests
Parameters: |
|
---|---|
Returns: | a dictionary of headers |
Given a list of statuses from several requests, determine if a quorum response can already be decided.
Parameters: |
|
---|---|
Returns: | True or False, depending on if quorum is established |
Is the given Origin allowed to make requests to this resource
Parameters: |
|
---|---|
Returns: | True or False |
Sends an HTTP request to multiple nodes and aggregates the results. It attempts the primary nodes concurrently, then iterates over the handoff nodes as needed.
Parameters: |
|
---|---|
Returns: | a swob.Response object |
Transfer legal headers from an original client request to dictionary that will be used as headers by the backend request
Parameters: |
|
---|
Bases: swift.proxy.controllers.base.ResumingGetter
Bases: object
Yields nodes for a ring partition, skipping over error limited nodes and stopping at the configurable number of nodes. If a node yielded subsequently gets error limited, an extra node will be yielded to take its place.
Note that if you’re going to iterate over this concurrently from multiple greenthreads, you’ll want to use a swift.common.utils.GreenthreadSafeIterator to serialize access. Otherwise, you may get ValueErrors from concurrent access. (You also may not, depending on how logging is configured, the vagaries of socket IO and eventlet, and the phase of the moon.)
Parameters: |
|
---|
Log handoff requests if handoff logging is enabled and the handoff was not expected.
We only log handoffs when we’ve pushed the handoff count further than we would normally have expected under normal circumstances, that is (request_node_count - num_primaries), when handoffs goes higher than that it means one of the primaries must have been skipped because of error limiting before we consumed all of our nodes_left.
Bases: object
Will skip num_bytes into the current ranges.
Params num_bytes: | |
---|---|
the number of bytes that have already been read on this request. This will change the Range header so that the next req will start where it left off. |
|
Raises: |
|
Indicates whether or not the request made to the backend found what it was looking for.
Parameters: | src – the response from the backend |
---|---|
Returns: | True if found, False if not |
If client_chunk_size is set, makes sure we yield things starting on chunk boundaries based on the Content-Range header in the response.
Sets our Range header’s first byterange to the value learned from the Content-Range header in the response; if we were given a fully-specified range (e.g. “bytes=123-456”), this is a no-op.
If we were given a half-specified range (e.g. “bytes=123-” or “bytes=-456”), then this changes the Range header to a semantically-equivalent one and it lets us resume on a proper boundary instead of just in the middle of a piece somewhere.
Remove the first byterange from our Range header.
This is used after a byterange has been completely sent to the client; this way, should we need to resume the download from another object server, we do not re-fetch byteranges that the client already has.
If we have no Range header, this is a no-op.
Assume an object is composed of N records, where the first N-1 are all the same size and the last is at most that large, but may be smaller.
When a range request is made, it might start with a partial record. This must be discarded, lest the consumer get bad data. This is particularly true of suffix-byte-range requests, e.g. “Range: bytes=-12345” where the size of the object is unknown at the time the request is made.
This function computes the number of bytes that must be discarded to ensure only whole records are yielded. Erasure-code decoding needs this.
This function could have been inlined, but it took enough tries to get right that some targeted unit tests were desirable, hence its extraction.
Clear the cached info in both memcache and env
Parameters: |
|
---|
Force close the http connection to the backend.
Parameters: | src – the response from the backend |
---|
Decorator to check if the request is a CORS request and if so, if it’s valid.
Parameters: | func – function to check |
---|
Decorator to declare which methods should have any swift.authorize call delayed. This is so the method can load the Request object up with additional information that may be needed by the authorization system.
Parameters: | func – function for which authorization will be delayed |
---|
Get the info structure for an account, based on env and app. This is useful to middlewares.
Note
This call bypasses auth. Success does not imply that the request has authorization to the account.
Raises ValueError: | |
---|---|
when path can’t be split(path, 2, 4) |
Get the info structure for a container, based on env and app. This is useful to middlewares.
Note
This call bypasses auth. Success does not imply that the request has authorization to the container.
Get the info about accounts or containers
Parameters: |
|
---|---|
Returns: | the cached info or None if cannot be retrieved |
Get the keys for env (env_key) where info about object is cached
Parameters: |
|
---|---|
Returns: | a string env_key |
Get the info structure for an object, based on env and app. This is useful to middlewares.
Note
This call bypasses auth. Success does not imply that the request has authorization to the object.
Construct a cacheable dict of account info based on response headers.
Construct a cacheable dict of container info based on response headers.
Construct a cacheable dict of object info based on response headers.
Provide the timestamp of the swift http response as a floating point value. Used as a sort key.
Parameters: | resp – bufferedhttp response object |
---|
Helper function to update headers in the response.
Parameters: |
|
---|
Bases: swift.proxy.controllers.base.Controller
WSGI controller for account requests
HTTP DELETE request handler.
Handler for HTTP GET/HEAD requests.
HTTP POST request handler.
HTTP PUT request handler.
Bases: swift.proxy.controllers.base.Controller
WSGI controller for container requests
HTTP DELETE request handler.
Handler for HTTP GET requests.
Handler for HTTP GET/HEAD requests.
Handler for HTTP HEAD requests.
HTTP POST request handler.
HTTP PUT request handler.
Bases: swift.proxy.controllers.base.Controller
Base WSGI controller for object requests.
HTTP COPY request handler.
HTTP DELETE request handler.
Handler for HTTP GET requests.
Handle HTTP GET or HEAD requests.
Handler for HTTP HEAD requests.
HTTP POST request handler.
HTTP PUT request handler.
Yields nodes for a ring partition.
If the ‘write_affinity’ setting is non-empty, then this will yield N local nodes (as defined by the write_affinity setting) first, then the rest of the nodes as normal. It is a re-ordering of the nodes such that the local ones come first; no node is omitted. The effect is that the request will be serviced by local object servers first, but nonlocal ones will be employed if not enough local ones are available.
Parameters: |
|
---|
Bases: object
WSGI iterable that decodes EC fragment archives (or portions thereof) into the original object (or portions thereof).
Parameters: |
|
---|
Start pulling data from the backends so that we can learn things like the real Content-Type that might only be in the multipart/byteranges response body. Update our response accordingly.
Also, this is the first point at which we can learn the MIME boundary that our response has in the headers. We grab that so we can also use it in the body.
Returns: | None |
---|---|
Raises : | HTTPException on error |
Bases: swift.proxy.controllers.obj.BaseObjectController
Bases: object
This is here mostly to wrap up the fact that all EC PUTs are chunked because of the mime boundary footer trick and the first half of the two-phase PUT conversation handling.
An HTTP PUT request that supports streaming.
Probably deserves more docs than this, but meh.
Get 100-continue response indicating the end of 1st phase of a 2-phase commit or the final response, i.e. the one with status >= 200.
Might or might not actually wait for anything. If we said Expect: 100-continue but got back a non-100 response, that’ll be the thing returned, and we won’t do any network IO to get it. OTOH, if we got a 100 Continue response and sent up the PUT request’s body, then we’ll actually read the 2xx-5xx response off the network here.
Returns: | HTTPResponse |
---|---|
Raises : | Timeout if the response took too long |
Connect to a backend node and send the headers.
Returns: | Putter instance |
---|---|
Raises : | ConnectionTimeout if initial connection timed out |
Raises : | ResponseTimeout if header retrieval timed out |
Raises : | InsufficientStorage on 507 response from node |
Raises : | PutterConnectError on non-507 server error response from node |
Raises : | FooterNotSupported if need_metadata_footer is set but backend node can’t process footers |
Raises : | MultiphasePUTNotSupported if need_multiphase_support is set but backend node can’t handle multiphase PUT |
Returns the current status of the response.
A response starts off with no current status, then may or may not have a status of 100 for some time, and then ultimately has a final status like 200, 404, et cetera.
Call when there is no more data to send.
Parameters: | footer_metadata – dictionary of metadata items |
---|
Call when there are > quorum 2XX responses received. Send commit confirmations to all object nodes to finalize the PUT.
Call before sending the first chunk of request body
Bases: object
Decorator for Storage Policy implemenations to register their ObjectController implementations.
This also fills in a policy_type attribute on the class.
Bases: swift.proxy.controllers.obj.BaseObjectController
Takes a byterange from the client and converts it into a byterange spanning the necessary segments.
Handles prefix, suffix, and fully-specified byte ranges.
Parameters: |
|
---|---|
Returns: | a 2-tuple (seg_start, seg_end) where
|
Will copy desired headers from from_r to to_r :params from_r: a swob Request or Response :params to_r: a swob Request or Response
Takes a byterange spanning some segments and converts that into a byterange spanning the corresponding fragments within their fragment archives.
Handles prefix, suffix, and fully-specified byte ranges.
Parameters: |
|
---|---|
Returns: | a 2-tuple (frag_start, frag_end) where
|
Bases: object
WSGI application for the proxy server.
Check the configuration for possible errors
Mark a node as error limited. This immediately pretends the node received enough errors to trigger error suppression. Use this for errors like Insufficient Storage. For other errors use error_occurred().
Parameters: |
|
---|
Check if the node is currently error limited.
Parameters: | node – dictionary of node to check |
---|---|
Returns: | True if error limited, False otherwise |
Handle logging, and handling of errors.
Parameters: |
|
---|
Handle logging of generic exceptions.
Parameters: |
|
---|
Get the controller to handle a request.
Parameters: | req – the request |
---|---|
Returns: | tuple of (controller class, path dictionary) |
Raises : | ValueError (thrown by split_path) if given invalid path |
Get the ring object to use to handle a request based on its policy.
Parameters: | policy_idx – policy index as defined in swift.conf |
---|---|
Returns: | appropriate ring object |
Entry point for proxy server. Should return a WSGI-style callable (such as swob.Response).
Parameters: | req – swob.Request object |
---|
Called during WSGI pipeline creation. Modifies the WSGI pipeline context to ensure that mandatory middleware is present in the pipeline.
Parameters: | pipe – A PipelineWrapper object |
---|
Sorts nodes in-place (and returns the sorted list) according to the configured strategy. The default “sorting” is to randomly shuffle the nodes. If the “timing” strategy is chosen, the nodes are sorted according to the stored timing data.
paste.deploy app factory for creating WSGI proxy apps.