Stein Series Release Notes

2.21.1

バグ修正

• Sharding improvements

  • The container-replicator now only attempts to fetch shard ranges if the remote indicates that it has shard ranges. Further, it does so with a timeout to prevent the process from hanging in certain cases.

  • The container-replicator now correctly enqueues container-reconciler work for sharded containers.

  • Container metadata related to sharding are now removed when no longer needed.

• S3 API improvements

  • Unsigned payloads work with v4 signatures once more.

  • Multipart upload parts may now be copied from other multipart uploads.

  • CompleteMultipartUpload requests with a Content-MD5 now work.

  • Content-Type can now be updated when copying an object.

  • Fixed v1 listings that end with a non-ASCII object name.

• Background corruption-detection improvements

  • Detect and remove invalid entries from hashes.pkl

  • When object path is not a directory, just quarantine it, rather than the whole suffix.

• Static Large Object sizes in listings for versioned containers are now more accurate.

• When refetching Static Large Object manifests, non-manifest responses are now handled better.

• Cross-account symlinks now store correct account information in container listings. This was previously fixed in 2.22.0.

• Requesting multiple ranges from a Dynamic Large Object now returns the entire object instead of incorrect data. This was previously fixed in 2.23.0.

• When making backend requests, the proxy-server now ensures query parameters are always properly quoted. Previously, the proxy would encounter an error on Python 2.7.17 if the client included non-ASCII query parameters in object requests. This was previously fixed in 2.23.0.

2.21.0

新機能

• Change the behavior of the EC reconstructor to perform a fragment rebuild to a handoff node when a primary peer responds with 507 to the REPLICATE request. This changes EC to match the existing behavior of replication when drives fail. After a rebalance of EC rings (potentially removing unmounted/failed devices), it's most IO efficient to run in handoffs_only mode to avoid unnecessary rebuilds.

• O_TMPFILE support is now detected by attempting to use it instead of looking at the kernel version. This allows older kernels with backported patches to take advantage of the O_TMPFILE functionality.

• Add slo_manifest_hook callback to allow other middlewares to impose additional constraints on or make edits to SLO manifests before being written. For example, a middleware could enforce minimum segment size or insert data segments.

• Fixed an issue with multi-region EC policies that caused the EC reconstructor to constantly attempt cross-region rebuild traffic.

• Fixed an issue where S3 API v4 signatures would not be validated against the body of the request, allowing a replay attack if request headers were captured by a malicious third party.

• Display crypto data/metadata details in swift-object-info.

• formpost can now accept a content-encoding parameter.

• Fixed an issue where multipart uploads with the S3 API would sometimes report an error despite all segments being upload successfully.

• Multipart object segments are now actually deleted when the multipart object is deleted via the S3 API.

• Swift now returns a 503 (instead of a 500) when an account auto-create fails.

• Fixed a bug where encryption would store the incorrect key metadata if the object name starts with a slash.

• Fixed an issue where an object server failure during a client download could leave an open socket between the proxy and client.

• Fixed an issue where deleted EC objects didn't have their on-disk directories cleaned up. This would cause extra resource usage on the object servers.

• Fixed issue where bulk requests using xml and expect 100-continue would return a malformed HTTP response.

• 様々な他のマイナーなバグ修正と改善。

バグ修正

• Fixed a cache invalidation issue related to GET and PUT requests to containers that would occasionally cause object PUTs to a container to 404 after the container had been successfully created.

• SLO クラウドへの POST が X-Static-Large-Object メタデータを変更できる、競合状態を削除しました。

• 消去コード付きオブジェクトへの範囲リクエストでの稀なソケットリークを修正しました。

• Fix SLO delete for accounts with non-ASCII names.

• Fixed an issue in COPY where concurrent requests may have copied the wrong data.

• Fixed time skew when using X-Delete-After.

• SLO 読み込みへの 206 Partial Content 応答で ETag ヘッダーを送信します。

• Fix SLO delete for accounts with non-ASCII names.

• Fixed an issue in COPY where concurrent requests may have copied the wrong data.

• Fixed a bug in how Swift uses eventlet that was exposed under high concurrency.

• Prevent PyKMIP's kmip_protocol logger from logging at DEBUG. Previously, some versions of PyKMIP would include all wire data when the root logger was configured to log at DEBUG; this could expose key material in logs. Only the kmip_keymaster was affected.

• Fixed an issue where a failed drive could prevent the container sharder from making progress.

• Fixed a bug in how Swift uses eventlet that was exposed under high concurrency.

2.20.0

新機能

• S3 API compatibility updates

  • Swift can now cache the S3 secret from Keystone to use for subsequent requests. This functionality is disabled by default but can be enabled by setting the secret_cache_duration in the [filter:s3token] section of the proxy server config to a number greater than 0.

  • s3api now mimics the AWS S3 behavior of periodically sending whitespace characters on a Complete Multipart Upload request to keep the connection from timing out. Note that since a request could fail after the initial 200 OK response has been sent, it is important to check the response body to determine if the request succeeded.

  • s3api now properly handles x-amz-metadata-directive headers on COPY operations.

  • s3api now uses concurrency (default 2) to handle multi-delete requests. This allows multi-delete requests to be processed much more quickly.

  • s3api now mimics some forms of AWS server-side encryption based on whether Swift's at-rest encryption functionality is enabled. Note that S3 API users are now able to know more about how the cluster is configured than they were previously, ie knowledge of encryption at-rest functionality being enabled or not.

  • s3api responses now include a '-' in multipart ETags.

    For new multipart-uploads via the S3 API, the ETag that is stored will be calculated in the same way that AWS uses. This ETag will be used in GET/HEAD responses, bucket listings, and conditional requests via the S3 API. Accessing the same object via the Swift API will use the SLO Etag; however, in JSON container listings the multipart upload etag will be exposed in a new "s3_etag" key. Previously, some S3 clients would complain about download corruption when the ETag did not have a '-'.

  • S3 ETag for SLOs now include a '-'.

    Ordinary objects in S3 use the MD5 of the object as the ETag, just like Swift. Multipart Uploads follow a different format, notably including a dash followed by the number of segments. To that end (and for S3 API requests only), SLO responses via the S3 API have a literal '-N' added on the end of the ETag.

  • The default location is now set to "us-east-1". This is more likely to be the default region that a client will try when using v4 signatures.

    Deployers with clusters that relied on the old implicit default location of "US" should explicitly set location = US in the [filter:s3api] section of proxy-server.conf before upgrading.

  • Add basic support for ?versions bucket listings. We still do not have support for toggling S3 bucket versioning, but we can at least support getting the latest versions of all objects.

• Fixed an issue with SSYNC requests to ensure that only one request can be running on a partition at a time.

• Data encryption updates

  • The kmip_keymaster middleware can now be configured directly in the proxy-server config file. The existing behavior of using an external config file is still supported.

  • Multiple keymaster middlewares are now supported. This allows migration from one key provider to another.

    Note that secret_id values must remain unique across all keymasters in a given pipeline. If they are not unique, the right-most keymaster will take precedence.

    When looking for the active root secret, only the right-most keymaster is used.

  • Prevent PyKMIP's kmip_protocol logger from logging at DEBUG. Previously, some versions of PyKMIP would include all wire data when the root logger was configured to log at DEBUG; this could expose key material in logs. Only the kmip_keymaster was affected.

• Fixed an issue where a failed drive could prevent the container sharder from making progress.

• Storage policy definitions in swift.conf can now define the diskfile to use to access objects. See the included swift.conf-sample file for a description of usage.

• The EC reconstructor will now attempt to remove empty directories immediately, while the inodes are still cached, rather than waiting until the next run.

• Added a keep_idle config option to configure KEEPIDLE time for TCP sockets. The default value is the old constant of 600.

• Add databases_per_second to the account-replicator, container-replicator, and container-sharder. This prevents them from using a full CPU core when they are not IO limited.

• Allow direct_client users to overwrite the X-Timestamp header.

• 様々な他のマイナーなバグ修正と改善。