fix(storage): send explicit Content-Length on every S3 request - #255
Merged
Conversation
Uploading large (chunked) files to S3-compatible storage such as GCS failed with HTTP 411 "POST requests require a Content-length header". S3::call() handed a streamed body to the transport without a Content-Length header. The cURL adapter then streams a body of unknown size with Transfer-Encoding: chunked, and emits no length at all for an empty-body POST (createMultipartUpload) — both of which S3-compatible services reject with 411. hashBody() already reads the whole (seekable) body once to sign it, so count the bytes there and set an explicit, signed content-length header for every request (0 for empty bodies). The value equals the full body size, matching what the transport sends for a size-known stream. Ports the fix from the archived utopia-php/storage PR #171 into the monorepo. Fixes appwrite/appwrite#13548
ArnabChatterjee20k
requested review from
Meldiron,
abnegate,
eldadfux and
loks0n
as code owners
September 10, 2026 12:21
Contributor
|
Regression coverage for the HTTP 411 fix, driving the real S3::call() with a capturing PSR-18 client: - a streamed write sends Content-Length equal to the exact body size (never Transfer-Encoding: chunked) - an empty-body multipart POST (createMultipartUpload) sends Content-Length: 0 Both requests are what GCS rejected before the fix.
CI's storage check failed on two lint rules in the new regression test: pint's single_line_empty_body (collapse the empty CapturingClient constructor body) and rector's AssertEmptyNullableObjectToAssertInstanceof (assertNotNull on a nullable object -> assertInstanceOf). Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UQ1CyiMrnx7CNQDi85gxKa
The empty-body Content-Length test drove the protected createMultipartUpload() through a test-only subclass, coupling it to an implementation helper. Multi-chunk prepare() reaches the same multipart-initiation POST through the public API, so drive it that way and drop the ContentLengthS3 subclass. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UQ1CyiMrnx7CNQDi85gxKa
abnegate
approved these changes
Sep 10, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Uploading large (chunked) files to S3-compatible storage — reported against GCS — fails with:
Reported downstream in appwrite/appwrite#13548 (self-hosted 2.0, files 15 MB–550 MB).
Cause
S3::call()handed a streamed body to theclienttransport without aContent-Lengthheader. The cURL adapter then:StreamInterface::getSize()returnsnull) withTransfer-Encoding: chunked, andcreateMultipartUpload).Many S3-compatible services (GCS in particular) reject both with 411.
Fix
hashBody()already reads the whole (seekable) body once to sign it, so it now also counts the bytes and returns the length.call()sets an explicit, signedcontent-lengthheader on every request (0for empty bodies). The value is the full body size, so it matches what the transport already sends for a size-known stream — no behavioural change there, correct headers for the streamed / empty-body cases. No new reads of the body are introduced.Ports the fix from the archived standalone
utopia-php/storagePR #171 into the monorepo (packages/storage).Testing
php -lclean; singlehashBodycaller updated.Fixes appwrite/appwrite#13548