HDDS-15515. Support object Content-Type end-to-end in S3 Gateway#10472
Conversation
The S3 Gateway never persisted an object's Content-Type: it was only forwarded
to HeaderPreprocessor for SigV4 signing, GET did not set it from the object and
HEAD hard-coded binary/octet-stream, so CopyObject could not preserve/replace it.
Store the request Content-Type in the OM key metadata (mirroring the existing
ETag-in-metadata convention), return it on GET/HEAD (default binary/octet-stream),
and propagate it on CopyObject (COPY keeps the source value, REPLACE uses the
request value). A user-supplied x-amz-meta-content-type is remapped to avoid
colliding with the stored Content-Type. Fixes s3-tests
test_object_copy_{retaining,replacing}_metadata and copy_verify_contenttype.
|
Cc @peterxcli |
|
Thanks @rich7420 for finding out the issue. I got similar root cause for test case failure in MINT test. Didn't know about your patch, I have marked my jira as duplicate https://issues.apache.org/jira/browse/HDDS-15591. Will review the patch. Test failure : |
|
no problem, thanks @Gargi-jais11 to mention that |
|
Add a testcase to |
…date reserved-key remap; add MPU/HEAD tests - GET no longer reflects the request Content-Type header; it is based on the stored object Content-Type with response-content-type still overriding. - Consolidate the ETag/Content-Type reserved-key remap+rebuild into a single RESERVED_METADATA_KEYS table and drop the duplicate CONTENT_TYPE constant. - Add tests: HEAD x-amz-meta-content-type round-trip and CreateMultipartUpload Content-Type storage.
|
@Gargi-jais11 thanks for your comments! |
Gargi-jais11
left a comment
There was a problem hiding this comment.
Thanks @rich7420 for updating the PR. LGTM! , +1.
|
@Gargi-jais11 and @chungen0126 |
What changes were proposed in this pull request?
The S3 Gateway never persisted an object's
Content-Type. The requestContent-Typewas only forwarded toHeaderPreprocessorfor SigV4 signing(
X-Ozone-Original-Content-Type) and was never stored in the OM key metadata;GetObjectdid not set it from the object andHeadObjecthard-codedbinary/octet-stream. Because the value is never stored,CopyObject/UploadPartCopycannot preserve or replace it, so copied (and plain) objectsconsistently default to octet-stream.
This is an end-to-end gap, not a copy-only issue. This PR adds object
Content-Typesupport across the read/write/copy paths, aligning with AWS S3:Content-Typeinthe OM key metadata, mirroring the existing ETag-in-metadata convention.
Content-Type, defaulting tobinary/octet-streamwhen absent (theresponse-content-typequeryparameter still overrides).
COPY(default) preserves the sourceobject's
Content-Type(it rides along in the copied source metadata);REPLACEapplies theContent-Typefrom the copy request.x-amz-meta-content-typeis remapped to a dedicated key soit does not collide with the object's stored
Content-Type. The reserved-keyremap/rebuild that previously existed only for ETag is now factored into a
shared helper and reused for both.
Before vs after (response):
response-content-typebinary/octet-streamWhat is the link to the Apache JIRA
https://issues.apache.org/jira/browse/HDDS-15515
How was this patch tested?
https://github.com/rich7420/ozone/actions/runs/27210927693