Detect layer compression by content, not media type

[mattpollock]

What

extractLayer chose the decompressor from the layer's declared media type: only types.OCILayerZStd was treated as zstd, and everything else fell through to gzip. That breaks for zstd layers labeled with a non-OCI media type — most commonly BuildKit pushing with Docker schema2, which emits application/vnd.docker.image.rootfs.diff.tar.zstd. The media-type switch routes those to gzip.NewReader, so in fails with:

write rootfs: extract image: gzip: invalid header

This is increasingly common as people adopt zstd-layer base images (e.g. hardened/minimal bases) and push them through standard docker/build-push-action without forcing OCI media types.

Fix

Detect the compression from the layer's own leading magic bytes instead of trusting the media-type label:

• zstd -> 28 b5 2f fd
• gzip -> 1f 8b
• otherwise -> read as an uncompressed tar

So gzip, zstd, and uncompressed layers all unpack regardless of whether they're labeled OCI or Docker media types. mediaType is kept in the signature for compatibility but no longer drives the decision.

Repro

# zstd layers under Docker schema2 media types (oci-mediatypes=false)
docker buildx build --output type=image,name=$IMG,push=true,compression=zstd,force-compression=true,oci-mediatypes=false .

get of $IMG fails with gzip: invalid header before this change; succeeds after.

Notes

• No behavior change for existing gzip / OCI-zstd images (same decompressors, just selected by content).
• Single file (commands/rootfs.go). Happy to add a regression test fixture (a Docker-schema2 zstd image alongside the existing OCI-zstd one in testdata/setup-images) if you'd like it in this PR.

[linux-foundation-easycla]

• ❌ - login: @mattpollock / name: Matthew Pollock. The commit (0c8261b) is not authorized under a signed CLA. Please click here to be authorized. For further assistance with EasyCLA, please visit our EasyCLA portal and chat with our support bot.

[taylorsilva]

Thanks for the PR! This LGTM, just make one final clean up of the func signature and then I'll merge. How annoying is it to have standards and then NOT have them followed 😭

P.S. Can you complete the EasyCLA too, please and TY 🙏

[taylorsilva]

This func is private and only used in one place. Let's just update the signature and update where it's called, which should be higher up in this file.

Suggested change

	// uncompressed tar regardless of how the layer is labeled. mediaType is kept
	// for signature compatibility but no longer drives the decision.
	func compressionReader(layer io.ReadCloser, mediaType types.MediaType, bar *mpb.Bar) (*tar.Reader, compressReaderClose, error) {
	// uncompressed tar regardless of how the layer is labeled. mediaType is kept
	// for signature compatibility but no longer drives the decision.
	func compressionReader(layer io.ReadCloser, bar *mpb.Bar) (*tar.Reader, compressReaderClose, error) {

And update here:

registry-image-resource/commands/rootfs.go

Lines 83 to 91 in 1627112

	mediaType, err := layer.MediaType()
	if err != nil {
	return err
	}
	tr, crc, err := compressionReader(lyr, mediaType, bar)
	if err != nil {
	return err
	}

[taylorsilva]

nit: for easy lookup reference in the future link to https://en.wikipedia.org/wiki/List_of_file_signatures

Suggested change

	// Layer compression magic numbers.
	// Layer compression magic numbers. Can find more at:
	// https://en.wikipedia.org/wiki/List_of_file_signatures