fix(safetensors): prefer bounded framing over zlib magic

[mldangelo-oai]

Summary

• recognize a complete, bounded SafeTensors JSON header before accepting weak compression magic
• preserve SafeTensors security scanning when the little-endian header length begins with zlib bytes (78 9c)
• retain existing routing for genuine zlib payloads

Root cause

File routing checked compression magic before strong SafeTensors framing. A valid SafeTensors header length such as 0x9c78 therefore looked like zlib and bypassed the SafeTensors scanner.

Validation

• PROMPTFOO_DISABLE_TELEMETRY=1 PYTHONPATH=$PWD /Users/mdangelo/code/modelaudit/.venv/bin/python -m pytest tests/scanners/test_safetensors_scanner.py tests/utils/file/test_filetype.py -q (383 passed)
• PYTHONPATH=$PWD /Users/mdangelo/code/modelaudit/.venv/bin/python -m mypy modelaudit/utils/file/detection.py tests/scanners/test_safetensors_scanner.py tests/utils/file/test_filetype.py (Success: no issues found)
• Ruff check, Ruff format check, and git diff --check passed

The regression test demonstrates that the 78 9c fixture now reaches the SafeTensors scanner and still reports malicious metadata, while the existing compression-routing suite covers genuine zlib controls.

[github-actions]

Workflow run and artifacts

Performance Benchmarks

Compared 12 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 12 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 1.410s -> 1.403s (-0.5%).

Workload	Benchmark	Target	Size	Files	Baseline	Current	Change	Status
padded-multi-stream-upload	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload	multi_stream_padded	4.1 KiB	1	656.8us	613.6us	-6.6%	stable
warm-cache-rescan	tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan	release-candidate	547.3 KiB	32	108.02ms	103.46ms	-4.2%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex]	nested_hex	130 B	1	562.0us	582.4us	+3.6%	stable
direct-malicious-upload	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload	malicious_reduce	52 B	1	520.4us	514.6us	-1.1%	stable
suspicious-pickle-intake	tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake	suspicious-intake	183.8 KiB	4	144.47ms	142.91ms	-1.1%	stable
duplicate-heavy-registry	tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot	registry-snapshot	915.2 KiB	13	387.64ms	383.60ms	-1.0%	stable
mixed-model-repository	tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository	release-candidate	547.3 KiB	32	477.15ms	481.59ms	+0.9%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64]	nested_base64	98 B	1	547.8us	552.5us	+0.9%	stable
clean-training-checkpoint	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint	safe_large	278.2 KiB	1	108.91ms	108.00ms	-0.8%	stable
nested-payload-review	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw]	nested_raw	78 B	1	550.2us	554.3us	+0.8%	stable
single-checkpoint-preflight	tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load	single_checkpoint.pkl	183.0 KiB	1	69.98ms	70.21ms	+0.3%	stable
chunked-upload-stream	tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream	chunked_stream	278.2 KiB	1	110.90ms	110.72ms	-0.2%	stable

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 988dd28e70

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

Exact head 0472a9598d26c81c3735e2a15abb58e4e43f27ce is synced with origin/main and includes the review fixes for oversized fail-closed routing, compressed-suffix precedence, genuine-zlib control coverage, and recursive JSON handling.

@codex review

[mldangelo-oai]

@codex review

[mldangelo-oai]

Refreshed exact head a1c21368b5e56036bd87866a832ee77a056a976e with a normal merge of current origin/main. Focused routing, picklescan, lint, format, and type checks are green.

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Breezy!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

Exact current head 768adad1251e6876928b95c3fc1d8a59cb6b5ce9 includes the latest normal merge from origin/main; focused regression and static checks are green.

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Delightful!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

Exact current head baa6230dd5df7ddc0553ffdbab8809c13c74ec11 includes the latest normal merge from origin/main and the related S901 attribution change. Combined routing/attribution regressions, the 460-test routing surface, lint, format, and types are green.

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Nice work!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[mldangelo-oai]

Exact current head 5d491ac7bbb952c3b0b0e872a28ee1a4e931eeb4 includes the latest normal merge from origin/main, including the related SafeTensors empty-offset fix. All 469 routing/SafeTensors/compression tests, lint, format, and types are green.

@codex review

[chatgpt-codex-connector]

Codex Review: Didn't find any major issues. Keep them coming!

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".