fix(safetensors): accept empty tensor offsets - #1607
Conversation
Performance BenchmarksCompared
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 84cf296b57
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…pty-safetensor-offsets
Match upstream bit-width overflow and native offset bounds, and keep overflow regressions compatible with the supported safetensors version range.
|
@codex review Please review exact head |
|
Codex Review: Didn't find any major issues. Nice work! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…pty-safetensor-offsets
…pty-safetensor-offsets
|
@codex review\n\nPlease review exact head |
|
@codex review Please review exact head |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: d632d62120
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
@codex review Please review exact head |
|
Codex Review: Didn't find any major issues. Breezy! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…pty-safetensor-offsets
|
@codex review Please review exact head |
|
Codex Review: Didn't find any major issues. Keep it up! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
…pty-safetensor-offsets
|
@codex review Please review exact head |
|
Codex Review: Didn't find any major issues. Keep it up! ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Summary
usizeoffset and bit-size overflow validationsafetensors.numpyround-trip regression, malformed zero-byte negative control, deterministic equal-start ordering coverage, and simulated native-width regressionsReal-world reproduction
The AgentBox Hugging Face audit found the same false-positive critical finding in three pinned FP8 artifacts from:
Comfy-Org/Wan_2.2_ComfyUI_Repackaged@f97505f0d38bea4897c970db66cb5f97f73676deThe
scaled_fp8tensor has shape[0]and equal in-bounds offsets. The pinned control artifact is 14,296,064,656 bytes with SHA-256aa2f6b6f4cfc8a75a273a075db31730530f93320a996b153b8825205c3a3c498. Its exact 181,600-byte header prefix has SHA-256fa2be54fd2d897749cd61fcadfdefead28fe958b9376d8d5ec91bd68289da031.The SafeTensors format specification explicitly permits tensors with a zero dimension and defines stored byte size as
END - BEGIN.After this change, a sparse exact-header replay passes offset and size validation with no issues. A tensor with shape
[1]and offsets[0, 0]still fails critically because its expected four bytes do not match the zero-byte range.Validation
67 passedintests/scanners/test_safetensors_scanner.pywith bothsafetensors==0.7.0and the supported floorsafetensors==0.4.0418 files)473 source files)12,890 passed, 381 skippedbefore two local baseline/environment failures: the read-only sandbox blocks/home/dev-user/.modelaudit(the cache test passes with writableHOME), and a merged-main cached-shard alias test fails independently of the tensor-entry delta; fresh CI is the authoritative cross-platform run