fix: dispatch logical model directory scanners by mldangelo-oai · Pull Request #1609 · promptfoo/modelaudit

mldangelo-oai · 2026-06-10T02:55:56Z

Summary

dispatch recognized logical model directories only after the normal child walk has established path containment and size/time budgets
bind directory-owner findings to pre/post SHA-256 snapshots and fail closed on source mutation, incomplete snapshots, traversal, or owner exceptions
scope owner snapshots to files the logical scanner can actually read, avoiding duplicate hashing of opaque SavedModel variable shards
tighten Orbax/JAX directory routing so large ordinary directories, generic metadata.json, and Hugging Face model_index.json do not claim JAX ownership
make unexpected JAX failures operational and close the SavedModel extended-probe stale-size race
harden core hashing against symlink swaps and non-regular files

Security properties

external marker symlinks are rejected before owner dispatch
max_file_size, max_total_size, and remaining timeout apply before logical-directory analysis
owner evidence cannot be reported against a different final child hash without an explicit inconclusive operational result
SavedModel assets that change during bounded inspection fail closed
malicious Orbax restore functions and padded SavedModel pickle controls remain detected

Validation

558 passed: JAX, SavedModel, scanner registry, base scanner, and regular hash suites
7 passed: focused owner containment, budget, TOCTOU, dispatch, and no-rehash core regressions
422 passed: full JAX, SavedModel, and registry suites after routing/probe hardening
Ruff check and format: clean for every changed Python file
targeted mypy: clean for all changed source modules
git diff --check: clean

Fresh exact-head independent AgentBox review, Codex Security review, and GitHub CI are in progress for b4e608050f6c9d82e1090eaf2153cb90803a0a3f.

github-actions · 2026-06-10T02:58:07Z

Workflow run and artifacts

Performance Benchmarks

Compared 12 shared benchmarks with a regression threshold of 15%.
Status: 0 regressions, 0 improved, 12 stable, 0 new, 0 missing.
Aggregate shared-benchmark median: 1.463s -> 1.469s (+0.3%).

Workload	Benchmark	Target	Size	Files	Baseline	Current	Change	Status
`direct-malicious-upload`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_direct_malicious_upload`	`malicious_reduce`	52 B	1	466.8us	455.6us	-2.4%	stable
`suspicious-pickle-intake`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_suspicious_pickle_intake`	`suspicious-intake`	183.8 KiB	4	142.77ms	145.32ms	+1.8%	stable
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_hex]`	`nested_hex`	130 B	1	554.8us	561.3us	+1.2%	stable
`warm-cache-rescan`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_warm_cached_repository_rescan`	`release-candidate`	547.3 KiB	32	113.47ms	114.73ms	+1.1%	stable
`single-checkpoint-preflight`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_single_checkpoint_before_load`	`single_checkpoint.pkl`	183.0 KiB	1	75.17ms	74.34ms	-1.1%	stable
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_raw]`	`nested_raw`	78 B	1	524.2us	519.1us	-1.0%	stable
`chunked-upload-stream`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_chunked_upload_stream`	`chunked_stream`	278.2 KiB	1	116.29ms	115.25ms	-0.9%	stable
`mixed-model-repository`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_release_candidate_repository`	`release-candidate`	547.3 KiB	32	487.80ms	490.73ms	+0.6%	stable
`padded-multi-stream-upload`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_padded_multi_stream_upload`	`multi_stream_padded`	4.1 KiB	1	587.4us	584.6us	-0.5%	stable
`clean-training-checkpoint`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_clean_training_checkpoint`	`safe_large`	278.2 KiB	1	113.45ms	112.96ms	-0.4%	stable
`duplicate-heavy-registry`	`tests/benchmarks/test_scan_benchmarks.py::test_scan_duplicate_registry_snapshot`	`registry-snapshot`	915.2 KiB	13	411.89ms	412.54ms	+0.2%	stable
`nested-payload-review`	`tests/benchmarks/test_picklescan_benchmarks.py::test_picklescan_nested_payload_review[nested_base64]`	`nested_base64`	98 B	1	518.2us	518.8us	+0.1%	stable

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 5c044d0347

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4614a78124

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mldangelo-oai · 2026-06-10T17:13:36Z

@codex review

Please review exact head 9e24ca41a3af78e74e459ffeb2aeb395b7070d99, especially owner-source traversal scope, non-regular entry handling, and malformed Orbax routing false positives/false negatives.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 9e24ca41a3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…rectory-scanner-dispatch

…rectory-scanner-dispatch # Conflicts: # modelaudit/utils/file/detection.py

mldangelo-oai · 2026-06-10T23:53:11Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 22b8bb0051

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

mldangelo-oai · 2026-06-11T00:09:42Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: eb4e95659b

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

mldangelo-oai · 2026-06-11T00:34:05Z

@codex review

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 2ef788535c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…covery-20260611

mldangelo-oai · 2026-06-11T01:59:58Z

@codex review

Exact head: dca5b7b1fd459ab34e63868adfffc16df3d74139

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dca5b7b1fd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: bf8f570c97

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts: # modelaudit/scanner_registry_metadata.py # modelaudit/scanners/jax_checkpoint_scanner.py # modelaudit/utils/file/detection.py # tests/scanners/test_scanner_registry.py

# Conflicts: # modelaudit/core.py

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 96ff330eaf

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 42e25635f7

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

# Conflicts: # modelaudit/core.py

fix: dispatch logical model directory scanners

5c044d0

chatgpt-codex-connector Bot reviewed Jun 10, 2026

View reviewed changes

Comment thread modelaudit/core.py Outdated

Comment thread modelaudit/core.py Outdated

Comment thread modelaudit/scanner_registry_metadata.py

fix: harden logical directory scanner dispatch

4614a78

github-code-quality Bot found potential problems Jun 10, 2026

View reviewed changes

Comment thread modelaudit/scanners/tf_savedmodel_scanner.py Fixed

chatgpt-codex-connector Bot reviewed Jun 10, 2026

View reviewed changes

Comment thread modelaudit/core.py Outdated

Comment thread modelaudit/core.py Outdated

Comment thread modelaudit/scanners/jax_checkpoint_scanner.py Outdated

mldangelo-oai added 2 commits June 9, 2026 20:44

test: type directory owner budget cases

b4e6080

fix: harden directory owner dispatch

9e24ca4

mldangelo-oai enabled auto-merge (squash) June 10, 2026 17:14

chatgpt-codex-connector Bot reviewed Jun 10, 2026

View reviewed changes

Comment thread modelaudit/core.py Outdated

Comment thread modelaudit/core.py Outdated

Comment thread modelaudit/scanners/jax_checkpoint_scanner.py Outdated

mldangelo-oai disabled auto-merge June 10, 2026 17:24

mldangelo-oai added 3 commits June 10, 2026 21:32

Merge remote-tracking branch 'origin/main' into mdangelo/codex/fix-di…

d29928b

…rectory-scanner-dispatch

fix: harden directory owner source snapshots

96849fe

Merge remote-tracking branch 'origin/main' into mdangelo/codex/fix-di…

22b8bb0

…rectory-scanner-dispatch # Conflicts: # modelaudit/utils/file/detection.py

github-code-quality Bot found potential problems Jun 10, 2026

View reviewed changes

Comment thread modelaudit/scanners/tf_savedmodel_scanner.py Fixed

chatgpt-codex-connector Bot reviewed Jun 11, 2026

View reviewed changes

Comment thread modelaudit/utils/file/detection.py

fix: clean savedmodel change fallback

eb4e956

chatgpt-codex-connector Bot reviewed Jun 11, 2026

View reviewed changes

Comment thread modelaudit/core.py

fix: refine jax json probe routing

2ef7885

chatgpt-codex-connector Bot reviewed Jun 11, 2026

View reviewed changes

Comment thread modelaudit/scanners/tf_savedmodel_scanner.py Outdated

mldangelo-oai added 2 commits June 10, 2026 18:55

fix: finish directory scanner review repairs

bbf7eac

Merge remote-tracking branch 'origin/main' into codex/pr1609-local-re…

dca5b7b

…covery-20260611

github-code-quality Bot found potential problems Jun 11, 2026

View reviewed changes

Comment thread modelaudit/core.py Fixed

chatgpt-codex-connector Bot reviewed Jun 11, 2026

View reviewed changes

Comment thread modelaudit/scanners/tf_savedmodel_scanner.py Outdated

Comment thread modelaudit/scanners/jax_checkpoint_scanner.py Outdated

Comment thread modelaudit/core.py Outdated

fix: close directory owner review gaps

8ddaf0c

chatgpt-codex-connector Bot reviewed Jun 11, 2026

View reviewed changes

Comment thread modelaudit/core.py Outdated

mldangelo-oai added 10 commits June 11, 2026 17:20

fix: prune jax owner namespace descent

82929c8

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

6ebf1e2

fix: preserve executable openvino sidecars

1177f77

fix: preserve streaming openvino sidecars

cba0e77

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

6fa2125

fix: count streaming openvino sidecars once

e162440

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

00f8a01

fix: trust windows hf owner aliases

a5a38ac

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

1e8da4a

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

7e177be

github-code-quality Bot found potential problems Jun 11, 2026

View reviewed changes

Comment thread tests/test_huggingface_symlinks.py Fixed

mldangelo-oai added 8 commits June 11, 2026 22:03

fix: report hf owner special entries precisely

3ba998b

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

37ef38f

fix: fallback to hf owner raw link targets

7f11a0f

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

87d2cd5

test: use one core import style

c239028

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

a668594

# Conflicts: # modelaudit/scanner_registry_metadata.py # modelaudit/scanners/jax_checkpoint_scanner.py # modelaudit/utils/file/detection.py # tests/scanners/test_scanner_registry.py

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

96ff330

# Conflicts: # modelaudit/core.py

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

cd4fcf8

chatgpt-codex-connector Bot reviewed Jun 12, 2026

View reviewed changes

Comment thread modelaudit/scanners/jax_checkpoint_scanner.py Outdated

Comment thread modelaudit/scanners/jax_checkpoint_scanner.py Outdated

mldangelo-oai added 3 commits June 12, 2026 01:27

fix: preserve orbax metadata supplemental scan

5283bb3

fix: fail closed on ambiguous orbax routing

e1cdef3

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

42e2563

chatgpt-codex-connector Bot reviewed Jun 12, 2026

View reviewed changes

Comment thread modelaudit/scanners/jax_checkpoint_scanner.py Outdated

mldangelo-oai added 3 commits June 12, 2026 02:55

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

df50aa9

# Conflicts: # modelaudit/core.py

fix: avoid jax owner routing for ordinary large dirs

340a130

Merge remote-tracking branch 'origin/main' into campaign/pr-1609

349563e

mldangelo-oai merged commit 45532e3 into main Jun 12, 2026
29 checks passed

mldangelo-oai deleted the mdangelo/codex/fix-directory-scanner-dispatch branch June 12, 2026 04:14

Conversation

mldangelo-oai commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Summary

Security properties

Validation

Uh oh!

github-actions Bot commented Jun 10, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance Benchmarks

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mldangelo-oai commented Jun 10, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mldangelo-oai commented Jun 10, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

mldangelo-oai commented Jun 11, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

mldangelo-oai commented Jun 11, 2026

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

mldangelo-oai commented Jun 11, 2026

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

chatgpt-codex-connector Bot left a comment

mldangelo-oai commented Jun 10, 2026 •

edited

Loading

github-actions Bot commented Jun 10, 2026 •

edited

Loading