Fix Hatch takeover leaving direct-ref permission (#674)

[Mikola Lysenko (mikolalysenko)]

LLM Description written by Claude Code:claude-opus-5-5

Fixes #674

Summary

After a hosted→vendored takeover of two or more Hatch packages, rollback, remove and vendor --revert used to leave allow-direct-references = true (plus empty [tool] / [tool.hatch] tables) behind. That silently turned off Hatchling's direct-reference guard and kept the pyproject dirty. With this change the pyproject comes back byte-exact.

Root cause

The takeover in commands/vendor.rs calls restore_upstream with one hosted pin per candidate inside the vendoring loop. When the first Hatch package is vendored, the other packages' hosted direct references are still live, and so is the permission hosted mode added for them. The vendored hatch_permission record snapshotted that state as its original. Later packages clone the same record. So once the last reference was unwired, the revert "restored" a permission the user never had.

Fix

• vendor/pypi_hatch.rs::wire: a new permission record now checks the project for a direct reference the vendored ledger doesn't own (anything that isn't a {root:uri}/.socket/vendor/… reference). If there is one, the permission it finds is held for those references, so the record's original is the document with that permission removed, along with any tables that leaves empty. The hosted unwind already applies this rule: it drops the permission once no project direct reference is left. Because the decision is made once when the record is created, every package that clones the record inherits it, whatever the revert order.
• utils/hatch.rs: drop_direct_reference_permission and permission_keys now live here. The hosted unwind (redirect/upstream/pypi.rs) and the vendored ledger share them, so both lanes produce the same bytes.
• revert is unchanged. A permission the user set themselves (with no non-vendored direct reference live at wiring time) is still recorded and restored verbatim.

I didn't restructure the generic per-pin takeover loop. It carries many per-candidate refusal gates, and the pnpm (#636) and uv (#670) leftovers have different mechanisms; #672 covers those.

Tests (red → green)

Test	Covers	Without fix	With fix
vendor::pypi_hatch::tests::takeover_through_hosted_unwind_reverts_byte_exact	#674 through the real hosted rewrite, discover_patched_refs, per-pin restore_upstream and vendored wire, reverted in both orders	FAILED (leftover [tool]…allow-direct-references = true)	ok
vendor::pypi_hatch::tests::takeover_snapshot_of_hosted_permission_is_not_restored	#674 in both revert orders, with the permission in pyproject and in hatch.toml [metadata] (sibling key kept)	FAILED (same leftover)	ok
vendor::pypi_hatch::tests::user_permission_survives_vendored_revert	Guard: a user-set permission (inline and in hatch.toml) survives a two-package vendored revert	ok	ok

Per-issue checklist:

• Hatch hosted→vendored takeover with two or more patches leaves allow-direct-references = true (plus empty [tool]/[tool.hatch] tables) behind after rollback or remove, disabling Hatchling's direct-reference guard #674: hosted→vendored with 2 patches, then rollback / remove / vendor --revert (all three go through pypi_hatch::revert) → byte-exact pyproject. Covered by the first two tests above.

Local validation

• cargo clippy --workspace --all-features -- -D warnings: clean.
• cargo test -p socket-patch-core --lib -- pypi_hatch hatch upstream: 87 passed.
• cargo test --workspace --all-features --no-fail-fast: everything passes except 12 tests that simulate write or removal failures with chmod / read-only dirs. The sandbox runs as root, which bypasses those permissions. None of them touch Hatch code. CI runs unprivileged.
• SOCKET_PATCH_HATCH_E2E_REQUIRED=1 SOCKET_PATCH_HATCH_E2E_VERSION=1.18.1 cargo test -p socket-patch-cli --all-features --test e2e_vex_build -- hatch:: --ignored: 4 passed.
• rustfmt --check is clean on the touched files. Note that cargo fmt --all -- --check fails on main itself (about 130 files, from 045d7ec), and CI doesn't run rustfmt. I kept those unrelated reformats out of this PR; only one existing line in utils/hatch.rs was reflowed.

Behaviour note

If a user had both their own direct reference and allow-direct-references = true before vendoring, the permission is kept as long as any project direct reference remains. It is dropped only once none is left, which is what hosted mode already does.

🤖 Generated with Claude Code

---

Note

Medium Risk
Changes vendored Hatch permission ledger semantics and shared TOML cleanup used on revert; well-covered by new tests but affects dependency wiring rollback paths.

Overview
Fixes #674: after a hosted→vendored Hatch takeover with multiple packages, revert/rollback no longer leaves allow-direct-references = true (and empty [tool] / [tool.hatch] tables) when the project should return to its pre-hosted bytes.

Vendored wiring now treats hatch_permission “original” as the file without the direct-reference permission when non-vendored project direct refs are still live at wire time (e.g. another package’s hosted pin during per-pin takeover). User-set permissions are still snapshotted verbatim.

Shared Hatch helpers — permission_keys and drop_direct_reference_permission — moved into utils/hatch.rs so hosted unwind (restore_hatch) and the vendored ledger apply the same cleanup rule.

Adds regression tests for takeover/revert order, full hosted→vendor lane, and user-owned permissions in pyproject.toml / hatch.toml.

^{Reviewed by Cursor Bugbot for commit d3076b5. Configure here.}

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

BugBot review

---

Generated by Claude Code

[cursor]

✅ Bugbot reviewed your changes and found no new issues!

Comment @cursor review or bugbot run to trigger another review on this PR

^{Reviewed by Cursor Bugbot for commit d3076b5. Configure here.}

[Mikola Lysenko (mikolalysenko)]

Ready for review — head d3076b5d00100d9e268df66c4e6d63f39ee18ef6.

• CI: 97/97 checks green on the head (3 skipped by path filters). Branch is up to date with main (0 behind).
• Bugbot: reviewed d3076b5 and found no issues. No review threads are unresolved.
• Reviewer focus: vendor/pypi_hatch.rs::wire now decides when the permission record is created whether a live non-vendored direct reference owns allow-direct-references. The permission helpers moved into utils/hatch.rs, so the hosted unwind and the vendored revert produce the same bytes. A permission the user set themselves is still restored verbatim (see user_permission_survives_vendored_revert).

---

Generated by Claude Code

[Mikola Lysenko (mikolalysenko)]

Reviewed d3076b5d00100d9e268df66c4e6d63f39ee18ef6 against main 045d7ec783d788bf3c5a1310724b51e09fb6505d: ready to merge as-is from this review. No actionable issue introduced by this diff.

The first vendored permission record now excludes permission held by remaining hosted references. Shared ownership survives selective removal, and final cleanup restores Hatchling’s direct-reference guard. The extracted hosted cleanup helper preserves its existing behavior.

Validation:

• 163 repository tests passed: Hatch/upstream/TOML restoration, PyPI mode migration, and Hatch VEX coverage.
• One additional probe passed 8 scenarios across both removal orders, inline/external metadata, and LF/CRLF. It verifies dry-run preservation and transactional refusal when the permission changes.
• 8 selected native lifecycle cases passed with Hatch 1.18.1 / Hatchling 1.32.4, including 3 fresh installs verifying patched modules, byte-exact restoration, retained user references/settings, and the expected post-restore guard decisions.
• 482 successful checks, 7 skipped; all 13 workflows terminal good. Exact-head Bugbot is clean, with no unresolved threads. Touched-file formatting and core production Clippy passed with the existing macOS unused_variables warning allowance.

A separate exploratory case found existing hosted rewriting removes a comment attached to the permission value. This reproduced before any vendoring with both this binary and a control whose Hatch code matches main; it is preserved separately from the eight passing cases and is not introduced here.