Skip to content

fix(gqt): watch the git files a worktree actually has in the build script - #1180

Open
ragnorc wants to merge 1 commit into
ModernRelay:mainfrom
ragnorc:fix/gqt-build-script-worktree-git-files
Open

ragnorc wants to merge 1 commit into
ModernRelay:mainfrom
ragnorc:fix/gqt-build-script-worktree-git-files

Conversation

@ragnorc

@ragnorc ragnorc commented Oct 26, 2026

Copy link
Copy Markdown
Contributor

In a git worktree, omnigraph-gqt's build script reran on every build. It watched <root>/.git/index and <root>/.git/HEAD, but in a worktree .git is a file, so neither path exists, and Cargo treats a missing rerun-if-changed path as changed on every build. Each cargo test -p omnigraph-gqt then recompiled the crate and relinked its 350 MB worker binary. On macOS the first launch of a newly linked binary waits while XProtect scans it, so local GQT cases failed preflight against their 10-second budget, including cases unrelated to the change under test.

The build script now asks git for each control file's path (git rev-parse --git-path, joined to the repository root when git prints it relative): HEAD, the index, packed-refs, and the ref HEAD names when it is on a branch. It watches only files that exist. In a worktree these resolve to the worktree's private HEAD and index and to the common directory's refs. A plain clone gets the same two files as before plus the branch ref and packed-refs, so a commit that leaves the index unchanged also refreshes GQT_SOURCE_REVISION. omnigraph-bench/build.rs already resolves the git directory the same way, and no other build script watches git files.

Measured in a worktree with cargo check -p omnigraph-gqt --tests --locked after a first full check. With the old script, each of two further runs compiled omnigraph-gqt again. With this one, both compiled nothing; the second of the original pair finished in 1.16 s.

…ript

The build script watched <root>/.git/index and <root>/.git/HEAD. In a
worktree .git is a file, so neither exists, and Cargo reruns a script
whose watched path is missing on every build, recompiling the crate and
relinking its worker binary each time. Ask git for each control file's
path (HEAD, the index, packed-refs, the current branch's ref) and watch
only files that exist.
@ragnorc
ragnorc requested a review from azimafroozeh as a code owner October 26, 2026 13:03

ragnorc commented Oct 26, 2026

Copy link
Copy Markdown
Contributor Author

Recommendation: revise.

The worktree rebuild loop is real, and this script is the right place to stop it. One ref update still leaves GQT_SOURCE_REVISION on the old commit. Close that hole before merge. The rest of the change holds.

Why

GQT_SOURCE_REVISION is the git commit baked into the GQT worker. RFC 0045 counts that commit as build identity. The worker checks it against the parent process.

In a linked worktree, .git is a file. The merge-base script watches <root>/.git/index and <root>/.git/HEAD. Those paths do not exist. Cargo 1.97.1 treats a missing rerun-if-changed path as dirty on every build. The message is the file ... is missing. Each check compiles the crate again. On macOS the new worker binary waits on XProtect, and a 10 second GQT preflight can fail for an unrelated case.

The same .git file shape is a submodule. The old script rebuilds there too.

What and how

The script asks git for each control path (git rev-parse --git-path): HEAD, index, packed-refs, and the name from git symbolic-ref -q HEAD when HEAD is a branch. It joins that path to the repository root. Path::join keeps an absolute path as absolute, which is what git prints for a worktree. It emits rerun-if-changed only when the path is a file.

That filter is required. A missing path brings the rebuild loop back.

What I measured

Toolchain: cargo 1.97.1, git 2.43.0. Merge base da1c46368c33876d61615add0ceb6778f9ce2b58. Head b34eb30a7d6ee33b16fb90869ad2dc28617fca3c. The build.rs blob is 7d22276813a74c630f52cd22b825942ae3a2f5dd.

GQT cannot express a build-script fingerprint. I did not run the corpus. I ran cargo check -p omnigraph-gqt --offline -v on the exact script bytes.

A linked worktree of this repository at the PR commit, with the PR build.rs:

  • Check 1 ran the script. It watched the worktree-private HEAD and index (/workspace/.git/worktrees/<name>/), the common packed-refs, and the loose branch ref. GQT_SOURCE_REVISION was b34eb30a7d6ee33b16fb90869ad2dc28617fca3c. Zero watched paths were missing.
  • Checks 2 and 3 were Fresh omnigraph-gqt (0.176 s and 0.166 s). The build script did not run.
  • touch on the private HEAD made the next check dirty, then the check after that was fresh. Cargo follows that absolute path.

The merge-base script in that same worktree: checks 2 and 3 were dirty with the file crates/omnigraph-gqt/../../.git/index is missing, and each one compiled the crate again. <root>/.git/index and <root>/.git/HEAD do not exist.

A plain clone (.git is a directory), same script bytes: old and new are both fresh on the second and third check. The bug is the gitfile layout, not a normal checkout.

Also confirmed with the same bytes:

  • Branch switch (other to main): dirty because HEAD changed. The baked revision matches the new commit. The watched ref changes from refs/heads/other to refs/heads/main.
  • Detached HEAD: symbolic-ref adds no branch ref. The second check is fresh. An empty commit updates the HEAD file, the next check reruns, and the baked revision is the new commit.
  • Loose ref plus git reset --soft HEAD~1: the new script reruns because refs/heads/main changed, and the baked revision matches. The old script stays Fresh and keeps the pre-reset commit. This part of the PR is a real fix.
  • Submodule (.git is a gitfile): the new script is fresh on checks 2 and 3 and watches .git/modules/<name>/. The old script is dirty on every check. Two paths are missing.
  • git commit --allow-empty after git pack-refs --all: the index bytes stay the same and the index mtime changes. Cargo reruns because of the index. The baked revision updates. A normal commit is not the hole below.
  • git ls-files as the script runs it does not rewrite the index bytes (fsmonitor and untrackedCache were on).

Blocking

After git pack-refs --all, the loose branch ref is gone. The script watches HEAD, the index, and packed-refs. It does not watch refs/heads/<branch>, because that path is not a file.

git reset --soft HEAD~1 then creates the loose ref again. It does not change HEAD, the index bytes, the index mtime, or packed-refs. The next cargo check is Fresh. The baked revision stays on the old commit (aec33abb9fa0fa08c2e094ef409ce4c9a35d0bb6) while git rev-parse HEAD is c64ee22ba358aa14d0161e2c152c450afa89099e.

git gc packs refs. git reset --soft is how people undo a commit before they run tests again. The recorded build identity is then wrong. The worker check still passes, because the parent and the worker share one stale binary. RFC 0045 uses this revision as reproduction identity.

omnigraph-bench already has the rule that covers this, in emit_reference_rerun_path: when the loose ref is absent, watch the nearest existing parent directory, and stop before the git directory. Do that for the branch ref only. Do not do it for a missing packed-refs or a missing index, and do not watch the git directory. A directory watch reruns when the loose file appears (cargo 1.97.1: creating refs/heads/main under a watched refs/heads is dirty). The next build sees the file and watches the file.

I ran that directory fallback on the same pack-refs plus reset --soft sequence. The check was dirty because refs/heads changed, and the baked revision matched git rev-parse HEAD.

While the directory is the watch, another ref in that directory also causes a rebuild. That is one extra rebuild. The following build watches the single file. Bench already accepts that cost. It is cheaper than a false commit id, and it does not restore the every-build loop.

Optional

GIT_DIR still selects the repository. With GIT_DIR pointed at a second clone, this script baked that clone's HEAD and watched that clone's HEAD, index, packed-refs, and branch ref. The merge-base git rev-parse HEAD already follows GIT_DIR. Bench clears GIT_* and passes --git-dir and --work-tree (sanitized_git_command). Use that when this script next grows a git call. I would not block the worktree fix on it.

Keep one rule for "which git files invalidate a baked commit." Bench and this script already disagree. A third copy will disagree again. Put the directory rule in this file now. Move emit_git_rerun_paths into one shared file when the next caller appears. A new crate is more surface than this bug needs.

A test that reads the build-script output and asserts every rerun-if-changed path exists would match build_provenance_watches_only_paths_that_exist in omnigraph-bench. On a normal CI checkout the old .git/index path exists, so that test would not have failed for this bug. It still catches a later missing path in a worktree. The packed-ref case needs a disposable repository. Do not pack-refs the developer's checkout from a unit test.

Liability

The change removes a rebuild that does not depend on source edits, in every layout where .git is a file. It also makes git reset --soft refresh the revision when the loose ref already exists. The old script does not.

It adds a second git-watch policy. That policy is silent when the branch ref is packed and the next update creates the loose file without touching the index. Five more build scripts of this shape would be five path lists. The durable owner is the bench rule: an existing file, or the nearest existing directory for a branch ref, never a missing path, never the whole git dir.

Alignment

PRs 1174, 1176, 1177, and 1178 do not touch this script. PR 1153 packages the workspace with naersk and src = ./.. A flake source has no .git. Both scripts panic at source revision unavailable (build.rs line 12) before they set GQT_SOURCE_REVISION. A missing git binary panics one line earlier, on read source revision (NotFound). That failure is the same on the merge base. Leave it. A fake revision would make a gitless build claim an identity it does not have.

Limitations

I did not link the 350 MB GQT worker and I did not run the GQT corpus. In the omnigraph worktree, the package manifest was reduced to this build script and an empty library so Cargo did not compile the engine. The script bytes were the PR blob. Cargo's fingerprint is the behavior under test. The plain-clone and ref-update runs used those same bytes in small git repositories, not the full engine crate.

Inline comment on crates/omnigraph-gqt/build.rs line 69 (RIGHT), the .filter(|path| path.is_file()) line:

The is_file filter is what stops the worktree rebuild loop. Keep it. It also drops the branch ref when that ref is only in packed-refs.

git pack-refs --all deletes the loose ref. The next build watches packed-refs and does not watch refs/heads/<branch>. git reset --soft then creates the loose ref again. It does not change HEAD, the index, or packed-refs. Cargo stays fresh, and GQT_SOURCE_REVISION stays on the old commit.

Reproduced on cargo 1.97.1 with this script: after pack-refs --all, reset --soft HEAD~1 left the check Fresh with baked revision aec33abb9fa0fa08c2e094ef409ce4c9a35d0bb6 while git rev-parse HEAD was c64ee22ba358aa14d0161e2c152c450afa89099e.

omnigraph-bench already handles this in emit_reference_rerun_path: if the loose ref is absent, watch the nearest existing parent directory, and do not watch the git directory itself. Do that for the branch ref only (controls that start with refs/). Do not emit the missing loose path. Cargo treats that path as dirty on every build (the file ... is missing). A directory watch reruns when the loose file appears. The following build sees the file and watches the file.

While the directory is the watch, a change to another ref in that directory also rebuilds. That cost is one extra rebuild, then the script watches the single file again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant