Skip to content

Keep branch removals from deleting a fork the main thread serves, and retry them when a worker lets go - #3763

Merged
kriszyp merged 6 commits into
mainfrom
claude/branch-removal-retry-followups
Oct 30, 2026
Merged

kriszyp merged 6 commits into
mainfrom
claude/branch-removal-retry-followups

Conversation

@dawsontoth

@dawsontoth dawsontoth commented Oct 29, 2026 •

Copy link
Copy Markdown
Contributor

A badger with a broom waits beside a burrow where a dormouse still sleeps and turns to sweep the next one the moment its door bell rings

⊙ Problem

Kris's review of #3600 found four gaps in its owed-removal retry:

  • With threads.count: 0, main is the only worker and serves its apps until restart. The retry never counted main as a holder, so a drop deleted a fork main still served.
  • An unreadable root config stopped the retry timer.
  • A removal waiting on a worker ran again only on its timer, up to an hour later.
  • A crash-left `removal-pending`.partial was never retried and kept the app's branches directory forever.

🤔 Does this belong in one follow-up?

Yes. The first deletes a fork main still serves, and the others leak forks or delay removing them. The fifth finding is an older race that needs a design choice, so it is #3758.

💡 Solution

A drop keeps the forks main serves until the next start. The retry timer survives an unreadable config. A worker's exit or load report triggers a retry. The retry counts a crash-left .partial as owed and clears it before the marker.

⚖️ Alternatives

🤔 The planning review's alternative was adopted

Framing-Verdict: better-alternative-exists (8b40215737fa)

I planned a new config helper. Codex showed readRootConfigEntry rejects malformed files without refreshing config. I used it, removed the .partial before the marker, and added the tests it asked for.

Also considered

  • Count main inside poolMayHold. withPoolFor also reads it, and with no workers a pool restart replaces nothing.
  • Unload the app from main on drop. Nothing in Harper unloads an application.
  • Reuse onThreadExit. It can fire before the worker leaves workers, and it never sees a load report.

🔧 Changes

Part of #3042. Follows up #3600.

✅ Verification

  • Unit (branchRemovalRetry.test.js): five new tests, and each fails on main. They cover:
  • Integration (branched-database-package.test.ts, Linux): a threads.count: 0 start loads the package through the real loader. A restart: true drop, then a drop of the absent app, both keep the fork while the app still answers, and the next start removes it. On main, the first drop answers branchesRemoved: true. It does not check the base table, which With threads.count 0, an application's writes to its branched databases land in the base table #3767 shows leaks on main too.
  • Full gates in a Linux container, each failure compared by name with main:
    • test:integration:all: 3006 pass, 0 fail (main: 3005, 0).
    • test:unit:main: 3 of 8456 fail, all 3 also on main (nonInteractiveSpawn, and two tests that read the container's LOGGING_LEVEL).
    • test:unit:resources: 1 fails, DerivedIndexRuntime epoch handoff, also failing alone on main 3 of 3.
    • Under LMDB the touched suites pass, with forks skipped.
  • PR QA: skipped @ af3c6c0 — dispatch unavailable in this session.

🤖 Generated with Claude Code

🤖 Generated by Anthropic Claude (Opus 5.5); posted via @dawsontoth.

Release target: main — fixes code from #3600, which is only on main (v5.4); not a backport

Related PRs: #3600 overlaps (merged; this follows up its review)
Complexity: moderate

Late Grok findings

3 surviving (highest minor) — Grok reviewed fc4fb33
  • minor — unitTests/components/branchRemovalRetry.test.js:140 — the main-serves test stubs componentLoader.applicationsPlacedOnThisThread with sinon, which AGENTS.md:218 rules out. mainServesApplication reads the function through a native import() (components/operations.ts:2080), and that import keeps its own copy of the CommonJS exports. There are two
  • nit — unitTests/components/fixtures/idleWorker.js:3 — new comments describe the code next to them: idleWorker.js:3, branchRemovalRetry.test.js:139 and :158. The comment at resources/branchDatabase.ts:867 explains why the partial file is removed before the marker, which is allowed. The rest of the Gemini group is folded in here
  • nit — components/operations.ts:2078 — mainServesApplication calls import('./componentLoader.ts') each time it runs. It only runs on the drop and retry path, never per request, and after the first load the import resolves from the module cache. So the cost is negligible; hoisting the import is optional

Not standing: 1 merged. Grok ran 20m (20m from launch to report); the fold adjudication (cursor-grok@fc4fb3315d45-2941044b, claude) took 1m.

Deferred findings

Late Grok findings

none surviving — Grok reviewed af3c6c0

No finding survived adjudication.

Not standing: none. Grok ran 18m (18m from launch to report); the fold adjudication (cursor-grok@af3c6c00d994-cc79f5d5, claude) took 1m.

Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=7; full=1 @ af3c6c0

Review-Attention: study ~20m (critical: branchDatabase.ts; hot: manageThreads.ts; decisions: temp-file-counts-as-debt, shutdown-suppresses-wakeups-only, main-serves-waits-for-restart, unreadable-config-polls, do-less-alternative) @ af3c6c0

dawsontoth and others added 3 commits October 29, 2026 12:54
… retry them when a worker lets go

Follow-ups to the owed-removal retry from harper#3600, from the review it got before merging:

- With threads.count 0 the main thread is the only worker and serves every app it placed until Harper restarts. A drop
  now keeps that app's forks ("serving"), on a present or absent app and with restart either way, and the retry leaves
  them for the start pass instead of closing main's handle and deleting the storage.
- The retry reads the root config with readRootConfigEntry and keeps its timer when the file cannot be read or parsed,
  instead of taking that as "deployed" and stopping.
- manageThreads tells listeners when a worker exits or reports what it loaded; a pending retry runs then, instead of
  waiting up to an hour for its backoff timer.
- Clearing the marker first removes a `removal-pending`.partial a crash left, so the app's branches directory goes too.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… comments that restate the code

The main-serves unit test stubbed the loader's placements, which the repo's test rules forbid and which a native
dynamic import can bind before the stub; the zero-worker integration test already exercises it with the real loader.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heck

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@dawsontoth dawsontoth added this to the v5.4 milestone Oct 29, 2026

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces support for retaining and cleaning up branched database storage when the main thread acts as the sole worker (i.e., when threads.count is 0). It adds mechanisms to retry pending branch removals upon worker exits or load reports, and updates the cleanup process to handle partial marker files. The review feedback identifies two critical issues: first, a test simulating a crash during marker writing is flawed because the final marker is not deleted, masking a leak; second, applicationsWithPendingBranchRemoval must be updated to check for .partial files to prevent directories from being leaked permanently if a crash occurs mid-write.

Comment thread resources/branchDatabase.ts Resolved
dawsontoth and others added 2 commits October 29, 2026 13:39
…s's worker count alone

A worker started outside safe mode resolves threads.preload once for the whole mocha process, so the preload suite
that runs later in test:unit:main saw none; threadCount also rewrote the process-wide worker count.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A crash between writing `removal-pending`.partial and renaming it, after the drop removed the application, left only
the temp file, which neither the census of owed removals nor its recheck looked for, so that fork was never retried.
Only a drop that discards the branches writes one, and the retry still waits while the application is deployed or
held.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ring shutdown

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

@kriszyp kriszyp left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 Reviewed with Muse

Comment thread components/operations.ts
workerReleasesWatched = true;
manageThreads.onWorkerRelease(() => {
if (manageThreads.isProcessShuttingDown()) return;
if (branchRemovalRetryTimer || branchRemovalPassRunning) void retryPendingBranchRemovals();

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nit] Worker-release wake-ups depend on an implicit timer invariant; document that every timer-worthy debt leaves a timer armed to prevent future retry loss

@kriszyp
kriszyp merged commit 4751823 into main Oct 30, 2026
107 of 108 checks passed
@kriszyp
kriszyp deleted the claude/branch-removal-retry-followups branch October 30, 2026 13:25
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants