Repository navigation
Keep branch removals from deleting a fork the main thread serves, and retry them when a worker lets go - #3763
Conversation
… retry them when a worker lets go
Follow-ups to the owed-removal retry from harper#3600, from the review it got before merging:
- With threads.count 0 the main thread is the only worker and serves every app it placed until Harper restarts. A drop
now keeps that app's forks ("serving"), on a present or absent app and with restart either way, and the retry leaves
them for the start pass instead of closing main's handle and deleting the storage.
- The retry reads the root config with readRootConfigEntry and keeps its timer when the file cannot be read or parsed,
instead of taking that as "deployed" and stopping.
- manageThreads tells listeners when a worker exits or reports what it loaded; a pending retry runs then, instead of
waiting up to an hour for its backoff timer.
- Clearing the marker first removes a `removal-pending`.partial a crash left, so the app's branches directory goes too.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
… comments that restate the code The main-serves unit test stubbed the loader's placements, which the repo's test rules forbid and which a native dynamic import can bind before the stub; the zero-worker integration test already exercises it with the real loader. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…heck Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
There was a problem hiding this comment.
Code Review
This pull request introduces support for retaining and cleaning up branched database storage when the main thread acts as the sole worker (i.e., when threads.count is 0). It adds mechanisms to retry pending branch removals upon worker exits or load reports, and updates the cleanup process to handle partial marker files. The review feedback identifies two critical issues: first, a test simulating a crash during marker writing is flawed because the final marker is not deleted, masking a leak; second, applicationsWithPendingBranchRemoval must be updated to check for .partial files to prevent directories from being leaked permanently if a crash occurs mid-write.
…s's worker count alone A worker started outside safe mode resolves threads.preload once for the whole mocha process, so the preload suite that runs later in test:unit:main saw none; threadCount also rewrote the process-wide worker count. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A crash between writing `removal-pending`.partial and renaming it, after the drop removed the application, left only the temp file, which neither the census of owed removals nor its recheck looked for, so that fork was never retried. Only a drop that discards the branches writes one, and the retry still waits while the application is deployed or held. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ring shutdown Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
| workerReleasesWatched = true; | ||
| manageThreads.onWorkerRelease(() => { | ||
| if (manageThreads.isProcessShuttingDown()) return; | ||
| if (branchRemovalRetryTimer || branchRemovalPassRunning) void retryPendingBranchRemovals(); |
There was a problem hiding this comment.
[nit] Worker-release wake-ups depend on an implicit timer invariant; document that every timer-worthy debt leaves a timer armed to prevent future retry loss
⊙ Problem
Kris's review of #3600 found four gaps in its owed-removal retry:
threads.count: 0, main is the only worker and serves its apps until restart. The retry never counted main as a holder, so a drop deleted a fork main still served.`removal-pending`.partialwas never retried and kept the app'sbranchesdirectory forever.🤔 Does this belong in one follow-up?
Yes. The first deletes a fork main still serves, and the others leak forks or delay removing them. The fifth finding is an older race that needs a design choice, so it is #3758.
💡 Solution
A drop keeps the forks main serves until the next start. The retry timer survives an unreadable config. A worker's exit or load report triggers a retry. The retry counts a crash-left
.partialas owed and clears it before the marker.⚖️ Alternatives
🤔 The planning review's alternative was adopted
Framing-Verdict: better-alternative-exists (8b40215737fa)
I planned a new config helper. Codex showed
readRootConfigEntryrejects malformed files without refreshing config. I used it, removed the.partialbefore the marker, and added the tests it asked for.Also considered
poolMayHold.withPoolForalso reads it, and with no workers a pool restart replaces nothing.onThreadExit. It can fire before the worker leavesworkers, and it never sees a load report.🔧 Changes
Part of #3042. Follows up #3600.
components/operations.ts:mainServesApplication. It is true when main is the worker and its loader placed the app, and it loads the placement lookup once.dropApplicationreads it under the lock and checks forservingafterkeepBranchesand beforeloaded, with its own message.retryBranchRemovalchecks the root-config entry and then main. It does not log the parser's message, which can quote the file.retryPendingBranchRemovalscall registers anonWorkerReleaselistener. That listener requests a pass only while the timer is set or a pass is running, and never during shutdown.server/threads/manageThreads.ts: addsonWorkerRelease. It is notified fromrecordLoadedApplications, and at the end of a worker's exit handler, afterworkers.spliceand any auto-restart. A listener that throws is caught and logged.resources/branchDatabase.ts: the owed-removal checks also count the marker's temp file, which only a discarding drop writes and a crash can leave unrenamed after the app is gone.clearPendingBranchRemovalremoves it durably before the marker.components/DESIGN.md: thedrop_componentnote lists the new retry triggers and what each kind of owed removal waits for.✅ Verification
branchRemovalRetry.test.js): five new tests, and each fails onmain. They cover:branched-database-package.test.ts, Linux): athreads.count: 0start loads the package through the real loader. Arestart: truedrop, then a drop of the absent app, both keep the fork while the app still answers, and the next start removes it. Onmain, the first drop answersbranchesRemoved: true. It does not check the base table, which With threads.count 0, an application's writes to its branched databases land in the base table #3767 shows leaks onmaintoo.main:test:integration:all: 3006 pass, 0 fail (main: 3005, 0).test:unit:main: 3 of 8456 fail, all 3 also onmain(nonInteractiveSpawn, and two tests that read the container'sLOGGING_LEVEL).test:unit:resources: 1 fails,DerivedIndexRuntimeepoch handoff, also failing alone onmain3 of 3.🤖 Generated with Claude Code
🤖 Generated by Anthropic Claude (Opus 5.5); posted via @dawsontoth.
Release target: main — fixes code from #3600, which is only on main (v5.4); not a backport
Related PRs: #3600 overlaps (merged; this follows up its review)
Complexity: moderate
Late Grok findings
3 surviving (highest minor) — Grok reviewed fc4fb33
unitTests/components/branchRemovalRetry.test.js:140— the main-serves test stubscomponentLoader.applicationsPlacedOnThisThreadwith sinon, whichAGENTS.md:218rules out.mainServesApplicationreads the function through a nativeimport()(components/operations.ts:2080), and that import keeps its own copy of the CommonJS exports. There are twounitTests/components/fixtures/idleWorker.js:3— new comments describe the code next to them:idleWorker.js:3,branchRemovalRetry.test.js:139and:158. The comment atresources/branchDatabase.ts:867explains why the partial file is removed before the marker, which is allowed. The rest of the Gemini group is folded in herecomponents/operations.ts:2078—mainServesApplicationcallsimport('./componentLoader.ts')each time it runs. It only runs on the drop and retry path, never per request, and after the first load the import resolves from the module cache. So the cost is negligible; hoisting the import is optionalNot standing: 1 merged. Grok ran 20m (20m from launch to report); the fold adjudication (
cursor-grok@fc4fb3315d45-2941044b, claude) took 1m.Deferred findings
205923d1d2d513b0nit data-integrity —resources/branchDatabase.ts— crash-left .partial from a Retry a branched database removal that drop_component could not finish, instead of only logging the failure #3600-only build revives removal intent a keepBranches drop cancelled — filed A crash-left branch-removal temp file from a pre-#3763 dev build could revive a cancelled removal #3773Late Grok findings
none surviving — Grok reviewed af3c6c0
No finding survived adjudication.
Not standing: none. Grok ran 18m (18m from launch to report); the fold adjudication (
cursor-grok@af3c6c00d994-cc79f5d5, claude) took 1m.Review-Coverage: authored=claude; ran=cursor-composer,gemini,codex,cursor-grok; late=cursor-grok:adjudicated; adjudicated=domain; declined=muse; rounds=7; full=1 @ af3c6c0
Review-Attention: study ~20m (critical: branchDatabase.ts; hot: manageThreads.ts; decisions: temp-file-counts-as-debt, shutdown-suppresses-wakeups-only, main-serves-waits-for-restart, unreadable-config-polls, do-less-alternative) @ af3c6c0