Skip to content

perf(enginehost): a retiring host gives the workspace up before it saves - #1972

Merged
santoshkumarradha merged 3 commits into
devfrom
perf/handover-retiring-host
Oct 26, 2026
Merged

santoshkumarradha merged 3 commits into
devfrom
perf/handover-retiring-host

Conversation

@santoshkumarradha

@santoshkumarradha santoshkumarradha commented Oct 26, 2026 •

Copy link
Copy Markdown
Member

Summary

A launch after the dev channel replaced the binary took about 15 s. The old host had retired itself, but it held the workspace lock while it saved its conversations, and it had already closed its socket. The new launch saw the lock held and assumed a host was starting. It waited the full 10 s birth wait, then opened the conversation in its own process.

 Host.shutdown
+  close listener, remove socket, release workspace lock
   for each conversation: Close   # waits for its last save, up to 30 s
-  close listener, remove socket, release workspace lock
 Attach
   lock free  -> spawn, wait for socket
-  lock busy  -> wait 10 s for a socket
+  lock busy  -> wait for a socket, and ask for the lock again on each turn;
+                when the holder lets go, spawn once

Each conversation still holds its own journal lock until its save finishes, so a new host cannot open a conversation that is still being written. The second change covers hosts from older builds, which still save while holding the lock.

A retiring host also holds a new host.saving lock from before it gives the slot up until its last conversation is closed. Retire and Stop (waitForGone) wait up to 1 s more on that lock once the slot is free, so an ordinary save finishes before the new host reopens those journals. The launch path never waits on it. The full suite caught this need: TestDifferentEnvironmentRestartsIdleRealHostAndReopensJournals failed without it.

Evidence

Tests count states and order, not time (internal/enginehost/handover_test.go):

  • TestARetiringHostLetsGoOfTheWorkspaceBeforeItsSaveFinishes: the host retires while a conversation's Close is blocked. The test checks that the lock is free, that a second host can start, and that the first host's late exit leaves the new socket in place.
    Before: the host kept the workspace lock while its conversation was still saving. After: pass.
  • TestRetireWaitsForTheSaveOfTheHostItAskedToGo: while the save is still running, the slot is free and Retire has not returned. Retire returns after the save.
  • TestALaunchStartsTheHostOnceAHolderOnItsWayOutLetsGo: the lock is held with no socket, Attach starts waiting, then the lock is released. The test checks that exactly one host is spawned and Attach connects to it.
    Before: engine host: no host answered (after 10 s). After: pass.

Rig on Spark (copied codeaf home, fresh 240k-file folder so the first save is slow): open the folder, quit, engine --stop, then launch again 1 s later.

first frame engine --stop client engine
dev 334cab9 11.03 s gave up after 10 s: "the host is still holding this workspace" in-process (vision line in the client's own stderr at +10 s)
this branch (d4db04e) 0.51 s 1.04 s (1 s save grace, then returns) new host; the old host keeps saving alongside it and exits on its own

In the dev run, the client's goroutine dumps show enginehost.waitForHost < Attach < localLink.dial < remote.Roam from 0.2 s to 10 s. The old host logged the same line as the laptop did: could not finish saving this chat: still saving this chat on this computer: context deadline exceeded.

Full suite: make check (LC_ALL=C) on Spark, fleet job 20261026-192124-002488, commit d4db04e. Every enginehost test passes, plus -race -count=20 on the new tests and the environment-restart test. Failures:

  • Already failing on dev: internal/release (2), internal/seniordev/util TestFindUpRootFirst.
  • Flaky under suite load, both pass alone (10/10 and 5/5): cmd/codeaf TestRunInCellSealsItsCallsAndContinuesFromThem (TempDir cleanup) and internal/syncsetup TestR7PersonalLeaderFallbackAndNoAutomaticRegain.

Merge Danger

Door: two-way

Blast Radius: host handover

Retire and Stop now return when the workspace is free and its saves are done, or 1 s after the slot frees, whichever comes first. Before, they waited up to 10 s and then failed. A launch that comes during a slow save and asks for the conversation still being written gets the existing "open in another window" landing: a new conversation with the held row armed. Before, it waited up to 10 s and then opened the conversation in its own process.

🤖 Generated with Claude Code

Two failing tests. A host on its way out keeps the workspace lock while its
conversations save, so no new host can start; and a launch that finds the
lock busy with nothing listening waits out the whole birth wait without
starting a host when the holder lets go.
A host on its way out closed its conversations, which waits for each one's
last save, and only then removed its socket and released the workspace lock.
In a large folder that save runs for its whole 30 s bound. A launch arriving
in that window found the lock busy and no socket, waited the full 10 s birth
wait for a host nobody was starting, and then opened its conversation in its
own process: about 15 s to the first frame.

The host now removes its socket and releases the lock first, then saves.
Each conversation keeps its own journal lock until its save is done. And a
launch that finds the lock busy now asks again on every turn of its wait, and
starts the host itself the moment the holder lets go, which covers hosts of
older builds that still save while holding the lock.
The full suite caught it: an environment restart retires the idle host and
the new host reopens its journals, which the old process was still closing.
A retiring host now holds host.saving from before it gives up the workspace
until its last conversation is closed, and waitForGone waits up to one second
for it once the slot is free. Launches that only need the slot never wait on
it, and a slow save past that second no longer holds anybody.

Also the change entry.
@santoshkumarradha
santoshkumarradha merged commit f961fbc into dev Oct 26, 2026
7 checks passed
@santoshkumarradha
santoshkumarradha deleted the perf/handover-retiring-host branch October 26, 2026 19:55
AbirAbbas added a commit that referenced this pull request Oct 26, 2026
…urrow v0.3.4

Brings the workspace-ownership branch up to dev cf282dd, which adds the
Raft relay (#1950), the two-computer fixes (#1967), #1972, #1974 and #1976.

- directory record types are now clusterstate aliases, so frame hints move
  into the replicated state machine: FrameHint and the frame_hints field on
  Cell, CellInit and Publish, bounded deterministically in the shared rules,
  dropped from listings and kept on single-cell reads. Signed v1 publishes
  with hints are accepted; unknown fields are still refused; existing
  canonical bytes are unchanged and no schema or protocol version moves.
- every take path records the send lease, including the workspace-set and
  identity-adopting restores; an already-held chat keeps its hold when the
  record fails.
- a removed computer cannot take through the folder-mapping road either, and
  unsent words and the pending send survive the root chooser.
- the seal refuses a folder holding home before taking the checkpoint lock.
- furrow pin follows dev at v0.3.4, which contains v0.3.2's non-mutating
  SQLite capture.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant