Skip to content

perf(hub): anonymous joins share one cached picture, then the batches since (#2519) - #2589

Merged
genekogan merged 3 commits into
mainfrom
agent/2519-join-cache
Nov 16, 2026
Merged

genekogan merged 3 commits into
mainfrom
agent/2519-join-cache

Conversation

@genekogan

@genekogan genekogan commented Nov 16, 2026 •

Copy link
Copy Markdown
Member

Problem

Every anonymous viewer that joins a world gets its own snapshot built on the world's thread: on Eden 2026 about 50 ms of main thread per join (Step 0 below), so 20 arrivals a second cost the world about a whole second of thread per second. This makes an arriving anonymous viewer cheap: the heavy first picture is built once every two seconds, however many arrive. Closes #2519.

Acceptance

  • A1 Oracle test: two product clients (Connection and Replica) join the same world, one with the cache off and one on, while 6 scripted Souls walk. Zero resyncs on either; equal replica.viewHash at the same heartbeat sequence for 60 s; equal sets of log sequences, talk ids, play sequences and take end ticks. — evidence: node --test apps/server/src/hub-join-cache.test.ts -> test "A1" passes: the cached client gets the shared picture built 1.5 s (30 ticks) earlier plus exactly the batches after it, in order; then 1,200 ticks (60 s) of 6 walking Souls; 0 resyncs on both, 0 mismatches, viewHash equal at every common heartbeat (>= 10), equal log sequences (from the later client's first one), talk ids, play sequences and take end ticks. Mutation check: replaying from the cache's seq + 2, or not replaying, fails it (the second through the behavioural oracle alone, with the structural asserts removed).
  • A2 A join during the 2 s after a recovery, and a resync, both converge (same oracle). — evidence: same file, test "A2" passes: a picture built 0.5 s before a recovery is not used by a join 0.5 s after it (the joiner gets a picture of now); a resync 30 s later is answered from the world as it is at the request; after 60 s, the same oracle with resyncs (0, 1).
  • A3 Every hub test passes unmodified with the flag off and on. — evidence: every apps/server/src/*.test.ts constructing a Hub (25 files) plus the new one: SIM_JOIN_CACHE=1 node --test --test-concurrency=4 <26 files> -> 163 pass, 0 fail; SIM_JOIN_CACHE=0 ... -> 163 pass, 0 fail.
  • A4 Main-thread time per anonymous join falls by at least 70 % against Step 0, measured the same way. — evidence: BRAIN_MODE=fake node step0.ts 30 (the Step 0 probe described below: bootEden + a real Hub, timing Hub.onMessage per hello) on agent/2519-meas-base then on base + this branch, back to back, two runs each (bar declared on the issue: candidate mean <= 0.30 x base mean) -> final head: base 50.6 / 48.5 ms, candidate 7.7 / 7.0 ms mean (ratios 0.15 / 0.14, -85 %); p50 44.9 / 43.6 -> 1.1 / 1.1 ms. First candidate commit (before the batch strings were shared): 53.5 / 47.7 -> 14.9 / 13.7 ms (0.28 / 0.29).
  • A5 viewers --viewers 0,200 --arrive-per-sec 20 --souls 24 --act-sec 10, 2 repeats, flag off and on, reported side by side; and the same with --dwell-sec 60 (joins never stop), which is the case this task is for. — evidence: six timed runs, all succeeded (even off, even on, dwell off, dwell on, uneven off, uneven on), side by side in the A5 table below. Cache on: the clock kept 0.93 to 0.99 in all 6 repeats, and first motion p95 stayed at or below 4.8 s. Cache off: 3 of 6 repeats collapsed (clock 0.70, 0.22 and 0.57; the last world restarted). The issue's expectation (clock >= 0.95 and heap within 150 MB of zero viewers) holds in 3 of 6 candidate repeats on both counts, and in 1 of 6 base repeats.

Step 0 (premise check; posted on the issue before the runs)

Probe: one in-process Eden 2026 world (bootEden, fake minds), a real Hub on a local socket, 30 anonymous joins with the page's URL hello (anonymousHello(true)), one world tick between joins. Timer: performance.now() from the hello entering Hub.onMessage to its return after the join's last message is handed to the socket. Shared workstation, load average 4 to 15 on 16 cores, recorded per run. No timed-run command runs a probe, so this one is local; the ratio is what is scored.

part of one join (ms, mean) base candidate (final)
host.snapshotView() 12.4 1.1 to 1.4 (one rebuild per 2 s, shared)
snapshot stringify + column packing 12.8 0 (once per form per rebuild)
the log list (tailEvents(150) + publicEvent) 23.9 0 (once per rebuild)
everything else about 1 about 1
join, mean / p50 50.6 / 44.9 7.7 / 1.1

Entity table: 5,168 entries, 1.72 MB as JSON, 410 KB in the column form.

A5 runs (souls-timed, timed-run.yml from agent/2519-meas-base, repeats 2)

Base (cache off) = origin/main 3d8da8a + merges of the two V0 fixes (PRs 2583, 2586) and the viewers bench (PR 2572), as the coordinator's measurement note requires. Candidate (cache on) = that + this branch at ec67c5a. The second commit here only shares replayed batch strings, so it can only lower per-join cost. The runner cannot set environment, so "flag off" is the base ref. All arms share --viewers 0,200 --arrive-per-sec 20 --souls 24 --act-sec 10. Runner: 48 CPUs, world on 2 physical cores, load generator on the rest. Host load average before and after each run: 7 to 20 (in each run's REPORT.md).

Figures are for the 200-viewer window of each repeat. Clock = tick service. Heap Δ = main heap max at 200 viewers minus the max in the 0-viewer window. Acts = achieved of 288 intended. CPU = world CPU in cores.

config run repeat clock heap Δ MB first motion p50 / p95 s acts CPU reconnects
even, off 43845116449 r1 0.990 +141 9.17 / 12.82 192 1.19 0
r2 0.696 +1,715 23.62 / none (26 missed) 26 1.37 414
even, on 43845127966 r1 0.991 +100 3.13 / 4.84 214 1.18 0
r2 0.994 −160 0.12 / 0.79 226 1.27 0
dwell 60, off 43845139493 r1 0.216 +3,177 none (135 missed) 18 1.63 412
r2 0.907 +3 0.41 / 13.29 189 1.49 0
dwell 60, on 43845152201 r1 0.933 0 0.41 / 2.63 239 1.45 0
r2 0.977 +267 0.14 / 1.15 172 1.27 0
uneven (dwell 60 + --arrive-burst 100@30), off 43845163760 r1 0.568, world restarted +1,485 none (100 missed) 5 — 349
r2 0.910 +139 0.36 / 10.31 133 1.46 0
uneven, on 43845174843 r1 0.960 −169 0.18 / 1.02 182 1.41 0
r2 0.945 +89 0.19 / 1.66 238 1.45 0

How to read it:

  • Cache off collapsed in 3 of 6 repeats, one per config. The clock fell to 0.70, 0.22 and 0.57, with a world restart in the last. The heap rose by 1.5 to 3.2 GB, and hundreds of reconnects followed.
  • Cache on never collapsed. The clock ranged from 0.933 to 0.994, the heap from −169 to +267 MB, and first motion p95 stayed between 0.79 and 4.84 s. Off, first motion p95 ran 10 to 13 s, or never came.
  • Against the issue's expectation:
    • Clock at least 0.95: met in 4 of the 6 candidate repeats. It was missed in dwell r1 (0.933) and uneven r2 (0.945).
    • Heap within 150 MB of zero viewers: met in 5 of 6. It was missed in dwell r2 (+267).
  • The zero-viewer windows are noisy in both arms. Acts achieved there ranged from 2 to 135 of 288, and the act queue p50 was about 10 to 16 s even with no viewers. Act counts are therefore weak evidence either way. The clock and the heap are the robust signals.
  • Two repeats per arm cannot show whether collapse is rarer than one in two with the cache, only that none of six collapsed.

Proof

Tier: T1

  • proof.json: not required at T1; the oracle tests and the bench runs above.
  • clips: none (T1, no visible change)
  • images: none

Deviations from the design

  • The log list is cached too. Step 0 showed it at 24 to 26 ms a join, more than the snapshot; the issue lists this alternative when the small lists exceed 2 ms. Talk (0.03 ms) is not cached.
  • Order: welcome, cached snapshot, cached log, ring batches after the snapshot's seq, then the rest. The log goes before the batches so a page that appends by seq (useSouls onLog) keeps time order.
  • The cache is used only for a hello with no token and no since (the page's entry join). A token-less reconnect too far behind keeps its fresh snapshot (its reason text differs, and it is rare).
  • Replayed batches are stringified once per cache (second commit): a booting world's first batch held 5 MB of log in the probe, and each cached joiner re-stringified it.
  • Validity also checks the ring and engine objects it was built from (every ring reset replaces the array), so any reset, not only a recovery event, drops it; a station era change clears it too (it re-sends a snapshot to everyone).
  • A resync is unchanged (always fresh), which meets "answered from a cache built after the request arrived".

Self-audit

  • Checks run: npx tsc --noEmit -p apps/server -> clean; node --test apps/server/src/hub-join-cache.test.ts -> 2 pass; A3 runs above -> 163/163 both ways; node .github/scripts/guards.mjs scrub --base origin/main -> 0 problems; node tools/changelog/compile.mjs --check -> valid.
  • Failed attempts: the first candidate replayed batches with a stringify per joiner; the probe's 5 MB boot batch made that 8 ms a join, fixed by sharing the strings (A4 above shows both).
  • Long tail / externalities: the hub holds the cached strings (about 0.4 MB columns form, 11 KB log, plus any replayed batches) until the next rebuild; nothing runs when nobody joins. The take-entities registry gets one snapshot table per 2 s instead of one per join, so it no longer churns its 16 recent tables under arrivals. Per-socket compression of the snapshot is still paid per socket (V3).
  • Known limits: a cached joiner's display log starts up to 2 s earlier than a fresh one's (it then gets the batches' events); non-human entity fields are, as for any connected viewer, carried by takes, not batches. The A4 probe runs locally because no timed-run command runs it.
  • Docs: docs/architecture/load.md (anonymous socket joins share one picture, and SIM_JOIN_CACHE).

Changelog

changes/2519-join-cache.md (perf).

Risk

A cached joiner could reach a different replica than a fresh one; the A1/A2 oracle covers walking Souls, recoveries and resyncs, and the client's heartbeat check would resync it in any case. Signed-in joins, resumes and resyncs are untouched. Roll back with SIM_JOIN_CACHE=0 in the world's environment, or a revert.

Time: started 09:49 UTC. Code, tests and A4 were done by 10:10. A5 was scored from 10:40 to 10:50. About 0.5 h of work against a 3 h ceiling; the 30 min wait on the runners is not counted.

Outside Touches: none.

🤖 Generated with Claude Code

genekogan and others added 2 commits November 16, 2026 05:01
… then the batches since (#2519)

A token-less page hello is answered from a join cache: the snapshot stringified
once per negotiated form and the log list, rebuilt every 2 s or after a recovery,
followed by every ring batch after it. SIM_JOIN_CACHE=0 turns it off.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…all joiners (#2519)

A world's first batch after boot can hold megabytes of log; each cached joiner
re-stringified it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@genekogan genekogan added the ci:run Open the CI window: run this ready PR's heavy CI jobs (docs/testing.md#ci-window) label Nov 16, 2026
@genekogan
genekogan marked this pull request as ready for review November 16, 2026 10:41
@github-actions

Copy link
Copy Markdown
Contributor

Proof check: proof-ok passes

Tier T1 from apps/server/src/hub-join-cache.test.ts, apps/server/src/hub.ts (node tools/proof/tier.mjs).
Required at T1 (docs/process/PROOF-POLICY.yml): focused node --test of touched files; typecheck of touched packages.

Check Status Detail
problem statement ok present
linked issue ok #2519
acceptance ok 5/5 met
carried from the issue ok all 5 issue criteria present
self-audit ok checks, failed attempts, long tail and limits filled
Acceptance criteria
Criterion Status Evidence
[x] A1 Oracle test: two product clients (Connection and Replica) join the met node --test apps/server/src/hub-join-cache.test.ts -> test "A1" passes: the cached client gets the shared picture buil
[x] A2 A join during the 2 s after a recovery, and a resync, both converge met same file, test "A2" passes: a picture built 0.5 s before a recovery is not used by a join 0.5 s after it (the joiner ge
[x] A3 Every hub test passes unmodified with the flag off and on. met every apps/server/src/*.test.ts constructing a Hub (25 files) plus the new one: `SIM_JOIN_CACHE=1 node --test --test
[x] A4 Main-thread time per anonymous join falls by at least 70 % against Ste met BRAIN_MODE=fake node step0.ts 30 (the Step 0 probe described below: bootEden + a real Hub, timing Hub.onMessage
[x] A5 viewers --viewers 0,200 --arrive-per-sec 20 --souls 24 --act-sec 10, met six timed runs, all succeeded (even off, [even

Proof links: 0 video, 0 image, no proof.json.

Updated on every edit, push, label and review. How to fill it: .claude/skills/journal/SKILL.md. T3-T4 also need the maintainer's sign-off (tools/proof/gate.mjs). A maintainer can waive with the proof:waived label.

@genekogan genekogan added merge:ready Hand the PR to the merge train (.github/workflows/merge-train.yml) ci:run Open the CI window: run this ready PR's heavy CI jobs (docs/testing.md#ci-window) needs:rebase The merge train could not rebase cleanly; rebase by hand, then relabel merge:ready and removed ci:run Open the CI window: run this ready PR's heavy CI jobs (docs/testing.md#ci-window) merge:ready Hand the PR to the merge train (.github/workflows/merge-train.yml) labels Nov 16, 2026
@genekogan

Copy link
Copy Markdown
Member Author

Merge train: this PR is behind main and could not be rebased automatically, so it left the train.

Conflicting files:

  • docs/architecture/load.md

Resolution attempt:
The merge is still in progress (not resolved).

Rebase onto main, push, remove needs:rebase and add merge:ready again.

@genekogan genekogan added ci:run Open the CI window: run this ready PR's heavy CI jobs (docs/testing.md#ci-window) merge:ready Hand the PR to the merge train (.github/workflows/merge-train.yml) and removed ci:run Open the CI window: run this ready PR's heavy CI jobs (docs/testing.md#ci-window) needs:rebase The merge train could not rebase cleanly; rebase by hand, then relabel merge:ready labels Nov 16, 2026
@genekogan
genekogan enabled auto-merge November 16, 2026 12:08
@genekogan
genekogan added this pull request to the merge queue Nov 16, 2026
Merged via the queue into main with commit 30fa31b Nov 16, 2026
105 of 119 checks passed
@genekogan
genekogan deleted the agent/2519-join-cache branch November 16, 2026 13:23
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:run Open the CI window: run this ready PR's heavy CI jobs (docs/testing.md#ci-window) merge:ready Hand the PR to the merge train (.github/workflows/merge-train.yml)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Scaling V1 · Anonymous newcomers share one cached picture of the world instead of each building their own

1 participant