agent: load-only flakes fixed at the cause (two real races); #137 follow-ups

[jaredLunde]

This PR fixes every test I saw fail only under heavy parallel load, each at its cause, not by retrying it or lengthening a timeout. Two of the eight turned out to be real product races. It also carries the three follow-ups from #137's verification.

How the flakes were found

• Load: 4 to 6 concurrent full beyond-ai-agent suites, run over and over, plus up to 40 CPU-burner processes (load average 50–75 on 16 cores).
• Tests failing there that this PR does not touch:
  • agent: OAuth fixture coverage (DELETE, streamed in-POST question); session lock is a record lock (serve_ws flake root cause) #144: tools::output sweep.
  • agent tests: prove MCP connect concurrency and executor yielding by structure, not wall clock #148: mcp_multiple_servers_connect_concurrently_not_sequentially, *_does_not_stall_the_runtime.
  • agent: a tool waits for its checkpoint; a closed session's reason always arrives #150, the Skills author's (agreed with them): serve_auth login, serve_service_isolation::a_failed_probe_means_no_tool_ever_runs_on_the_replica (agent: a tool waits for its checkpoint; a closed session's reason always arrives #150's close-reason fix covers it: 0/96 failures on their branch), and their own two.

Flakes fixed

Test	Cause	Fix	Proof it fails without the fix
mcp_events_nested::a_nested_elicitation_during_an_events_poll_reaches_the_owning_session	A real race. serve started the MCP Events hub before installing the session's elicitation and sampling gates. A server that asked during the very first events/poll was declined as "no client" (seen in a debug log: MCP elicitation unanswered; declining err=NoClient).	The hub starts after the gates are installed.	New test a_nested_elicitation_on_the_first_poll_waits_for_the_session_to_take_questions: a debug-only seam (BEYOND_AI_AGENT_TEST_SLOW_GATE_INSTALL_MS) holds the window open. It fails every time on the old order and passes on the new one.
mcp_events_receiver::a_retry_that_beats_the_resubscribe_after_a_restart_is_told_to_retry_not_to_stop	A real race. A restarted daemon answered 410, which tells the server to stop for good, to a retried delivery for a persisted callback. That happened when the delivery arrived before the events session had read its state and reserved the token.	From daemon start until the events session has reserved its persisted tokens, every unknown token is answered 503 (retry). The window is bounded by RESERVE_FOR.	A debug-only seam (BEYOND_AI_AGENT_TEST_SLOW_EVENTS_RESTORE_MS) holds the state read off, and the test's 800 ms sleep is gone. Without the fix it gets 410.
serve_uds::uds_stale_socket_file_is_reclaimed	The "stale" socket was a listener bound and then dropped inside the multi-threaded test process. A child forked by another test at that instant keeps the descriptor until its exec. A connect in that window succeeds, so the test talked to a socket the daemon had not yet replaced.	The stale node is bound as a datagram socket. No stream connect can reach it.	Structural: nothing can make a stream connect to a datagram node succeed.
mcp_stdio::a_panic_unwinding_past_the_exit_guard_still_sweeps	sweep_before_exit's wait is bounded by grace + 2 s, and a loaded host can stall a sweep thread past it.	After the wait, the exiting thread itself kills every server still running: by pid only if not reaped, by group otherwise. The test now reads "killed" from the kernel: gone, a zombie, or SIGKILL pending in SigPnd/ShdPnd. When the process is reaped is up to the scheduler.	Structural.
mcp_events::wire::many_events_in_one_chunk_drain_in_linear_time	The test bounded wall-clock time at 3 s.	It now counts bytes instead (test-only counter): bytes scanned for separators plus bytes moved by compaction must stay within 2× the input. A per-event drain or a rescan from the event start makes the count quadratic.	By construction.
serve_lifecycle::a_slow_lifecycle_consumer_does_not_delay_the_prompt_response	The test bounded the prompt response at 1.5 s against a 2 s slow collector.	The collector now holds every POST unanswered until the test releases it. The prompt response must arrive while answered == 0; afterwards the events are confirmed delivered. If emit awaited the network, the read would stall and fail.	By construction.
tool_reactor_stall::write_does_not_stall_the_runtime (#148's probe; failed on CI run 37614530412)	The probe, not the code. This branch already had #148's write_bytes(Vec<u8>) move. On the CI runner a 4 MB write took 0.69–0.87 ms of thread CPU inline. The tool's fixed per-call overhead (stat, create_dir_all, the hand-off: ~0.26 ms in a debug build) was a third of that baseline.	The edit and write subjects are now ~36 MB, so the baseline is milliseconds on any machine. Locally: write 0.20 ms of 24 ms, edit 1.3 ms of 37 ms, ls 0.34 ms of 7.8 ms. The 25% bar is unchanged.	60/60 under 40 CPU burners. Note: edit's hold grew from ~0.3 ms to ~1.3 ms with the 8× larger input. That suggests some per-byte work still runs on the executor, still under 4% of the baseline; worth a separate look.
serve_health (readyz)	The shard probe has a 1 s deadline. Under load, a caller can get "shard probe timed out" while the probe is still running; its real answer then lands in the memo.	readyz() waits out exactly that one answer. Any other answer, a 503 for any other reason included, is returned at once.	—
mcp_skills_integrity::a_listing_that_never_ends_is_cut_off_and_said_so	The fixture's listing had TTL 0, so the turn paged through all 64 pages a second time under the refresh's 30 s wall-clock bound.	The listing stays fresh for the run, so the cut-off is the page budget alone. The test also asserts exactly 64 skills/list calls.	—

#137 follow-ups

1. NFS lock safety. The key's maker re-reads it through the locked descriptor (FileLock::file()). With agent: OAuth fixture coverage (DELETE, streamed in-POST question); session lock is a record lock (serve_ws flake root cause) #144's OFD locks (merged since), no other descriptor's open or close can release the lock, NFS included, so the extra registry and test I had first written became moot and were dropped in the rebase. agent: OAuth fixture coverage (DELETE, streamed in-POST question); session lock is a record lock (serve_ws flake root cause) #144's own tests cover the OFD property.
2. The flush that was dropped. I documented, in place, why the prompt needs no drain: carried results cover a resumer restart; a placeholder reaches the path only during a run, whose end drains its mark; and resume_into_turn adds the resumer's own placeholder ids. I couldn't build a test for this one. No observable behaviour depends on the drain, and the race can't be forced through serve's surface.
3. Old-name key and the trash. A key still under the old name is trashed, restored and hard-deleted with its session.
   • Test: a_trashed_session_takes_an_old_name_key_along: gone from the session dir and present in .trash after delete, back and keyed after restore.

Stability

• Setup: 30 iterations of every fixed test, run alongside 4 continuously restarted full suites and 40 CPU burners (load average 61–75).
• Result: every fixed test passed 30/30, and also passed inside each concurrent suite run.
• Outside this PR: the only failures in those runs were agent tests: prove MCP connect concurrency and executor yielding by structure, not wall clock #148's tests (not yet merged).

Verification

• clippy (-D warnings, all targets), cargo fmt and dprint check are clean.
• lib: 1592 passed (after the rebase onto agent: OAuth fixture coverage (DELETE, streamed in-POST question); session lock is a record lock (serve_ws flake root cause) #144).
• After the rebase onto agent: OAuth fixture coverage (DELETE, streamed in-POST question); session lock is a record lock (serve_ws flake root cause) #144/agent: a tool waits for its checkpoint; a closed session's reason always arrives #150, these targets pass: tool_reactor_stall, mcp_events_nested, mcp_events_receiver, serve_lifecycle, serve_health, serve_uds, mcp_skills_integrity, serve_service_isolation, serve_service_session_end.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk