agent: MCP tasks follow-ups — run --continue resumes, unreachable stays pending, sealed journal

[jaredLunde]

This PR follows up on #130 (MCP Tasks, SEP-2663) and fixes every gap found since #130 merged. It is rebased onto #146 (771d91b) and is a single commit.

For each fix I reverted it with the tests unchanged and checked that the named test fails. Item 2's sealing test is the one exception: it confirms behaviour that already worked, so there was no fix to revert.

What changed since a896d67

• Journal key is per session, not per machine (item 9). At a896d67 the key lived in ~/.claude, so a session opened under another key (another replica, a fresh $HOME, another machine, an upgrade) ignored its whole journal. In service mode $HOME doesn't exist, so at a896d67 the journal was never trusted at all. That breakage came from a896d67's own per-HOME key, not from main: on main (3139d81), before journals were sealed, service-mode resume worked. The auditor ran mcp_tasks_service there and it passes.
• Placeholders are marked structurally (item 10). A placeholder used to be recognised by its text.
• Re-audit fixes at 639c321 (items 11–16 below).
• Rebased onto agent: file_lock module, unreachable deadline-free readers, LISTEN_PID operator note #141, agent: MCP Events follow-ups — panic exit, delivery by tag, permanent refusals, runtime restore #136, agent tests: no port is picked and released; mcp-login binds its callback first #145/tests: no crate picks a port and releases it; a semantic lint for it #147 and agent: MCP 401s keep the server's reason; over-cap events recognized in any key order #146.
  • The mcp_tasks* readers use agent: file_lock module, unreachable deadline-free readers, LISTEN_PID operator note #141's common::child_frames.
  • The server-down tests no longer release a port and rebind it. The test now holds a Relay listener for its whole life, and it relays to whichever fixture is up. The fixture's fixed-port option is gone.

What #135 already covers (dropped from this PR)

• Title-proof scripted mock: agent: MCP skills follow-ups, and the serve suite back in CI #135 has the fuller version, so mine is gone.
• Reserved-kind append_custom check: agent: MCP skills follow-ups, and the serve suite back in CI #135's session_store::is_host_custom_kind already covers the journal kinds; the new mcp_task_placeholder kind is added to it. My task test now relies on it.
• serve_approval expectation: agent: MCP skills follow-ups, and the serve suite back in CI #135 restored the original behaviour, so my change to that test is dropped too.

Items

#	Gap	Fix	Proving test
1	Only a unit test covered the sessionId ownership rule.	No fork or branch path copies custom entries: forks rewrite messages only, and switch_branch keeps the same id. So the e2e test uses the real case of a session file copied (restored or migrated) under a new id. The copy carries the unanswered call and the task record, and must not resume the original's task.	mcp_tasks_resume::a_journal_carried_under_another_session_id_is_not_resumed
2	Task ids can be bearer tokens.	Local: every append now tightens a session file that already existed with a looser mode to 0600 before writing; create already used 0600. Service mode: journal entries go through the tenant-keyed sealed segments, which needed no change.	session_store::tests::an_append_tightens_a_loosened_session_file_to_private; session_segments_sealing::mcp_task_journal_entries_are_sealed_and_read_back (confirmation only)
3	run --continue did not resume tasks.	run now saves the tasks it starts (RunJournal), and run --continue resumes the session's pending tasks before its turn. N3 (the resumer restarting immediately on a session switch) is already proven by switching_sessions_moves_the_resumer_with_it.	mcp_tasks_resume::run_continue_resumes_a_task_a_killed_run_left_in_flight
4	Title requests used up scripted replies from the test mock.	Fixed by #135's title mock (adopted).	#135's tests
5	A forged mcp_task record could trigger a poll.	Client forgery: #135's reserved-kind check. Planted lines: journal entries are authenticated (see item 9). On replay, entries that fail are ignored: a line appended to the .jsonl with write/edit never causes a poll and never reaches the model or get_messages as a result. A record must also match the call it names and name a configured server.	a_forged_task_record_from_a_client_is_refused_and_never_polled, journal_lines_planted_in_the_session_file_are_ignored_on_resume, mcp_resume::tests::{a_keyed_journal_refuses_unsealed_tampered_and_foreign_entries, pending_ignores_a_record_that_does_not_match_its_call}
6	A server that was down briefly turned a live task into a permanent error.	An unreachable server is no longer treated as a final result. The resume redials with backoff (250 ms, doubling, about 5 s in total). If it still can't connect, or the connection is lost for good mid-poll, the call gets a [MCP task result pending] placeholder, no result is saved, and a later prompt (serve) or start (run --continue) tries again. The real result then replaces the placeholder where it sits. A configured server that failed to connect at startup is kept dormant in the catalog for this. Only a terminal status, a -32602, or an expired TTL resolves a call.	run_continue_while_the_server_is_down_leaves_the_task_pending_then_delivers_it (the auditor's probe made into a test), serve_with_the_server_down_leaves_the_task_pending_then_delivers_it
7	The unconfigured-server filter was untested.	—	a_task_on_a_server_no_longer_configured_is_not_resumed (serve), run_continue_does_not_resume_a_task_on_a_server_no_longer_configured (run)
8	serve_abort_sends_tasks_cancel_for_sticky_task was flaky: the cancel flag could be read after it was created but before it was written.	Fixtures now write every file a test reads (cancel flag, orphan pidfiles, task state, exit marker, apps and skills markers) to a temp file and rename it into place. The test also waits for complete content within a deadline instead of just checking that the file exists.	With an in-place racy write and the old test, it fails with exactly the CI symptom (got ""); with the fix it passes.
9	MEDIUM: the journal key was per machine (~/.claude/mcp-task-journal.key). A session opened where the key differed (another service replica on shared storage, a wiped ~/.claude, another machine, an upgrade) silently lost every journal entry: no resume, no journaled results.	mcp_resume::JournalAuth, held by the session's store (active_journal / append_journal). Service mode (segments sealed under the tenant key): no MAC at all, because the storage already authenticates every line, which is stronger. Local: an HMAC key in a 0600 sidecar beside the session (<session>.jsonl.mcp-task-journal.json, see item 16; or inside a segmented session's directory). The session's first journal write creates it, so a session that never uses tasks gets no extra file. Concurrent first writers agree on one key (item 11). The key moves, trashes and restores with the session. Sessions without a key (from before this change, or not yet journaled) are read as before, and opening one writes nothing. The next journal write makes the key and records the MACs of the entries already there as accepted, so nothing that resumed before stops resuming.	mcp_tasks_resume::a_session_copied_to_a_fresh_home_still_resumes_and_shows_its_journal (copied to a fresh $HOME: resumes the pending task; copied again: the journaled result reaches the model and get_messages with no re-poll), mcp_tasks_service::a_task_left_in_flight_on_one_replica_resumes_on_another (two serve --service processes on one store: the owner is killed mid-task and the peer resumes it). Both fail on a896d67. Also session_store::tests::{a_session_keeps_its_journal_key_beside_it_and_refuses_planted_entries, a_session_without_a_journal_key_keeps_its_existing_entries}.
10	LOW: a placeholder was recognised by its [MCP task result pending] prefix, so a real tool result that began that way counted as unanswered and was resumed again.	Whoever puts a placeholder on the path (the serve resumer, run --continue) journals an mcp_task_placeholder entry for its tool_use id, sealed like the rest of the journal and reserved against client append_custom. answered_ids, pending, splice, materialize and resume_into_turn use that set and never look at the text.	mcp_tasks_resume::a_real_result_that_reads_like_a_placeholder_is_still_an_answer (fails on a896d67: the answered call gets polled again), mcp_resume::tests::a_placeholder_is_known_by_its_mark_not_its_content
11	MEDIUM (re-audit): two first writers didn't reliably agree on one key. publish_private used one fixed <sidecar>.tmp, and create_private unlinks whatever is there, so writers clobbered each other's temp. The auditor's probe saw 36/600 writers believe a key that wasn't on disk, and 353 error; their entries were then silently refused.	The key is made under an exclusive file_lock on the sidecar itself, then re-read once the lock is held: whoever got there first wrote it, and everyone after uses that key. It is written through the locked descriptor, in place. There is no temp file to share, and none is left beside the session. Every journal write adopts the key on disk (one small read), so a store opened before another writer made the key still seals under it. A torn or overwritten sidecar is replaced the same way. publish_private is gone.	session_store::tests::concurrent_first_journal_writers_all_use_one_key: 4 writer processes (this test binary re-run as journal_key_writer_child) and 8 threads journal at once, released by a go file. The test asserts that all 12 entries carry a mac that verifies against the one key on disk. It fails 3/3 with the old publish_private logic swapped back in.
12	LOW: no test checked that the sidecar actually moves to the trash.	—	a_trashed_session_takes_its_journal_key_along_and_brings_it_back: after delete the key is gone from the session dir and present in .trash; after restore it is back and the journal verifies.
13	LOW: nothing tested that sealed storage gets no key.	—	session_segments_sealing::mcp_task_journal_entries_are_sealed_and_read_back now journals through append_journal, which is where a key would be made. It asserts no key file inside or beside the session dir and no mac on any entry. With journal_key_path returning a path for codec storage, it fails (["mcp-task-journal.json", "000001.jsonl"]).
14	LOW: the flush-before-prompt step (serve's try_recv drain) was untested.	I couldn't make that race deterministic through serve's public surface. A queued prompt can overtake a journal write only when both are ready at one idle select!. While a host bash runs, a prompt is refused as busy, so it can't be queued. So the dependence is now structural instead, and the flush is removed. The flush only mattered when a resumer restarts at a prompt (needs_retry): the previous resumer's finished results might not have landed in the journal yet, and the new one would poll those tasks again. start_mcp_resumer!(&carried) now counts those results as answers whatever the channel holds. Placeholder marks don't need the flush: a placeholder reaches the path only inside a run, and the run-end drain journals its mark before the next prompt.	No deterministic e2e test (see the fix column). pending's unit tests cover a passed-in result answering its call.
15	LOW: the docs didn't say a model with write can corrupt the sidecar, not just delete it.	ARCHITECTURE and JournalAuth now say that deleting the key or overwriting it with anything unparseable makes the session keyless. The next journal write then makes a new key that accepts every line already in the file, planted ones included.	docs
16	LOW: path.with_extension(...) gave work.1 and work.2 one key file.	The key's name is now a suffix on the whole file name (<session>.jsonl.mcp-task-journal.json). A key under the old name is read, and the next journal write carries it over (same bytes) rather than making a new key. The suffixed key is handled by name in trash, restore and hard delete.	sessions_named_alike_keep_separate_journal_keys, a_journal_key_under_the_old_name_is_carried_over (also checks that a line planted before the carry-over is still refused)

Residual risk (items 5 and 9), stated honestly

A session .jsonl is ordinary file content. Locally, the journal key is a file beside the session that the agent's user can read. A model with write can also delete the key, or corrupt it by overwriting it with anything that does not parse. Either way the session becomes keyless, so the entries there are accepted, and its next journal write makes a new key that accepts every line already in the file, planted ones included. So a model that can also read files and goes looking can forge an entry. The transcript itself is unauthenticated, too: a model that can write the file could always plant an ordinary tool_result message. Locally, the seal turns "append a line" into a deliberate read-then-forge; it does not close forgery.

In service mode the store is out of the tools' reach (they run in the tenant's sandbox), so forgery is closed. All of this is documented in ARCHITECTURE "Durable MCP tasks".

The key is made lazily, at the first journal write, so a session that never uses tasks gets no extra file and nothing else in the session layout changes. The trade-off: a session that has never journaled has no key yet, so lines planted into it are accepted at the next open. That is no weaker than the admitted risk above, since a model that can write the file can already plant ordinary transcript messages. A session that has journaled refuses planted lines unless the model also deletes or reads the key.

Verification

• Unit tests: lib 1574 passed.
• Integration tests: after this round, the mcp_tasks*, session_segments*, serve_harness_deadlines and run_session_management targets pass locally. The previous round ran the full suite.
• Mutation checks: each new test fails without its fix. Items 11, 13 and 16 were checked by reverting each fix; items 12 and 16 also follow from their assertions.
• Lint and format: clippy (-D warnings, all targets) is clean; cargo fmt and dprint check pass.
• mcpkit interop (SEP-2663 reference server, unchanged since a896d67): slow_compute, confirm_delete, multi_input, failing_job and test_tool_with_task pass, with no stray tasks/cancel.

🤖 Generated with Claude Code

https://claude.ai/code/session_01JimHGjsfk2Ktm5GxyZJKKk