Skip to content

feat(archive): add agent turn-metric (kind 44200) local archive#1555

Merged
wpfleger96 merged 5 commits into
mainfrom
duncan/agent-metric-archive
Jul 6, 2026
Merged

feat(archive): add agent turn-metric (kind 44200) local archive#1555
wpfleger96 merged 5 commits into
mainfrom
duncan/agent-metric-archive

Conversation

@wpfleger96

@wpfleger96 wpfleger96 commented Jul 6, 2026

Copy link
Copy Markdown
Collaborator

Summary

Extends the local-save archive primitive from #1442 to add a "subscribe to all my agents' turn metrics" special case. Kind-44200 (NIP-AM agent turn metrics) is archived via the persistent /query proof path — unlike the observer frames (24200) which use the ephemeral local-validation path, the relay stores and #p-gates 44200 events so relay verification suffices.

Content is decrypted at ingest and stored as plaintext AgentTurnMetricPayload JSON, allowing token-usage calculators to read archive.db directly without the owner key. Fail-closed: decrypt error → drop, never store ciphertext or garbage.

Default off for OSS builds; baked on for internal builds via BUZZ_BUILD_AGENT_METRIC_ARCHIVE_DEFAULT (enabled in buzz-releases#50).

Changes

Rust (src-tauri):

  • archive/pipeline.rs: route owner_p+44200 to the persistent bucket; add #p filter arm; change WriteRow to owned String fields to support stored plaintext; decrypt kind-44200 rows at commit time with buzz_core_pkg::agent_turn_metric::decrypt_agent_turn_metric — fail-closed on any error
  • archive/store.rs: add merge_owner_p_kinds — reads existing kinds JSON, unions in the new kind, and writes back inside a single unchecked_transaction() so concurrent seeders can't clobber the shared owner_p row
  • archive/mod.rs: add KIND_AGENT_TURN_METRIC = 44200 constant; thread owner_keys clone into commit_archive; add merge_save_subscription_kinds Tauri command; 4 pipeline unit tests + run_batch_sync_with_keys helper (routing, decrypt-success-plaintext, decrypt-fail-closed, 24200-still-ephemeral)
  • commands/agent_metric_archive.rs (new): agent_metric_archive_default_enabled() Tauri command + OSS-default-false unit test; registered in commands/mod.rs + lib.rs
  • build.rs: rerun-if-env-changed + presence remap for BUZZ_BUILD_AGENT_METRIC_ARCHIVE_DEFAULT

TypeScript:

  • kinds.ts: add KIND_AGENT_TURN_METRIC = 44200
  • agentMetricArchivePreference.ts (new): identity-scoped localStorage gate (key prefix buzz:agent-metric-archive-default-seeded)
  • useAgentMetricArchiveSeed.ts (new): seeds owner_p [44200] once/identity on internal builds via the atomic mergeSaveSubscriptionKinds command — the union happens under the DB transaction, so a concurrent observer seed can't clobber it
  • useObserverArchiveSeed.ts: switched to the same atomic mergeSaveSubscriptionKinds path (symmetric fix — the shared-row collision was latent here too)
  • tauriArchive.ts: add agentMetricArchiveDefaultEnabled() and mergeSaveSubscriptionKinds(kind) bindings
  • AppShell.tsx: call useAgentMetricArchiveSeed next to useObserverArchiveSeed
  • LocalArchiveSettingsCard.tsx: separate AgentMetricArchiveSection toggle; both toggles merge kinds so observer + metric subscriptions coexist in the single owner_p row (PK is scope_type+scope_value)
  • useAgentMetricArchiveSeed.test.mjs (new) + useObserverArchiveSeed.test.mjs: mirrored suites covering seed-once, merge, and concurrent-seed behavior

Stack

Stack: #1442 → this PR

Related: squareup/buzz-releases#50 (bakes BUZZ_BUILD_AGENT_METRIC_ARCHIVE_DEFAULT on for internal builds)

@wpfleger96 wpfleger96 marked this pull request as draft July 6, 2026 21:18
Base automatically changed from duncan/local-save-archive to main July 6, 2026 21:41
npub1mn7jgtj4w2pd0g0zeuhxsa6jy6p0rewxz4kujt98my82ahfmp72sxjexk7 and others added 4 commits July 6, 2026 17:45
Subscribe to all owned agents' turn-metric events (kind 44200) as a
special case of the #1442 local-save archive primitive. Mirrors the
observer-feed (24200) wiring but uses the persistent /query proof path
since the relay stores and #p-gates 44200 events.

Key design decisions:
- owner_p+44200 routes to the persistent bucket (relay is source of
  truth); owner_p+24200 stays on the ephemeral path unchanged.
- Decrypt at ingest (NIP-44, agent→owner); store plaintext payload JSON
  so token-usage calculators can read archive.db directly without the
  owner key. Fail-closed: decrypt error → drop, never store ciphertext.
- owner_p filter arm added to plan_archive: {ids, #p, kinds}.
- Both seed hooks (observer + metric) now merge existing owner_p kinds
  before upserting, preventing concurrent first-run seeds from clobbering
  each other. Observer toggle likewise merges rather than stomping.
- Build-time flag BUZZ_BUILD_AGENT_METRIC_ARCHIVE_DEFAULT (presence
  remap → BUZZ_DESKTOP_BUILD_AGENT_METRIC_ARCHIVE_DEFAULT=1); default
  off for OSS, baked on for internal builds via buzz-releases.
- Separate toggle in LocalArchiveSettingsCard with its own enabled state.
- KIND_AGENT_TURN_METRIC = 44200 added to kinds.ts.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
…merge

Two concurrent seed hooks (observer 24200 + metric 44200) each fire on
first internal-build run with no prior owner_p row. The old TS-side
read-then-write merge raced: each hook read [] before the other had
written, so last writer clobbered the first kind.

Add store::merge_owner_p_kinds: reads existing kinds JSON, unions in
new_kind, and writes back under a single SQLite unchecked_transaction.
Concurrent callers serialize on the SQLite write lock — last writer
always reads the first writer's committed row.

Add merge_save_subscription_kinds Tauri command in archive/mod.rs (same
module as the other subscription commands; registered in lib.rs).

Both seed hooks and their deps interfaces now call this single command
with just their own kind, replacing the list+merge+create pattern.
tauriArchive.ts gains a mergeSaveSubscriptionKinds() binding.

Tests added:
- store.rs: create-when-none, adds-kind-to-existing, idempotent,
  concurrent-interleave (observer writes first, metric second, both
  kinds survive)
- useAgentMetricArchiveSeed.test.mjs: updated to new deps interface,
  adds test_concurrent_seeds_both_kinds_survive (Promise.all interleave)
- useObserverArchiveSeed.test.mjs: symmetric update to new interface

File-size overrides bumped: archive/mod.rs +30 (command), store.rs
+110 (new fn + 4 tests — first override for store.rs).

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
Replace DEFERRED unchecked_transaction() with BEGIN IMMEDIATE via
execute_batch so the write lock is acquired before the SELECT.

With DEFERRED, two concurrent first-run seed hooks (observer + metric)
both read the empty owner_p snapshot, then race to write; the loser
receives SQLITE_BUSY_SNAPSHOT which busy_timeout does not retry, leaving
one kind silently absent until app restart.

With IMMEDIATE, the second caller blocks at BEGIN until the first
commits, then reads the committed row and merges its kind in. Both
callers succeed and the final row contains [24200, 44200].

Also fixes the inaccurate doc-comment that claimed DEFERRED gave an
exclusive write lock for the read-modify-write (it did not), and
replaces the mislabeled sequential-single-connection test with a real
two-connection WAL regression test using std::thread + Barrier that
exercises the actual concurrent write path.

Co-authored-by: Will Pfleger <pfleger.will@gmail.com>
Signed-off-by: Will Pfleger <pfleger.will@gmail.com>
@wpfleger96 wpfleger96 force-pushed the duncan/agent-metric-archive branch from 41146f8 to e7d4de2 Compare July 6, 2026 21:47

@wesbillman wesbillman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM — approve. CI is fully green (all checks pass/skip, incl. both E2E Integration shards). The design is sound and the risky bits are well-tested.

What's good

  • Fail-closed decrypt at ingest is the right call — decrypt error → drop, never store ciphertext or garbage. Both the success path (plaintext stored, ciphertext verifiably absent) and the wrong-key path (0 rows written) have direct unit tests.
  • Routing split is conservative: 44200 → persistent /query proof path, 24200 stays ephemeral, and unknown kinds under owner_p default to ephemeral rather than silently getting relay-proofed. Tested in both directions.
  • merge_owner_p_kinds with BEGIN IMMEDIATE is a genuinely good TOCTOU fix, and the two-connection WAL regression test (tempfile + Barrier + real threads) exercises the actual race instead of a sequential approximation. It also retro-fixes the same latent clobber in the observer seed — nice symmetric cleanup.
  • Kind range validation on the Tauri command boundary, rerun-if-env-changed for the new build flag, OSS-default-false test.

Minor notes (follow-up material, not blockers)

  1. The settings-card toggles still do the non-atomic TS-side merge the seed hooks just got fixed for. handleObserverToggle/handleMetricToggle read kinds from React subs state, then createSaveSubscription overwrites the full list — and toggle-off can deleteSaveSubscription the whole shared owner_p row if state is stale, taking the other kind with it. Risk is low (user-paced clicks, reload() after each), and there's no atomic remove-kind command yet so it can't fully use the new path — but a remove_save_subscription_kind sibling would close the loop. Worth a queued follow-up.
  2. Mixed row shapes in archived_events: 44200 rows now store payload JSON while everything else stores Nostr Event JSON. Any consumer doing Event::from_json(raw_json) on an unfiltered owner_p read will choke. Today's only reader (useObserverEvents) filters kinds: [24200] so it's safe, but this is an invariant future readers must know — a one-liner in the read_archived_events doc comment would cheaply pin it.
  3. check-file-sizes overrides keep ratchetingmod.rs 1465→1700 is its second bump with "split queued" noted both times. The justifications are honest, but the mod_tests.rs split should actually land soon or the override comment block becomes the debt.
  4. Plaintext-at-rest is a deliberate tradeoff (stated in the PR body): E2E-encrypted metrics become plaintext in local archive.db so calculators can read without the owner key. Local-only DB, owner's own machine, opt-in for OSS — reasonable, just noting it's a real design decision, not an accident.

Reviewed by Pinky (Buzz agent) on behalf of Wes.

@wpfleger96 wpfleger96 enabled auto-merge (squash) July 6, 2026 22:23
@wpfleger96 wpfleger96 merged commit 429f54b into main Jul 6, 2026
25 checks passed
@wpfleger96 wpfleger96 deleted the duncan/agent-metric-archive branch July 6, 2026 22:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants