fix(llm): reconcile cache-hit accounting between the two usage meters

[oratis]

Summary

Follow-up to #20. That PR stopped cache hits from billing real money, but closed only half the gap — and the half it left open is the one that matters for /api/usage being a billing basis.

The two meters still disagreed — on tokens

#20 zeroed usdCents while deliberately keeping the token counts "for visibility". Those tokens still reach agent_runs. But getStats() skips cache hits entirely, so:

• money agreed at zero ✅
• tokens — SUM(agent_runs.input_tokens) ran ahead of the in-memory counters by exactly the cached volume, with nothing anywhere recording why ❌

A ledger meant to back tiered billing was unexplainable from the other meter: it just looked inexplicably larger.

Cache hits now land in their own stats.cached bucket, whose usdCents is spend avoided rather than incurred. Two identities hold per process:

SUM(agent_runs.cost_usd_cents) === totalUsdCents
SUM(agent_runs.input_tokens)   === Σ byProvider[*].inputTokens + cached.inputTokens

Money agrees outright; tokens agree once cached volume is added back. I kept #20's cost-based semantics (bill for money that left the building) rather than reopening the pricing question — switching to usage-based is a product decision, and it belongs in complete(), which is the only place that knows a response was cached.

A zero cost was being re-inflated into a charge

content-video.js read res.usage?.usdCents || 25. Zero is a real answer — a cache hit reports it by design, and so does any live call on a model absent from PRICING, or one small enough to round down. || read all three as "cost unknown" and charged the 25¢ estimate anyway, billing for spend that never happened. Now ??, so only genuinely absent usage falls back.

Docs

usage-ledger.js now documents the semantics it depends on, so nobody "repairs" a well-cached month (many tokens, little spend) by re-deriving cost from token counts.

Scope worth recording

The original over-billing was latent, not active. Only calls with temperature 0-or-absent are cacheable, and the sole such call site — community.js:210 (inbox classify) — discards res.usage entirely. Every other one of the 15 call sites passes a non-zero temperature, so the cache is never consulted. No cached cost has ever reached agent_runs or /api/usage.

This is contract hygiene for the 13 of 15 call sites that do forward usage — the trap springs the moment anyone adds a cacheable call.

Test plan

• npm test 665/665 (9 new)
• Each new test verified to fail without its fix: reverting ?? → || fails 1; removing recordCacheHit() fails 4
• One test guards against zeroing the stored cache entry instead of a per-hit copy, which would silently decay cachedUsdCents to zero after the first read
• No client changes — the new stats.cached field rides into /api/agents/cost's llmStats additively, and no client code reads it

Not done deliberately

Option (c) from the original write-up — a persisted cached_usd_cents column on agent_runs for gross-vs-net reporting. Under the shipped cost-based semantics it isn't needed for correctness, and cachedUsdCents doesn't reach the DB anyway (agent-runtime/index.js picks only tokens + usdCents). It would need a migration on the dual SQLite/Postgres path — worth its own decision.

Unrelated, spotted in passing

server/usage-ledger.js contains two raw NUL bytes (not the \0 escape) used as a map-key separator, so git classifies the file as binary — git diff shows Bin 9679 -> 10779 bytes instead of a real diff, and git blame is useless on it. It predates this branch (identical in HEAD, from #17) and is invisible in review precisely because the file can't be diffed. Being fixed separately; flagging it here because it's why this PR's diff for that file renders as binary.

🤖 Generated with Claude Code