feat(usage): surface prompt-cache tokens in usage maps

[a-florian-h]

Closes the visibility half of fosferon/gc_daemon#3 (finding F2).

Problem

OpenAI-compatible providers report prompt-cache hits in usage.prompt_tokens_details.cached_tokens (OpenAI, z.ai, Kimi, DeepSeek); Anthropic reports cache_read_input_tokens / cache_creation_input_tokens. usage_from_openai/1 and the Anthropic usage map dropped all of them — so callers can neither measure cache hit rates nor compute true input cost, and cost estimators re-bill cached tokens at full price.

As of Aug 2026 every major provider discounts cached input ~90% (z.ai GLM-5.x ~81%: $0.26 vs $1.40/1M), and agent workloads are the best-case consumer (a large stable tool-manifest + system prefix resent every turn). Measured on z.ai via a temporary log line: a 5-turn agent conversation shows 8,384/8,437 cached on steady-state turns — a ~99% hit rate that was completely invisible.

Change

• openai.ex: usage_from_openai/1 adds :cached_tokens when prompt_tokens_details.cached_tokens is present. Covers both the non-streaming response and the streamed {:usage, _} event (extract_stream_event/1 shares the parser). Providers that don't report it are unaffected — the key is simply omitted.
• anthropic.ex: usage map adds :cached_tokens (from cache_read_input_tokens) and :cache_creation_input_tokens (writes, billed 1.25x).
• response.ex: document the extended usage shape.
• Tests: streamed final-usage event with/without prompt_tokens_details (values from the real z.ai capture above).

No behavior change beyond the extra usage keys.

Verification

mix test: 345 tests + 2 properties, 0 failures (includes the 2 new cases).

Related

• fosferon/gc_daemon#3 — the umbrella issue (F1 prefix stability, F3 usage ledger, F4 daemon-side output trimming).
• Note for gc_daemon consumers: its dep pin ~> 0.4.0 predates streaming-usage support entirely (added 0.4.1+); upgrading to 0.6.x gets include_usage streaming + this change together.

[Copilot]

Pull request overview

Surfaces provider-reported prompt-cache token counts in the standardized usage maps so downstream callers can measure cache hit rates and compute discounted cached-input costs correctly.

Changes:

• OpenAI: parse and expose usage.prompt_tokens_details.cached_tokens as :cached_tokens in usage maps (including streamed final {:usage, ...} events).
• Anthropic: expose prompt-caching usage as :cached_tokens (cache_read_input_tokens) and :cache_creation_input_tokens (cache_creation_input_tokens).
• Documentation + tests: document the extended usage shape and add streaming-usage tests for cached vs non-cached usage payloads.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated no comments.

File	Description
test/llm_core/llm/openai_test.exs	Adds streaming tests verifying :cached_tokens is surfaced (or omitted) in final usage events.
lib/llm_core/llm/response.ex	Updates Response moduledoc to document the extended usage keys for caching.
lib/llm_core/llm/openai.ex	Adds maybe_put_cached_tokens/2 and wires it into OpenAI usage parsing.
lib/llm_core/llm/anthropic.ex	Extends Anthropic usage map to include cache read/write token counts.

Suppressed comments (1)

lib/llm_core/llm/openai.ex:261

• usage_from_openai/1 only attaches :cached_tokens for the (prompt_tokens + completion_tokens) usage shape. If an OpenAI-compatible provider returns a usage map containing prompt_tokens_details.cached_tokens alongside only total_tokens (the fallback clause), :cached_tokens will be silently dropped, which undermines the stated goal of surfacing cache hits whenever they’re reported.

    |> maybe_put_cached_tokens(usage)
  end

  defp usage_from_openai(%{"total_tokens" => total}) do
    %{total_tokens: total}

---

💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.