🚰 llmtap

A transparent observability proxy for the Anthropic API.

Point your client at llmtap instead of the API — one environment variable, zero code changes — and every call is forwarded to Anthropic (streaming and all) while llmtap records a trace, tracks tokens / cost / cache in real time, and prints a live dashboard.

$ llmtap up --port 8400 --trace run.jsonl
llmtap listening on http://127.0.0.1:8400
  → upstream https://api.anthropic.com
  → trace    run.jsonl

  point your client here:
  export ANTHROPIC_BASE_URL=http://127.0.0.1:8400

   #  model                      in→out   cache         $     lat
   1  claude-opus-4-8           412→128       0   $0.0053    2.1s
   2  claude-opus-4-8          1620→64     1208   $0.0036    1.4s ~
   3  claude-opus-4-8           980→512       0   $0.0177    3.9s

── session ──
  calls       3  (0 errors)
  tokens      3,012 in / 704 out  (1,208 from cache)
  total spend $0.0266

---

Why

Most LLM observability means SDK wrappers, decorators, or a SaaS account. llmtap sits on the wire instead: it's a plain HTTP proxy, so anything that talks to the Anthropic API — any language, any framework, any agent — is instrumented by changing one env var.

• Zero integration. export ANTHROPIC_BASE_URL=http://localhost:8400. Done.
• Streaming-aware. SSE responses pass straight through to your client while llmtap reconstructs the message and usage for the trace.
• Cost & cache, live. Per-call tokens, dollars, and cache hits as they happen, with a session rollup on exit.
• One ecosystem. The trace it writes is the same format read by rewind (replay/fork calls) and cachelens (diagnose cache misses).
• Zero dependencies. Pure stdlib.

Install

pip install llmtap

Use

llmtap up                       # listens on 127.0.0.1:8400, writes llmtap.jsonl
llmtap up --port 9000 --trace myagent.jsonl
llmtap up -q                    # no per-call line, just record

Then run your app against it:

export ANTHROPIC_BASE_URL=http://127.0.0.1:8400
python your_agent.py

Your ANTHROPIC_API_KEY stays in your client and is forwarded upstream untouched — llmtap never stores it.

Afterwards, feed the trace to the rest of the toolkit:

cachelens doctor myagent.jsonl   # why is my cache missing?
rewind ls myagent.jsonl          # what did the agent actually do?

Verify it works (no key, no network)

git clone https://github.com/oavlloh-wq/llmtap && cd llmtap
pip install -e .
llmtap selftest

selftest spins up a fake upstream, runs the proxy against it, and checks forwarding, streaming passthrough, and trace recording end-to-end — entirely offline.

How it works

llmtap is a threaded HTTP proxy. For each request it forwards method, path, body, and headers (your API key included) to the upstream. Non-streaming responses are relayed and parsed for usage; streaming (SSE) responses are piped to your client byte-for-byte while llmtap accumulates the events to reconstruct the final message, stop reason, and token usage for the trace. It strips hop-by-hop headers and disables upstream compression so responses stay parseable. No request or response body is sent anywhere but to you and Anthropic.

Roadmap

• Web dashboard (live timeline, per-model rollups)
• Budget guardrails (warn / block past a spend threshold)
• Redaction hooks for PII before writing traces
• Multi-provider upstreams
• Prometheus / OpenTelemetry export

PRs welcome.

License

MIT

---

_{Part of a small LLM-agent toolkit: llmtap (capture) · rewind (replay) · cachelens (cache). Drop a ⭐ if it's useful.}