Tracing LiteLLM in LangSmith — three approaches

LiteLLM gives you one API for 100+ model providers. LangSmith gives you one place to see what they're all doing. There are three ways to connect them — this repo demos all three, each following best practices. All guidance below is sourced from the official LangChain docs.

Setup

cp .env.example .env          # fill in LANGSMITH_API_KEY + a provider key
uv sync

uv run approach_1_traceable.py            # LangSmith SDK (@traceable)
uv run approach_2_callback.py             # LiteLLM langsmith callback (async)
uv run approach_3_langchain_litellm.py    # langchain-litellm (ChatLiteLLM, auto-traced)
uv run pipeline.py "How does LangSmith help debug LLM apps?"   # nested multi-step trace

Which approach should I use?

Approach	File	Use when	Tracing is…
1. @traceable	approach_1_traceable.py	Sync scripts; you want to trace your own pipeline logic around the call	Explicit (decorator)
2. LiteLLM callback	approach_2_callback.py	Async services; you want LiteLLM to emit model-level logs itself	Explicit (callback)
3. langchain-litellm	approach_3_langchain_litellm.py	You're in the LangChain/LangGraph ecosystem already	Automatic

→ Approaches 1 & 2: Trace LiteLLM · Approach 3: ChatLiteLLM, Routers & proxies

Best practices

1. Pick one instrumentation layer — don't double-trace. @traceable, the LiteLLM langsmith callback, and ChatLiteLLM each emit traces. Enabling more than one for the same call produces duplicate traces. Choose the row above that fits your app and stick to it. → Trace LiteLLM

2. Mark direct model calls with run_type="llm" and supply attribution. A raw LiteLLM call isn't a "supported integration," so LangSmith only renders a full LLM span when you tell it. The complete checklist: (1) run_type="llm", (2) OpenAI/Anthropic-format messages in/out, (3) ls_provider + ls_model_name in metadata, (4) usage_metadata for tokens. LiteLLM returns OpenAI-shaped responses, so (1)–(2) and token parsing come for free; approach_1 and pipeline.py set (3) dynamically from the response so cost attribution works per model. (Approach 3 handles all of this automatically.) → Log LLM calls

3. Wrap each step for nesting — not just the entrypoint. @traceable propagates trace context automatically: a traced function calling other traced functions yields one parent run with child runs per step. That tree (per-step latency + I/O) is the payoff. pipeline.py shows it: research → plan / investigate / synthesize → call_model. → Annotate code with @traceable

4. Tag and annotate traces with metadata. Attach environment, user_id, version, or a correlation ID — via the decorator (tags=, metadata=), per-call langsmith_extra={...}, or the LangChain config={"tags": [...], "metadata": {...}} (shown in approach_3). This is what makes traces filterable later. → Add metadata and tags

5. In production, sample and/or trace conditionally. Sampling = probabilistic volume control (statistically representative); conditional tracing = deterministic control (disable for zero-retention clients, route by tenant). Combine both for cost control at scale. → Sample traces · Conditional tracing

6. Mask sensitive data instead of disabling tracing. When inputs/outputs contain PII, mask them rather than turning tracing off — you keep structure and latency without storing the payload. → Mask inputs and outputs

7. Flush before short-lived processes exit. Traces ship in the background, so scripts can drop the last one on exit. With the callback (approach_2), set litellm.langsmith_batch_size = 1 and add a brief await asyncio.sleep(1) before the process ends. → Trace LiteLLM

8. Set the region endpoint if you're not on US. Default is US (GCP). For EU/APAC/AWS set LANGSMITH_ENDPOINT (e.g. https://eu.api.smith.langchain.com) or your key won't authenticate. → Trace with LangChain

Files

File	What it shows
approach_1_traceable.py	LangSmith SDK @traceable around a completion() call, with cost attribution.
approach_2_callback.py	Async acompletion() with litellm.success_callback = ["langsmith"].
approach_3_langchain_litellm.py	ChatLiteLLM — auto-traced LangChain runnable, tags/metadata via config.
pipeline.py	Nested multi-step trace; swap a model string to route across providers in one trace.
.env.example	Required env vars.