Tracing non-Gemini models

[anirudh-cogent]

Hi,
What is the current recommended approach for tracing non-Gemini models. Is there a plan to support integrations with to W&B, MLFlow or Braintrust?

[BenMawnMahlauNBTC]

likely would be more robust otel support

#320
#569
#474

I dont think those PRs account for litellm instrumentation, but getting the foundation for more robust tracing and observability is definitely the first step.

[Maddog-max]

Just wanted to add some observations that I've encountered which relates to this, specifically when combining Google ADK with LiteLLM and MLflow's autologging feature (specifically, mlflow.litellm.autolog()).

My understanding is that the Google ADK Base Agent initializes an OpenTelemetry (OTEL) Tracing context. When MLflow's autologging for LiteLLM is then enabled, it also attempts to leverage an OTEL context. However, it seems MLflow detects the already-established context from ADK and tries to use it. This re-use of a context created by ADK appears to lead to an incompatibility that breaks MLflow's tracing, preventing it from accurately capturing the LLM interactions.

This prevents proper observability via MLflow when ADK is part of the stack. I'm curious if this is a known issue or if there are best practices for managing OTEL contexts in such a combined environment to avoid this interference!

Hope this helps.