Skip to content

First-class OpenTelemetry support when running ADK agents outside Google Cloud #474

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
rwang1987 opened this issue Apr 30, 2025 · 4 comments
Assignees
Labels
core Issues related to the core interface and implementation

Comments

@rwang1987
Copy link

Is your feature request related to a problem? Please describe.
In our company, we are piloting the Agent Development Kit to run AI agents entirely on-prem (inside an air-gapped Google Distributed Cloud rack).
While opentelemetry-api and opentelemetry-sdk are part of ADK’s dependencies, the current documentation only explains how telemetry is emitted when agents run on Vertex AI Agent Engine. There is no guidance or helper for configuring OTLP exporters when we deploy agents on our own Kubernetes cluster.

Describe the solution you’d like

  • A simple telemetry config block (or CLI flags) that lets us specify:
    • OTLP endpoint / headers
    • Service name / version
    • Sampling strategy
  • Optional automatic instrumentation for:
    • LLM requests (trace_call_llm)
    • Tool invocations (trace_tool_call)
    • Agent-to-agent (A2A) messaging
  • Docs + example showing how to push traces to an on-prem collector (Jaeger, Tempo, GCP Ops Agent in hybrid mode, etc.).

Describe alternatives you’ve considered

  • Manually initialising the OTEL SDK in our main.py and decorating each callback.
    Works, but duplicates boilerplate and misses deeper internals (e.g. planner traces).
  • Exporting logs only – doesn’t give us span correlation across micro-services.

Additional context
We already have an OpenTelemetry collector running inside our data-centre; exposing ADK spans would let us view agent reasoning latency alongside the rest of our CI/CD traces.

Thanks for considering!

@aabmass
Copy link
Member

aabmass commented Apr 30, 2025

@rwang1987 have you seen #320? cc @michaelsafyan

@rwang1987
Copy link
Author

Thank you @aabmass and @michaelsafyan will you also help to update the ADK documentation to provide instructions of OTel export part? That will be super cool! If that's the case, I guess we can close this ticket?

@michaelsafyan
Copy link

If you don't mind, it would be helpful for tracking and for prioritization/acceptance discussions with other contributors/maintainers to leave this open until the request is fully resolved.

But, yes, we would also like to ensure that documentation is updated and contains appropriate examples.

Realistically, the pull request that Aaron linked to will probably need to be broken down into smaller pull requests to make it in (as well as for simplifying incremental testing of its various pieces). Is there anything you can share with us regarding the level of urgency of this request? (Also helpful as a prioritization signal for us).

@rwang1987
Copy link
Author

@michaelsafyan Yes, for sure, I would love to keep this ticket open!
And I'm working in Ciena, leading to build an AI agent infrastructure which helps our R&D organization to create AI agents for internal use. We are exploring to use Google ADK as the agentic framework, combine with AgentSpace deploy on-prem. And if we could get the Open Telemetry feature supported by ADK sooner, it will definitely help us a lot! Btw that we're currently interacting with Brendan Sikorski and couple other folks at Google on the on-prem deployment option with AgentSpace. If you or @boyangsvl could join our discussion, that would be super awesome! So that Ciena will be more confident to collaborate with Google with our AI infrastructure!
My work email: [email protected].
Looking forward to connect!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
core Issues related to the core interface and implementation
Projects
None yet
Development

No branches or pull requests

5 participants