Skip to content

feat(adk): expose gen_ai token usage Prometheus metric - #2673

Open
MartinForReal wants to merge 1 commit into
kagent-dev:mainfrom
MartinForReal:feat-go-token-metrics
Open

feat(adk): expose gen_ai token usage Prometheus metric#2673
MartinForReal wants to merge 1 commit into
kagent-dev:mainfrom
MartinForReal:feat-go-token-metrics

Conversation

@MartinForReal

Copy link
Copy Markdown

What

Implements the GenAI token-usage metrics observability gap from #2148 for the Go ADK runtime. The Go ADK (google.golang.org/adk/v2) ships spans only, no metric instruments, so kagent supplies the stdlib Prometheus instrument itself, in the shape of the open/dirty #2149 but rebuilt from scratch against today's go/adk source and aligned with the Python runtime's attribute set per #2458.

How

  • pkg/telemetry/metrics.go — a gen_ai_client_token_usage Prometheus histogram (semconv gen_ai.client.token.usage, semconv-recommended buckets) on the default registry.
  • Labels align with what upstream Google ADK Python emits for the same instrument (one dashboard across runtimes): gen_ai_token_type (input/output), gen_ai_request_model, gen_ai_response_model (falls back to request model), gen_ai_provider_name (semconv well-known value via SemconvProviderName), gen_ai_agent_name, gen_ai_operation_name (chat), error_type.
  • Gating — recording and the /metrics endpoint are default-OFF behind OTEL_METRICS_ENABLED=true, matching the existing OTEL_TRACING_ENABLED / OTEL_LOGGING_ENABLED gates.
  • pkg/a2a/server/server.go — serves /metrics on the A2A mux when the gate is on, excluded from request tracing/span flushing.
  • pkg/a2a/executor.go — records token usage from the ADK AfterEventCallback: skips partial (streaming) events so one observation is recorded per LLM call; input = PromptTokenCount, output = CandidatesTokenCount + ThoughtsTokenCount from UsageMetadata.
  • cmd/main.go / pkg/config — model + provider labels resolved at startup; agent name from AppName.
  • docs/genai-token-metrics.md — documents the metric, labels, gating, and scraping.

Testing

  • go build ./adk/..., go vet, and golangci-lint (repo config) are clean.
  • go test -race passes for the changed packages (telemetry, a2a, a2a/server, config).
  • Two focused test files: metrics_test.go (record / skip-zero / disable-gate / response-model fallback / labels / provider-name mapping / /metrics handler) and executor_metrics_test.go (partial events not double-counted, per-LLM-call aggregation).

Notes / decisions

  • Pod scrape annotations: the annotation piece of chore(deps): bump the go-minor-patch group in /go with 3 updates #2099 targeted go/core/internal/controller/translator/agent/manifest_builder.go, which no longer exists on main after the controller/runtime rework. That path was dropped rather than reinvented; operators can add prometheus.io/* annotations and this is documented. Flag if you want the annotation automation wired against the current runtime-provisioning location.
  • make -C go lint could not be run here: make and the kube-api-linter plugin are unavailable in this environment. Lint was run with the repo's .golangci.yaml minus the plugin (0 issues); kube-apilinter only lints api/v1alpha CRD paths, untouched here.

Refs: #2148

Record the OpenTelemetry GenAI-semconv gen_ai.client.token.usage metric from the Go ADK runtime using the native Prometheus client, served at /metrics. Labels follow the upstream Google ADK Python runtime so one dashboard works across both runtimes (gen_ai.token.type, gen_ai.request.model, gen_ai.response.model, gen_ai.provider.name, gen_ai.agent.name, gen_ai.operation.name, error.type). Recording and the scrape endpoint are gated default-OFF behind OTEL_METRICS_ENABLED, matching the existing OTEL_TRACING_ENABLED / OTEL_LOGGING_ENABLED gates.

Signed-off-by: MartinForReal <fanshangxiang@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant