Skip to content

feat(adk): surface prompt-cache hit metrics via genai cached content token count - #2676

Open
MartinForReal wants to merge 1 commit into
kagent-dev:mainfrom
MartinForReal:feat/prompt-cache-cached-token-metrics
Open

feat(adk): surface prompt-cache hit metrics via genai cached content token count#2676
MartinForReal wants to merge 1 commit into
kagent-dev:mainfrom
MartinForReal:feat/prompt-cache-cached-token-metrics

Conversation

@MartinForReal

Copy link
Copy Markdown

Summary

Closes #2669 — surfaces provider-reported prompt-cache hit / cached-token counts from the kagent model adapters into genai.GenerateContentResponseUsageMetadata.CachedContentTokenCount, so cached prompt-token spend and prompt-cache hit rate are observable via A2A task usage and the llm_response trace attributes instead of being silently dropped.

Adapter changes

Populate CachedContentTokenCount (int32) at each adapter, mapping the provider's cached-prompt-token counter; defaults to 0 when the provider omits the field (nil-safe):

  • OpenAI Responsesgo/adk/pkg/models/openai_responses.go: ResponseUsage.InputTokensDetails.CachedTokens
  • OpenAI Chat (streaming + non-streaming) — go/adk/pkg/models/openai_adk.go: PromptTokensDetails.CachedTokens
  • Anthropic (streaming + non-streaming) — go/adk/pkg/models/anthropic_adk.go: Usage.CacheReadInputTokens
  • Amazon Bedrock Converse (streaming + non-streaming) — go/adk/pkg/models/bedrock.go: Usage.CacheReadInputTokens
  • Python OpenAI runtimepython/packages/kagent-adk/src/kagent/adk/models/_openai.py: prompt_tokens_details.cached_tokens

This is gap 1 ("populate the cached-token counts in the adapters") from #2669. Emitting a dedicated gen_ai.token.type=cached metric remains the separately-tracked follow-up surfaced in #2148/#2149, and upstream ADK already records gen_ai.usage.cache_read.input_tokens once CachedContentTokenCount is non-zero.

Tests

  • Go unit tests added/updated for the cached-token mapping:
    • anthropic_adk_test.go — new TestAnthropicNonStreamingCachedTokens
    • openai_responses_test.go — responses non-streaming, GenerateContent, and streaming cached-token assertions
    • openai_adk_test.go — chat completion cached-token assertion
  • Python tests/unittests/models/test_openai.py — asserts cached_content_token_count is populated and defaults to 0 when absent.

Validation

  • go build ./...
  • go vet ./...
  • go test ./adk/pkg/... ✅ (note: go/adk/pkg/session shows a pre-existing Windows-only TempDir lockfile cleanup failure unrelated to this change; all models package tests pass)
  • gofmt -l on changed Go files: empty ✅
  • Python unit tests for the changed file could not be executed in this workspace because dependency installation failed on repeated files.pythonhosted.org TLS (HandshakeFailure) downloads — an environment/network issue, not a code defect (recorded in notes; CI will exercise them).

This branch is DCO-signed (git commit -s). Not a draft.

…token count

Populate GenerateContentResponseUsageMetadata.CachedContentTokenCount at each
model adapter so provider cached prompt-token counts flow into A2A task usage
and the llm_response trace attribute (closes kagent-dev#2669).

- OpenAI Responses: map ResponseUsage.InputTokensDetails.CachedTokens
- OpenAI chat (streaming + non-streaming): map PromptTokensDetails.CachedTokens
- Anthropic (streaming + non-streaming): map Usage.CacheReadInputTokens
- Bedrock Converse (streaming + non-streaming): map Usage.CacheReadInputTokens
- Python OpenAI runtime: map prompt_tokens_details.cached_tokens

Maps to 0 when the provider omits the field. The Go field is int32.

Tests: openai responses + chat, and anthropic adapter cached-token mapping,
plus python assertions for cached content token count.

Signed-off-by: MartinForReal <fanshangxiang@gmail.com>
@MartinForReal
MartinForReal requested review from a team and supreme-gg-gg as code owners September 3, 2026 07:20
@github-actions github-actions Bot added the enhancement New feature or request label Sep 3, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[FEATURE] Surface prompt-cache hit metrics (gen_ai cached tokens) from model adapters / ADK metrics

1 participant