feat(adk): surface prompt-cache hit metrics via genai cached content token count - #2676
Open
MartinForReal wants to merge 1 commit into
Open
Conversation
…token count Populate GenerateContentResponseUsageMetadata.CachedContentTokenCount at each model adapter so provider cached prompt-token counts flow into A2A task usage and the llm_response trace attribute (closes kagent-dev#2669). - OpenAI Responses: map ResponseUsage.InputTokensDetails.CachedTokens - OpenAI chat (streaming + non-streaming): map PromptTokensDetails.CachedTokens - Anthropic (streaming + non-streaming): map Usage.CacheReadInputTokens - Bedrock Converse (streaming + non-streaming): map Usage.CacheReadInputTokens - Python OpenAI runtime: map prompt_tokens_details.cached_tokens Maps to 0 when the provider omits the field. The Go field is int32. Tests: openai responses + chat, and anthropic adapter cached-token mapping, plus python assertions for cached content token count. Signed-off-by: MartinForReal <fanshangxiang@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes #2669 — surfaces provider-reported prompt-cache hit / cached-token counts from the kagent model adapters into
genai.GenerateContentResponseUsageMetadata.CachedContentTokenCount, so cached prompt-token spend and prompt-cache hit rate are observable via A2A task usage and thellm_responsetrace attributes instead of being silently dropped.Adapter changes
Populate
CachedContentTokenCount(int32) at each adapter, mapping the provider's cached-prompt-token counter; defaults to 0 when the provider omits the field (nil-safe):go/adk/pkg/models/openai_responses.go:ResponseUsage.InputTokensDetails.CachedTokensgo/adk/pkg/models/openai_adk.go:PromptTokensDetails.CachedTokensgo/adk/pkg/models/anthropic_adk.go:Usage.CacheReadInputTokensgo/adk/pkg/models/bedrock.go:Usage.CacheReadInputTokenspython/packages/kagent-adk/src/kagent/adk/models/_openai.py:prompt_tokens_details.cached_tokensThis is gap 1 ("populate the cached-token counts in the adapters") from #2669. Emitting a dedicated
gen_ai.token.type=cachedmetric remains the separately-tracked follow-up surfaced in #2148/#2149, and upstream ADK already recordsgen_ai.usage.cache_read.input_tokensonceCachedContentTokenCountis non-zero.Tests
anthropic_adk_test.go— newTestAnthropicNonStreamingCachedTokensopenai_responses_test.go— responses non-streaming,GenerateContent, and streaming cached-token assertionsopenai_adk_test.go— chat completion cached-token assertiontests/unittests/models/test_openai.py— assertscached_content_token_countis populated and defaults to 0 when absent.Validation
go build ./...✅go vet ./...✅go test ./adk/pkg/...✅ (note:go/adk/pkg/sessionshows a pre-existing Windows-only TempDir lockfile cleanup failure unrelated to this change; all models package tests pass)gofmt -lon changed Go files: empty ✅files.pythonhosted.orgTLS (HandshakeFailure) downloads — an environment/network issue, not a code defect (recorded in notes; CI will exercise them).This branch is DCO-signed (
git commit -s). Not a draft.