fix(eve/harness): allow explicit bedrock anthropic inference-profile override (#1314) - #1532
Open
iroiro147 wants to merge 1 commit into
Open
fix(eve/harness): allow explicit bedrock anthropic inference-profile override (#1314)#1532iroiro147 wants to merge 1 commit into
iroiro147 wants to merge 1 commit into
Conversation
…override (vercel#1314) `detectPromptCachePath` inferred the `anthropic-direct` path for Bedrock only when the resolved model id contained `anthropic`. AWS application inference profile ids can be opaque (`us.app-profile-…`), so a Bedrock-resolved Anthropic model received no `cacheControl`/`cachePoint` markers and repeated turns paid full uncached input cost. Introduce an optional `PromptCacheHint` channel. Agents can declare the underlying provider family via `modelOptions.providerOptions.bedrock.inferenceProfileTarget = "anthropic"`; the tool-loop reads the hint from the resolved `RuntimeModelReference` and passes it to `detectPromptCachePath`. The classifier consults hints only after provider-name and model-id heuristics and only when the provider name already includes `bedrock`, so a hint cannot accidentally rewrite the classification of a non-Bedrock provider. Refs vercel#1314 Signed-off-by: Sarthak Singh <sarthak.singh@juspay.in>
Contributor
|
@iroiro147 is attempting to deploy a commit to the Vercel Team on Vercel. A member of the Team first needs to authorize it. |
This was referenced Aug 2, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
AWS Bedrock application inference profiles can have opaque ids that omit the
underlying provider name (
us.app-profile-7f3a2b1c).detectPromptCachePathinferred the
anthropic-directpath only when the resolved model id containedanthropic, so an Anthropic model reached through an opaque profile classifiedas
noneand received nocacheControl/cachePointmarkers — repeatedturns paid full uncached input cost (#1314).
Changes
prompt-cache.ts— addPromptCacheHintinterface and accept it indetectPromptCachePath. The Bedrock branch now checkshints.bedrockInferenceProfileTarget === "anthropic"after the model-idheuristic. Hints are gated on
providerName.includes("bedrock")andconsulted after the heuristic so they cannot accidentally rewrite
non-Bedrock providers or disable an inferred path.
tool-loop.ts—resolveActiveRuntimeModelreturns the activeRuntimeModelReference; the step runner derivesPromptCacheHint | undefinedvia the new
resolvePromptCacheHintshelper (readsreference.providerOptions.bedrock.inferenceProfileTarget === "anthropic"),then passes the hint to
detectPromptCachePath.Author usage
Tests
Five new
detectPromptCachePathcases inprompt-cache.test.tscovering theopaque-profile no-hint path, the override path, the non-Bedrock guard, and
hint-vs-heuristic precedence. Full harness workspace suites pass:
buzz-agent --libproxy of 703/703 (46 files),tsc -p tsconfig.json --noEmitclean.
Refs #1314