Skip to content

Commit debd3c7

Browse files
authored
feat(settings): user-facing prompt-cache TTL for Anthropic profiles (C6) (#2271)
* fix(openai-native): send body prompt_cache_key for cache affinity What: the OpenAI-native handler now sends a task-scoped `prompt_cache_key` in the Responses request body on both body builders (streaming and completePrompt), and no longer sends the `session_id` request header. The key is `metadata.taskId`, falling back to the handler session id, clamped to OpenAI's 64-character limit, and omitted for models whose catalog entry has `supportsPromptCache: false`. Why: OpenAI keys prompt-cache routing off the body `prompt_cache_key` field. The `session_id` header we were sending is not part of the prompt-caching API and is ignored on the public Responses API, so every OpenAI-native task was leaving cache hit-rate on the table. The header also carries an underscore in its name, which hardened gateways (Envoy/nginx `underscores_in_headers`) reject outright, so it was a latent 400 for anyone behind such a proxy with a custom base URL. `prompt_cache_retention: "24h"` is untouched: it is the supported lifetime control on the pre-5.6 models that set it in the catalog, and the existing retention specs pass unmodified. Impact: cache affinity now follows the documented body field, stable across turns of one task and distinct across tasks. No usage accounting, cost, or other request field changes. openai-codex.ts and xai.ts are untouched: codex already sends a body key, and xai never had the header. * fix(cost): account cache writes per TTL class instead of one aggregate rate What: - Add `cacheWritesPrice1h` / `cacheWritesPrice12h` to the ModelInfo schema plus `deriveCacheWritesPrice1h` / `deriveCacheWritesPrice12h`, so the vendor multiplier relationship is stated once in model.ts. - Bill three write classes in `calculateApiCostInternal`: the 5m remainder is derived as total - 1h - 12h (clamped at 0), so the classes always sum to the aggregate and no token is billed twice or dropped. - Scale the TTL rates with `cacheWritesPriceMultiplier` in long-context pricing. - Carry `cacheWrite1hTokens` / `cacheWrite12hTokens` on `ApiStreamUsageChunk` and read them from Anthropic `usage.cache_creation.ephemeral_1h_input_tokens` (anthropic, minimax, anthropic-vertex) and OpenAI Responses `input_tokens_details.cache_write_tokens_details.ttl_12h_tokens` (openai-native `normalizeUsage` only). - Thread the split through the Task usage accumulator, including the cancel-path cost recomputation that runs when a provider yields no totalCost. - Populate the rates: 22 Anthropic entries, 19 Bedrock Anthropic-family entries, 7 OpenAI GPT-5.6/6-class entries. Why: Anthropic bills a 1h cache write at 2x input versus 1.25x for 5m, and OpenAI bills a 12h write at 2.5x versus 1.25x for 30m. The repo read only the aggregate `cache_creation_input_tokens` and priced every write at the short rate, so a 1h write under-reported by 37.5% and a 12h write by 50%. The vendors already report the per-TTL split; it was being discarded. Impact: Reporting only. No request shape changes: the repo emits no `ttl` field, so live traffic still reports 0/absent long-lived writes and pricing is byte-identical to before (asserted against the historical cost literals). When the TTL knob lands, a 1h write can no longer be billed at the 5m rate. 18 new tests fail on the base blob; the new cost code has no surviving mutants. * fix(cost): bill the TTL split on the telemetry recompute and in tier blocks What: - Pass `cacheWrite1h`/`cacheWrite12h` to the cost call inside `captureUsageData`. The bag already carried them and fed the shared accumulator vars, but the telemetry recompute passed only the aggregate write count, so a cancelled request's telemetry billed every write at the short-TTL rate. - Add `cacheWritesPrice1h`/`cacheWritesPrice12h` to the `tiers` schema and to the four Anthropic and four Bedrock 1M-context tier blocks, derived from each tier's own input price. - Carry the tier's TTL rates through all three tier-merge seams (anthropic, anthropic-vertex, bedrock) and derive them from the tier input price in the OpenAI service-tier seam, so a flex/priority or 1M-context write is not billed at the base tier's TTL rate. Why: Two under-reports survived the first pass. `captureUsageData` recomputes cost for telemetry when a provider yields no final cost chunk (the cancelled-stream path), and it dropped the split: 100k 1h tokens at claude-3-5-sonnet billed 0.375 instead of 0.6. Separately, the 1M-context tier merge overrode `inputPrice` but not the TTL rate, so the entry-level rate derived from the 200K input price survived: a 1M-context 1h write billed 6.0 instead of the tier's 12.0, a 50% under-report. Impact: Both paths now bill at the correct TTL rate. Three new tests fail on the pre-fix code (0.375 vs 0.6, and 6 vs 12); a fourth asserts the tier rate reaches the cost function. No request shape changes. * test(cost): correct the mixed-split test label to match its arithmetic What: - Rename "bills a mixed 5m/1h split per class (0.465 case)" to "(60/40 split, 0.51)". Why: The label cited a figure the test does not assert. A 60/40 split of 100k writes at 3.75/6.0 costs 0.15 + 0.36 = 0.51, not 0.465. The assertion was already correct; only the name was misleading, and a reviewer reading the label would have expected a different number. Impact: Test name only, no behaviour change. * test(openai-native): cover the 12h rate under each service tier What: - Add a per-tier case asserting the 12h write rate follows the tier's own input price (default 25.0, flex 12.5, priority 50.0 for gpt-6-astra). Why: The service-tier seam was changed to derive the TTL rates from the tier input price, but nothing pinned that behaviour. Without a test, a future edit could revert to the standard-tier 12h rate and a flex/priority 12h write would be mis-billed by 2x in either direction. Impact: Test only. Fails without the tier derivation (flex bills 2.5 instead of 1.25, priority 2.5 instead of 5.0). * feat(api): emit an opt-in prompt cache TTL gated by catalog capability What: add a `promptCacheTtl` provider-settings knob ("5m" | "1h"), a `ModelInfo.promptCacheTtlSupport` capability array, and a shared `resolveCacheTtl`/`cacheControl` helper that the anthropic, vertex and vercel-ai-gateway caching transforms plus the anthropic, anthropic-vertex, openrouter, vercel-ai-gateway and zoo-gateway providers thread through. Why: ADR-0006 phase 2. C5a (#2260) landed per-TTL write accounting, so a 1h cache write is billed at 2x instead of the 5m rate. This is the matching request-side mechanism: once the settings UI ships in C6, a user selecting 1h causes Anthropic-shaped providers to emit `cache_control: { ttl: "1h" }`. Impact: no behavior change by default. An unset knob resolves to `undefined` and the marker stays the byte-identical `{ type: "ephemeral" }` literal, so no `ttl` key is emitted unless the user opts in AND the resolved ModelInfo advertises the capability. Absent capability means 5m only (I2), capability is read from ModelInfo rather than id substrings, and every marker in a request shares one resolved TTL (I4). Dynamic catalogs (OpenRouter, the gateways) carry no capability data and therefore stay at 5m, as the ADR requires. * feat(settings): offer the prompt cache TTL control on Anthropic profiles What: adds the 1h cache TTL selector to the Anthropic provider panel, gated on the model's catalog capability, plus the four locale keys in all 18 locales. Why: ADR-0006 decision item 3 ships the knob last, after the accounting (C5a) and the mechanism (C5b) so no request can emit a 1h write that is not priced. The control is the user-facing half; it renders only where promptCacheTtlSupport advertises 1h (invariant I2), and selecting the default writes DEFAULT_PROMPT_CACHE_TTL (undefined) rather than a literal "5m" so an unset knob keeps emitting no ttl key (invariant I3). Impact: Anthropic profiles gain an opt-in control; default behavior and wire bytes are unchanged for every profile that leaves it alone. Test: webview-ui Anthropic.spec.tsx 8/8. * test(settings): cover the prompt cache TTL settings round trip What: adds the round-trip coverage the Persisted Setting Checklist asks for: extension-side survival through getStateToPostToWebview (set and unset), import/export schema preservation of both "1h" and "5m", and the SettingsView cachedState buffer (restore, buffer-until-Save, and the absence-vs-explicit-5m distinction). Why: the knob rides apiConfiguration, so a missing schema field or a missing state-post field would silently revert the control without any type error. The import/export case is falsified by removing the schema entry (test goes red), so it guards the field rather than the fixture. Impact: test-only; no production behavior change. Test: src 504/504 across the six affected suites, webview-ui 2235/2235. * fix(api,cost): hash long cache keys, clamp TTL splits, harden the TTL specs What: - openai-native: hash over-long task ids with sha256 hex instead of slicing to 64 chars. A slice drops the distinguishing tail, so two ids sharing a 64-char prefix collapsed into one routing bucket. sha256 hex is exactly 64 chars, so the result still fits OpenAI's cap. - openai-native: clamp the 12h write subset at zero as well as at the write total. A negative `ttl_12h_tokens` from a misbehaving gateway used to reach the Task accumulator and the webview as a negative count. - cost: cap the 1h and 12h write subsets at the remaining aggregate write total. A malformed split (1h + 12h > total) billed the excess; 150k reported 1h writes against a 100k total cost 0.9 instead of 0.6. - Task.spec: replace the fixed 50ms sleep with `vi.waitFor` on the captured telemetry call. The drain is fire-and-forget and touches the filesystem, so a fixed sleep races it on a loaded worker. - vercel-ai-gateway.spec: assert the TTL argument of the `addCacheBreakpoints` mock call. The transform is file-mocked, so `JSON.stringify(mockCreate)` can never contain `cache_control` and the old assertion could not fail. - ttl.spec: compare raw `JSON.stringify` output against literal expected strings in the byte-identical test. `toEqual` on round-tripped objects ignores key insertion order, so a key reorder passed while the wire bytes changed. Why: Seven CodeRabbit "Minor" threads on the C5a/C5b stack. Three are production correctness (cache-key collision, negative token count, overbilled split); three are assertions that could not fail. The seventh thread's premise, forwarding the TTL split into the telemetry recompute, already landed in b647ec7. Impact: Long task ids keep distinct cache buckets; malformed TTL splits can no longer over- or under-bill or leak negative counts. No request shape changes for the default path: the aggregate-only case still derives both subsets as zero. * test(webview): mount the Anthropic settings form in the visual gallery What: adds an "anthropic-settings" gallery story and a matching visual test that asserts the prompt cache TTL control's copy, default value, sibling alignment, contrast and overflow behavior. Why: the C6 control is a user-visible settings surface, and the repo asks for gallery coverage on changes a reviewer would notice at a glance. The story pins the default Anthropic model (claude-sonnet-4-5), whose catalog entry advertises promptCacheTtlSupport ["5m","1h"], so the capability-gated control actually renders. Deliberately no toHaveScreenshot baseline: committed baselines are rendered in the CI Linux container (visual-regression.yml) and run-docker.mjs states host-rendered baselines are not reachable, so a Windows-rendered PNG would fail the required webview-visual check. The assertions are geometric and textual instead, following the LayoutContracts.visual.tsx precedent, and they are falsifiable (a 7px injected offset turns the test red). Impact: test/visual-infra only; no production behavior change. Test: playwright -c playwright-ct.config.ts Anthropic.visual.tsx 1/1, stable across two consecutive runs; eslint and tsc clean. * fix(api,core): key prompt enhancement off the task id, not the handler session `completePrompt` built its `prompt_cache_key` from `this.sessionId`, which is regenerated for every handler instance. Prompt enhancement builds a fresh handler per request (`singleCompletionHandler`), so repeated enhancements of one task each started a new cache bucket and could not reuse the previous prefix. `CompletePromptOptions` now carries an optional `taskId`. `MessageEnhancer` sets it from the open task, `webviewMessageHandler` passes `currentCline?.taskId`, and the OpenAI native handler feeds it through the existing `getPromptCacheKey`, which already prefers `metadata.taskId` and falls back to the session id. Callers with no task keep the old behaviour. Tests: two new provider cases (task-scoped key across handler instances, session fallback without a task id) and two enhancer cases (task id forwarded, options omitted when no task is open). The provider case fails on the pre-fix code with the session uuid in place of the task id. * fix(settings): associate the prompt cache TTL label and pin the control What: type the six Anthropic spec mocks against the real control props, give the prompt cache TTL label an htmlFor bound to the trigger id, and add a label-activation assertion plus a pixel baseline to the visual test. Why: CodeRabbit review of PR #2271 raised three Minor findings. The untyped mocks let a wrong prop shape pass unnoticed, and the label wrapped no control, so clicking it did nothing for pointer users. Impact: label clicks now open the selector; the spec fails on a wrong-typed callback, and the visual test fails when the label association is removed. * test(webview): commit the container-rendered prompt cache TTL baseline What: add the anthropic-prompt-cache-ttl-dark.png baseline rendered by the pinned Playwright container in the webview-visual CI job. Why: webview-ui/AGENTS.md forbids creating committed baselines from host rendering, and Docker is unavailable on the authoring host, so the CI container fallback supplies the authoritative PNG. Impact: the visual test's screenshot assertion now has a baseline that matches CI, closing the last CodeRabbit finding on PR #2271. * test(api): cover the accepted prompt cache TTL on the gateway call site What: add a dedicated `anthropic/claude-3.5-sonnet` catalog row carrying `promptCacheTtlSupport: ["1h"]`, and a test asserting the third argument to `addCacheBreakpoints` resolves to `"1h"` when the knob is set. Why: the existing pair only proved the reject path (`lastCall[2]` undefined) and a bare `toHaveBeenCalled`. Nothing proved the accept path, so a regression that dropped the third argument at the call site stayed invisible. The real gateway catalog never sets `promptCacheTtlSupport`, so the capability has to be injected by the test catalog rather than assumed from a real model. Impact: test-only. The shared haiku row keeps no TTL capability, so the existing reject-path test still asserts fail-closed behavior.
1 parent a4f774b commit debd3c7

63 files changed

Lines changed: 2765 additions & 44 deletions

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
Lines changed: 84 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,84 @@
1+
// npx vitest run src/__tests__/cache-write-ttl-pricing.test.ts
2+
3+
import { describe, it, expect } from "vitest"
4+
5+
import { anthropicModels } from "../providers/anthropic.js"
6+
import { bedrockModels } from "../providers/bedrock.js"
7+
import { openAiNativeModels } from "../providers/openai.js"
8+
import {
9+
CACHE_WRITE_MULTIPLIER_12H,
10+
CACHE_WRITE_MULTIPLIER_1H,
11+
deriveCacheWritesPrice12h,
12+
deriveCacheWritesPrice1h,
13+
type ModelInfo,
14+
} from "../model.js"
15+
16+
/**
17+
* Catalog invariant for the TTL accounting program (ADR-0006 phase 1).
18+
*
19+
* A model that advertises prompt caching on a provider which can report a
20+
* long-lived cache write must carry the matching per-TTL write rate. Without it
21+
* the cost layer can only fall back to the aggregate write rate, which
22+
* under-reports every long-lived write (37.5% for Anthropic 1h, 50% for OpenAI 12h).
23+
*/
24+
describe("cache-write TTL pricing catalog invariant", () => {
25+
const anthropicFamilyEntries = Object.entries(anthropicModels as Record<string, ModelInfo>)
26+
27+
const bedrockAnthropicEntries = Object.entries(bedrockModels as Record<string, ModelInfo>).filter(([id]) =>
28+
id.startsWith("anthropic."),
29+
)
30+
31+
const openAiLongLivedEntries = Object.entries(openAiNativeModels as Record<string, ModelInfo>).filter(
32+
([id]) => id.startsWith("gpt-5.6") || id.startsWith("gpt-6"),
33+
)
34+
35+
it("every Anthropic catalog entry with prompt caching defines cacheWritesPrice1h", () => {
36+
const missing = anthropicFamilyEntries
37+
.filter(([, info]) => info.supportsPromptCache && info.cacheWritesPrice1h === undefined)
38+
.map(([id]) => id)
39+
40+
expect(missing).toEqual([])
41+
})
42+
43+
it("every Bedrock Anthropic-family entry with prompt caching defines cacheWritesPrice1h", () => {
44+
const missing = bedrockAnthropicEntries
45+
.filter(([, info]) => info.supportsPromptCache && info.cacheWritesPrice1h === undefined)
46+
.map(([id]) => id)
47+
48+
expect(missing).toEqual([])
49+
})
50+
51+
it("every OpenAI 5.6/6-class entry defines cacheWritesPrice12h", () => {
52+
expect(openAiLongLivedEntries.length).toBeGreaterThan(0)
53+
54+
const missing = openAiLongLivedEntries
55+
.filter(([, info]) => info.supportsPromptCache && info.cacheWritesPrice12h === undefined)
56+
.map(([id]) => id)
57+
58+
expect(missing).toEqual([])
59+
})
60+
61+
it("Anthropic 1h rates match the documented 2x input-price relationship", () => {
62+
for (const [id, info] of anthropicFamilyEntries) {
63+
if (info.cacheWritesPrice1h === undefined || info.inputPrice === undefined) continue
64+
expect(info.cacheWritesPrice1h, id).toBeCloseTo(info.inputPrice * CACHE_WRITE_MULTIPLIER_1H, 9)
65+
}
66+
})
67+
68+
it("OpenAI 12h rates match the documented 2.5x input-price relationship", () => {
69+
for (const [id, info] of openAiLongLivedEntries) {
70+
if (info.cacheWritesPrice12h === undefined || info.inputPrice === undefined) continue
71+
expect(info.cacheWritesPrice12h, id).toBeCloseTo(info.inputPrice * CACHE_WRITE_MULTIPLIER_12H, 9)
72+
}
73+
})
74+
75+
it("deriveCacheWritesPrice1h scales inputPrice by 2x and stays undefined without one", () => {
76+
expect(deriveCacheWritesPrice1h(3.0)).toBe(6.0)
77+
expect(deriveCacheWritesPrice1h(undefined)).toBeUndefined()
78+
})
79+
80+
it("deriveCacheWritesPrice12h scales inputPrice by 2.5x and stays undefined without one", () => {
81+
expect(deriveCacheWritesPrice12h(4.0)).toBe(10.0)
82+
expect(deriveCacheWritesPrice12h(undefined)).toBeUndefined()
83+
})
84+
})

‎packages/types/src/__tests__/openai-models.test.ts‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -122,6 +122,7 @@ describe("OpenAI native models", () => {
122122
inputPrice: 2,
123123
outputPrice: 10,
124124
cacheWritesPrice: 2.5,
125+
cacheWritesPrice12h: 5,
125126
cacheReadsPrice: 0.2,
126127
longContextPricing: {
127128
thresholdTokens: 272_000,
@@ -166,6 +167,7 @@ describe("OpenAI native models", () => {
166167
inputPrice: 0.1,
167168
outputPrice: 0.5,
168169
cacheWritesPrice: 0.125,
170+
cacheWritesPrice12h: 0.25,
169171
cacheReadsPrice: 0.01,
170172
longContextPricing: {
171173
thresholdTokens: 272_000,
Lines changed: 102 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,102 @@
1+
// npx vitest run src/__tests__/prompt-cache-ttl.test.ts
2+
3+
import { describe, it, expect } from "vitest"
4+
5+
import { anthropicModels, anthropicProviderDefinition } from "../providers/anthropic.js"
6+
import { vertexModels } from "../providers/vertex.js"
7+
import { openRouterDefaultModelInfo } from "../providers/openrouter.js"
8+
import {
9+
vercelAiGatewayDefaultModelId,
10+
VERCEL_AI_GATEWAY_PROMPT_CACHING_MODELS,
11+
} from "../providers/vercel-ai-gateway.js"
12+
import {
13+
DEFAULT_PROMPT_CACHE_TTL,
14+
modelInfoSchema,
15+
promptCacheTtlSchema,
16+
promptCacheTtlValues,
17+
type ModelInfo,
18+
} from "../model.js"
19+
import { baseProviderSettingsShape } from "../providers/common.js"
20+
import { providerIdentifiers } from "../provider-identifiers.js"
21+
import { globalSettingsSchema } from "../global-settings.js"
22+
23+
/**
24+
* Catalog invariant for the request-side TTL knob (ADR-0006 phase 2, PR-C5b).
25+
*
26+
* Emission is gated by `ModelInfo.promptCacheTtlSupport`, so a model that can
27+
* report a 1h cache write but carries no capability array would silently never
28+
* emit the TTL the user asked for. These tests pin both directions: capable
29+
* catalogs advertise it, and catalogs that cannot know stay absent (fail closed).
30+
*/
31+
describe("prompt cache TTL schema", () => {
32+
it("accepts exactly the 5m and 1h classes", () => {
33+
expect(promptCacheTtlValues).toEqual(["5m", "1h"])
34+
expect(promptCacheTtlSchema.parse("5m")).toBe("5m")
35+
expect(promptCacheTtlSchema.parse("1h")).toBe("1h")
36+
expect(promptCacheTtlSchema.safeParse("30m").success).toBe(false)
37+
expect(promptCacheTtlSchema.safeParse("12h").success).toBe(false)
38+
})
39+
40+
it("defaults to undefined so an unset knob emits no ttl key", () => {
41+
expect(DEFAULT_PROMPT_CACHE_TTL).toBeUndefined()
42+
})
43+
44+
it("is optional on ModelInfo", () => {
45+
const parsed = modelInfoSchema.parse({ contextWindow: 200_000, supportsPromptCache: true })
46+
expect(parsed.promptCacheTtlSupport).toBeUndefined()
47+
48+
const withCapability = modelInfoSchema.parse({
49+
contextWindow: 200_000,
50+
supportsPromptCache: true,
51+
promptCacheTtlSupport: ["5m", "1h"],
52+
})
53+
expect(withCapability.promptCacheTtlSupport).toEqual(["5m", "1h"])
54+
})
55+
})
56+
57+
describe("prompt cache TTL knob location", () => {
58+
it("lives in provider settings so handlers can see it", () => {
59+
// ApiHandlerOptions = Omit<ProviderSettings, "apiProvider">, so only a
60+
// provider-settings key reaches a handler (ADR-0006 decision item 2).
61+
expect(baseProviderSettingsShape.promptCacheTtl).toBeDefined()
62+
63+
const parsed = anthropicProviderDefinition.schema.safeParse({
64+
apiProvider: providerIdentifiers.anthropic,
65+
promptCacheTtl: "1h",
66+
})
67+
expect(parsed.success).toBe(true)
68+
expect(parsed.success && parsed.data.promptCacheTtl).toBe("1h")
69+
})
70+
71+
it("is not a global setting", () => {
72+
const shape = globalSettingsSchema.shape as Record<string, unknown>
73+
expect(shape.promptCacheTtl).toBeUndefined()
74+
})
75+
})
76+
77+
describe("prompt cache TTL capability catalog", () => {
78+
it("every Anthropic entry with prompt caching advertises both TTL classes", () => {
79+
const missing = Object.entries(anthropicModels as Record<string, ModelInfo>)
80+
.filter(([, info]) => info.supportsPromptCache && !info.promptCacheTtlSupport?.includes("1h"))
81+
.map(([id]) => id)
82+
83+
expect(missing).toEqual([])
84+
})
85+
86+
it("leaves the Vertex catalog without capability so it fails closed", () => {
87+
// PR-C5b scopes the catalog capability to the Anthropic entries; the Vertex
88+
// catalog is populated separately, so its models stay 5m-only for now.
89+
const withCapability = Object.entries(vertexModels as Record<string, ModelInfo>)
90+
.filter(([, info]) => info.promptCacheTtlSupport !== undefined)
91+
.map(([id]) => id)
92+
93+
expect(withCapability).toEqual([])
94+
})
95+
96+
it("leaves dynamic-catalog defaults without capability so they fail closed", () => {
97+
// OpenRouter and the gateway catalogs are fetched at runtime and carry no TTL
98+
// capability data, which ADR-0006 maps to "5m only" (invariant I2).
99+
expect(openRouterDefaultModelInfo.promptCacheTtlSupport).toBeUndefined()
100+
expect(VERCEL_AI_GATEWAY_PROMPT_CACHING_MODELS.has(vercelAiGatewayDefaultModelId)).toBe(true)
101+
})
102+
})

‎packages/types/src/model.ts‎

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -95,6 +95,20 @@ export type ModelParameter = z.infer<typeof modelParametersSchema>
9595
export const isModelParameter = (value: string): value is ModelParameter =>
9696
modelParameters.includes(value as ModelParameter)
9797

98+
/**
99+
* Prompt cache TTL classes a request-side cache marker can carry. `"5m"` is the
100+
* vendor default and is never emitted explicitly, so only `"1h"` changes the wire
101+
* shape and an unset knob stays byte-identical to a pre-knob build (ADR-0006 I3).
102+
*/
103+
export const promptCacheTtlValues = ["5m", "1h"] as const
104+
105+
export const promptCacheTtlSchema = z.enum(promptCacheTtlValues)
106+
107+
export type PromptCacheTtl = z.infer<typeof promptCacheTtlSchema>
108+
109+
/** Deliberately `undefined`, not `"5m"`: emitting `ttl: "5m"` would change every request's bytes. */
110+
export const DEFAULT_PROMPT_CACHE_TTL: PromptCacheTtl | undefined = undefined
111+
98112
/**
99113
* ModelInfo
100114
*/
@@ -112,6 +126,8 @@ export const modelInfoSchema = z.object({
112126
// When set to "24h", extended prompt caching will be requested; when omitted
113127
// or set to "in_memory", the default in‑memory cache is used.
114128
promptCacheRetention: z.enum(["in_memory", "24h"]).optional(),
129+
// Cache TTLs this model accepts. Absent means "5m only" (ADR-0006 I2, fail closed).
130+
promptCacheTtlSupport: z.array(promptCacheTtlSchema).optional(),
115131
// Capability flag to indicate whether the model supports an output verbosity parameter
116132
supportsVerbosity: z.boolean().optional(),
117133
// Capability flag to indicate whether the model exposes a user-configurable max output
@@ -136,6 +152,13 @@ export const modelInfoSchema = z.object({
136152
inputPrice: z.number().optional(),
137153
outputPrice: z.number().optional(),
138154
cacheWritesPrice: z.number().optional(),
155+
// Per-TTL cache-write rates. Vendors bill a longer cache lifetime at a higher
156+
// write rate than the default 5m/30m class, so a single `cacheWritesPrice`
157+
// under-reports whenever the response reports a long-lived write. Omitted
158+
// means "no long-lived writes are expected on this model"; the cost layer
159+
// derives a fail-closed rate from `inputPrice` if one still arrives.
160+
cacheWritesPrice1h: z.number().optional(),
161+
cacheWritesPrice12h: z.number().optional(),
139162
cacheReadsPrice: z.number().optional(),
140163
longContextPricing: z
141164
.object({
@@ -180,6 +203,11 @@ export const modelInfoSchema = z.object({
180203
inputPrice: z.number().optional(),
181204
outputPrice: z.number().optional(),
182205
cacheWritesPrice: z.number().optional(),
206+
// Per-TTL write rates for this tier. A tier that overrides inputPrice
207+
// must also override these, otherwise a long-context or flex/priority
208+
// write bills at the base tier's TTL rate.
209+
cacheWritesPrice1h: z.number().optional(),
210+
cacheWritesPrice12h: z.number().optional(),
183211
cacheReadsPrice: z.number().optional(),
184212
}),
185213
)
@@ -188,6 +216,34 @@ export const modelInfoSchema = z.object({
188216

189217
export type ModelInfo = z.infer<typeof modelInfoSchema>
190218

219+
/**
220+
* Multipliers vendors apply to the base input price for a cache write, by TTL class.
221+
* Anthropic: 5m = 1.25x, 1h = 2x. OpenAI: 30m = 1.25x, 12h = 2.5x.
222+
*/
223+
export const CACHE_WRITE_MULTIPLIER_5M = 1.25
224+
export const CACHE_WRITE_MULTIPLIER_1H = 2.0
225+
export const CACHE_WRITE_MULTIPLIER_12H = 2.5
226+
227+
/**
228+
* Derive the 1h cache-write rate from the base input price.
229+
*
230+
* Stated once here so every Anthropic-family catalog entry and every runtime
231+
* fallback agrees on the 2x relationship. Returns `undefined` when the model
232+
* has no input price to scale, which keeps the cost layer fail-closed: it will
233+
* fall back to `cacheWritesPrice` rather than invent a rate.
234+
*/
235+
export function deriveCacheWritesPrice1h(inputPrice: number | undefined): number | undefined {
236+
return inputPrice === undefined ? undefined : inputPrice * CACHE_WRITE_MULTIPLIER_1H
237+
}
238+
239+
/**
240+
* Derive the 12h cache-write rate from the base input price (OpenAI 12h class).
241+
* Same fail-closed contract as {@link deriveCacheWritesPrice1h}.
242+
*/
243+
export function deriveCacheWritesPrice12h(inputPrice: number | undefined): number | undefined {
244+
return inputPrice === undefined ? undefined : inputPrice * CACHE_WRITE_MULTIPLIER_12H
245+
}
246+
191247
export type ModelRecord = Record<string, ModelInfo>
192248

193249
export type RouterModels = Record<DynamicProvider | LocalProvider, ModelRecord>

0 commit comments

Comments
 (0)