Repository navigation
Conversation
A key lives in one geography, so its store can gate it exactly only if the budget is the key's own. user: records and the users state section are gone; key:<hash> holds team, groups, allow/deny, geography, limits, prepaid, budget (the cap) and this store's spent. Rate limits count under the key prefix, accrue moves spent on the key and reports key_spent/key_balance, spend-adjust names the key, and /v1/credits answers the caller's own cap. A push carrying a users section is 400; a key push strips the fields older control planes wrote. Deploy before the matching grove: old pushes get 400 until it lands. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A vendor-run tool (Anthropic web search and code execution, OpenAI's hosted search, file search, code interpreter, image generation) is billed per call or per container hour, and nothing here meters it. Each provider row now carries `denied_tools`, pushed from Grove's Denied Tool rows: `route` reads every `tools[].type` and top-level field off the client's own bytes and drops a row that denies one. With no row left the request is a 400 `<model> does not run <name>` before any dial and above meter, so it bills nothing. A row denying nothing runs everything, so an engine of ours beside the vendor row still serves. Anthropic fast mode (`speed: "fast"`) is twice the rate and unpriced: `vendorfields` drops it for every vendor and answers `X-Grove-Changed: speed=default`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A keyless hit on /anthropic or a non-Anthropic path under it was a 404 "no such path", where /v1/ answers 401. The alias now runs auth before refusing, so a missing key is told so in the Anthropic envelope, and the 404 names /anthropic/v1/messages as the path to use. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Passive health counted a vendor hop against its base URL, so every model on one front shared one counter: three 502/503/504s, connection errors or cut streams in a row, on any of them, took every model of that vendor to our own 503 for 60s. A vendor row has no sibling to steer to, so ejection only hid the vendor's own answer from the caller and their fallbacks. Decision.HealthTarget is blank for a vendor row, so meter and the fallback loser report count nothing, and markUnhealthy skips vendor rows whatever an older binary left in the store. Engine and ingress rows are ejected as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
`X-Grove-Metadata: app=hrms, trace=7f3a-91` lands on the access line as a `meta` object: which app spent a key several apps share, or the caller's own correlation id. Up to 16 pairs in 2 KB, keys 1-32 of [a-z0-9_.-], values 1-128 printable ASCII without a comma; anything else is a 400 that still leaves its line. Logged only, never counted. upstreamauth now takes X-Grove-Metadata and X-Grove-Session off every hop. The session header was relayed to engines and vendors as the caller sent it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The key record holds spent:<YYYY-MM> instead of a lifetime spent: accrue,
resolve and spend-adjust read and move the current month's field, a new
month reads 0 with nothing to reset, and a push strips the old lifetime
field. The memory store mirrors it (rollMonth).
GET /v1/credits answers {"balance", "spent"} only; a free team's key reads
zeros.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A 401/402/403 the key ring cannot move off (every key refused, a single key, an engine's own) is never relayed: the client gets 502 api_error with one of three messages, the access line says reason="credential <class>" with the vendor's status in upstream_status, and the error log keeps the vendor's words with the key id. A provider row's 400 or 429 is read to its end (64 KiB) before it is classed, because OpenAI's 429 insufficient_quota and Anthropic's 400 "credit balance" mean 402 (domain.ClassifyUpstream). pk:<id> stats split rejected into unauthorized, unpaid and forbidden (rejected stays their sum), and GET /grove-admin/provider-keys takes them: read and deleted in one transaction, so each pull starts from zero. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
… UTC day A push carries a `teams` section: `team:<Central Team>` holds what is the team's — limited, prepaid, log_payloads, the limits a key with none of its own runs under — and `KeyRecord.Under` applies it onto the key, resolved with the key and its groups in the one Lua call. A key whose team the store does not hold reads as pushed. Teams and keys are bucketed 256 ways by `BucketOf`, so one suspended team ships one bucket. Live usage lands under `usage:<prefix>:<YYYY-MM-DD>`, the UTC day this box served it; drains and acks carry the day, and `peek=1` on `GET /grove-admin/usage` answers the live counters under "" without moving them. Only a request the upstream took (`IsServed`) is counted. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r /anthropic Root `/v1/*` serves no dialect any more: a data path there is a 404 naming the base it belongs under. `/openai` is open to everything an engine answers on that shape and refuses only `/v1/messages`; `/anthropic` stays closed to its own. One `surface` handler strips the prefix, marks the dialect and refuses, for both bases. `GET /v1/credits` stays at root as the gateway's own answer. The SDK bases differ: OPENAI_BASE_URL=<gateway>/openai/v1 (that SDK's base carries the /v1 and appends /chat/completions), ANTHROPIC_BASE_URL= <gateway>/anthropic (appends /v1/messages). Proved with both SDKs against a local build. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A drain made one round trip per bucket: set-aside, then HGETALL for every unacked pair. At a lakh of keys that is a lakh of trips every hour. Each now goes through `pipelined`, 1000 commands a round trip, so neither the client nor Redis's output buffer holds the whole lot. SCAN takes 1000 a call. The set-aside script goes as EVAL with its body: inside a pipeline Script.Run queues EVALSHA and cannot see NOSCRIPT to fall back, and a Load first still races a SCRIPT FLUSH. A chunk that fails leaves the earlier ones set aside and unacked; the next drain answers them under their own id, as a failed single call did. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Retention only guards a control plane restored from backup. At hourly pulls every active key writes a drained hash an hour, so a week is ~17M hashes at a lakh of keys; three days covers a daily backup plus two days to notice and restore, at under half the memory. Unacked drains still never expire. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
One Pick served both kinds of row behind ~20 IsProvider checks, and the mechanics differ: an engine has slots, ejection and session pins; a vendor has a key ring, its own auth headers and key-walking retry. A model is now one kind — a push that mixes them is a 400, a stale stored mix routes to the engines once per warning — and Pick hands its whole table to engine.go or vendor.go. upstreamauth and retry split the same way. What a vendor reads differently moves out of the transform flag table into service/vendor: one Vendor per third party with its Headers and Quirks, Default for any the gateway has not met. Baseten's Bearer on its Anthropic front becomes its own Headers instead of a flag. From review: the caller's Authorization never travels onward, even to an engine with no key of its own. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Which credential a retry dials next is routing policy, not HTTP. KeyRing moves from the middleware into routing, and NextKey goes private now that nothing outside routing walks the ring. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Every service and transport test runs against the memory store, so a memory that drifted from redis would keep them green over a broken gateway. storetest runs the same cases against memory on every go test and against a real redis under the live tag. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
servicetier removed a caller's service_tier silently, while speed, dropped for the same pricing reason, is announced. The caller asked for a tier and is served at the default one, so X-Grove-Changed now carries service_tier=default. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The root README held every rule for every package in one 1.6k-line file. Each section now lives beside the code it describes; the root keeps the overview, the design, and an index pointing at each package README. Text is moved verbatim, only headings and cross-links change. vendor_quirks.md moves next to vendor.go for the same reason. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…harged `prepaid` is gone from the key and team records, the push and the wire: Exhausted is limited || spent >= budget, so a key with no cap pushed is refused at 402 until the control plane sets one. Metering tags every priced request p: and moves the key's spend; the f: tag and the free-holder path are deleted. /v1/credits answers every key its budget less its spend. A push that still carries `prepaid` is refused as an unknown field, so this ships before the Grove change that stops sending it. A `prepaid` field left in an existing hash is unread.
phot0n
marked this pull request as ready for review
October 18, 2026 15:27
 [High impact] The PR changes shared request checks, billing, state sync, and API routing across the gateway. Fix replay dates before merging so queued requests do not consume another month's cap. Reviews (1) · Last reviewed commit: "feat(metering)!: every key is gated on i..." · Reviewed by Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
Comment on lines
+88
to
+93
| now := time.Now() | ||
| keys := []string{"accrued:" + a.ID, "usage:" + repository.UsageBucket(a.Prefix, now)} | ||
| if a.Key != "" { | ||
| keys = append(keys, "key:"+a.Key) | ||
| } | ||
| landed, err := replayScript.Run(ctx, u.rdb, keys, accrueArgs(a)...).Int() | ||
| landed, err := replayScript.Run(ctx, u.rdb, keys, accrueArgs(a, now)...).Int() |
There was a problem hiding this comment.
If Redis recovers after a month boundary, Replay assigns queued requests to the new day and month. Last month's requests then consume the customer's new monthly cap. Carry the saved request time through Accrual and use it for both UsageBucket and spentField.
Prompt To Fix With AI
This is a comment left during a code review.
Path: internal/repository/redis/usage.go
Line: 88-93
Comment:
**Queued usage charges another month**
If Redis recovers after a month boundary, `Replay` assigns queued requests to the new day and month. Last month's requests then consume the customer's new monthly cap. Carry the saved request time through `Accrual` and use it for both `UsageBucket` and `spentField`.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.| -d '{"users":[{"name":"you","group":"acme,beta"}]}' | ||
| curl -XPUT localhost:8080/grove-admin/keys -H 'X-Grove-Admin-Token: tok' \ | ||
| -d "{\"keys\":[{\"key_hash\":\"$(printf gr_demo | sha256sum | cut -d' ' -f1)\",\"prefix\":\"dev\",\"user\":\"you\",\"status\":\"active\"}]}" | ||
| -d "{\"keys\":[{\"key_hash\":\"$(printf gr_demo | sha256sum | cut -d' ' -f1)\",\"prefix\":\"dev\",\"team\":\"you\",\"group\":\"acme\",\"status\":\"active\"}]}" |
There was a problem hiding this comment.
The sample key has no budget, so its request returns 402 before reaching the engine, not the documented 502. Add a positive budget and update the usage lookup to the dated hash; usage:dev no longer contains new counters.
Prompt To Fix With AI
This is a comment left during a code review.
Path: README.md
Line: 372
Comment:
**Quickstart stops before the engine**
The sample key has no `budget`, so its request returns 402 before reaching the engine, not the documented 502. Add a positive `budget` and update the usage lookup to the dated hash; `usage:dev` no longer contains new counters.
---
For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A key lives in one geography, so its store can gate it exactly only if the budget is the key's own. user: records and the users state section are gone; key: holds team, groups, allow/deny, geography, limits, prepaid, budget (the cap) and this store's spent. Rate limits count under the key prefix, accrue moves spent on the key and reports key_spent/key_balance, spend-adjust names the key, and /v1/credits answers the caller's own cap. A push carrying a users section is 400; a key push strips the fields older control planes wrote.
Deploy before the matching grove: old pushes get 400 until it lands.