Repository navigation
Conversation
An opted-in user's prompt or output carried any key pasted into it verbatim, so the payload log held live credentials. Every format OpenRouter's Secrets guardrail detects, under its labels, less its two Bitcoin WIF formats (no prefix, only a length and an alphabet ordinary ids share), plus Grove's own gr_ keys, becomes [SECRET:<format>]. A known prefix and at least 16 key characters is enough, whatever the length past that: OpenRouter's rule for most formats, and a superset of its match where it pins a length or a marker, so a key of an odd size still goes. The floor keeps names in code (hf_token, ghp_example) intact; Ethereum keys and JWTs keep their strict shape, since their prefix says too little. It runs on every JSON string that is not inline media, in one walk with the media placeholders: a kept base64 image can hold runs shaped like a key, and redacting them would corrupt it. A stream is scanned per event, so a key split across events is not caught. Each format's regexp is guarded by a literal it cannot match without, which keeps a megabyte of prose to ~9ms rather than ~550ms. A body with nothing to redact is still logged byte-for-byte. The sample keys in tests and the README drop the old gr_sk_ spelling. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
A connection that died without a FIN or an RST keeps taking new requests for as long as traffic keeps it busy: the 90s idle close only runs on a connection with nothing on it. Each request then hung until upstream_read_timeout, and the connection went only when the kernel gave up retransmitting, about 15 minutes at Linux defaults. An HTTP/2 connection that has delivered nothing for 15s is now pinged, and closed if no answer comes within 90s more (PingAfter and PingTimeout in proxy.Options). One answering ping is enough to keep it; a blip shorter than 90s costs nothing. It is set through Transport.HTTP2, which is why go.mod asks for Go 1.24. The README gains an Upstream connections section: what is pooled and for how long, when a dial and a handshake are paid, and the ping. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Where the memory limits live (the unit's MemoryMax, GOMEMLIMIT at 90% of it), what a SIGHUP child inherits, and what happens when pathway outgrows the cgroup. lame_duck is sized to at least one health-check interval of whatever sits in front, or a client gets connection refused instead of a retryable 503. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
The user record gains `limits`, a comma list of metric:window:value. `quota` admits through one Lua script after the credit gates: it reads every counter and counts the request only when all have room, so a refusal counts nothing and two requests cannot both take the last slot. Over a limit is a 429 with Retry-After, naming the limit that resets last. `meter` debits the answer's tokens after the accrual, in its own call: replayed later it would land in the wrong window, so it is never spooled. Counters are lim:<user>:<metric>:<window>:<bucket>, kept two windows. Windows (1m 1h 1d 1M) reset on the UTC clock. A push carrying a limit this binary cannot read is refused 400, naming the user. Also a live test that nothing the control plane pushes carries a TTL: the push is hash-gated, so an expired record would stay missing. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
phot0n
changed the title
phot0n
changed the title
A route row carried `modality`, one word out of four, which mixed what goes into a model with what comes out of it. It now carries two lists, `input_modalities` and `output_modalities`, in OpenRouter's words. The path gate reads the outputs: chat and messages need `text`, /v1/embeddings needs `embeddings`, the audio paths need `transcription`. A row that declares nothing is unrestricted, as a blank modality was. The inputs are carried and not read yet. `modality` is gone from the wire, and a push that still carries it is refused like any unknown field. Grove's matching change sends the two lists in its place, so the two ship together. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`maxtokens` named the output cap for a vendor by branching on the name "openai". That rule and a new one now come out of one table, `vendors`, keyed by the provider's name: the field the upstream reads the output cap from, and whether it is asked for usage on every chunk of a stream. - `vendorfields` replaces `maxtokens`: `max_completion_tokens` for OpenAI, `max_tokens` for DeepSeek, Baseten and any vendor the table does not name. Engines we run get the body as sent. - `streamusage` also forces `stream_options.continuous_usage_stats` for an engine of ours and for Baseten, so a stream that is cut is billed by the last chunk that arrived. OpenAI answers that field with a 400 and DeepSeek ignores it, so neither is sent it. The README lists the fields the gateway knows on each of the two client shapes, and what each upstream was measured to do. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r wrote The swap back to the client's id matched only the exact id sent upstream. DeepSeek answers `deepseek-v4-flash` as `deepseek-flash`, so the client read the vendor's own name. The value of `"model"` is now replaced whatever it holds, with any spacing after the colon. A value that is not a string, never closes, or runs past 256 bytes passes through as it came. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The usage tee keeps the first and the last line that names usage. A stream cut in the middle of a line left half a frame as the last one, and it took the place of the last frame that could be read. A trailing line without its newline is now kept only when usage can be read out of it. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
`reason` was only ever an ingress's X-Grove-Reason, so a dial that was refused or timed out left it blank on the access line. It now holds what the client was told: `upstream unavailable` or `upstream timed out`. Such a hop still counts against the upstream. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A JSON body may carry `fallbacks`, up to 3 model names. When the model asked for cannot serve, the request moves to the next one named, in order. The gateway never picks a fallback of its own. - A new stage, `fallback`, sits below `meter` and above `retry`: a model's whole key ring is walked before the next model is tried, and the request is billed once, on the model that answered. - A fallback is tried when the model has nowhere to go (no healthy server, or every one full), when its upstream answers 5xx, or with a key failure `retry` could not rotate away. A fallback's own refusal, any status from 400, moves on to the next. A 4xx from the model asked for is relayed, and a response that has begun is never moved. - A fallback must be granted to the caller and have a route on this surface, or it is passed over without a dial. The list is not judged when the request arrives. - The client is told by `X-Grove-Fallback` and by the body's `model`. Usage, pricing, the access line's `fallback` and what an ingress is asked for follow the model that served. - Every attempt the client never saw, a rotated key or a model moved off, leaves an `attempt` line on the access log, joined by rid. - A held answer whose body broke off no longer aborts the request. What the proxy learned of an attempt is now one `Outcome` on the request state, so undoing an attempt is one assignment. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A body that was not JSON, or had no `model`, went on to the grant check, which refused "model ''" with a 403. The body stage now answers it: 400 `request body is not a JSON object`, or 400 `request names no model`. An upload form with no model part and an upgrade with none in its query get the same 400. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
phot0n
changed the title
…the client Every header an upstream sent was passed on, bar its request ids. A Baseten answer showed the client twelve `x-baseten-*` headers, the rate limits of the account we call it with, `via` and `alt-svc`. OpenAI's would add its organisation and its edge's cookie. The proxy now relays an allowlist: Content-Type, Content-Length, Content-Encoding, Content-Disposition, Cache-Control, Retry-After, and what an upgrade's handshake needs. An ingress's X-Grove-Engine and X-Grove-Reason are still read by the gateway and no longer passed on. The gateway's own headers are untouched. The README gains a Headers section: what a client sees, what stops at the gateway, and every X-Grove-* header. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
A fallback that could not take the request (a text-only model, a request with an image) was dialled, refused it, and the next was tried: one wasted dial each. `input_modalities` was pushed on every route row and read by nothing. Each fallback is held, at its turn, to what the request carries: the image and file parts of `messages`, by their declared `type`, on either shape, those inside a tool result or an Anthropic document included. One that declares its inputs and lacks one is skipped without a dial, and says so in the process log. A row that declares nothing is unrestricted, and the model asked for is never judged. Audio and video parts are not looked for yet. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
closes #1