fix: stop rejecting valid Anthropic keys — verify against /v1/messages, not /v1/models#18
Merged
Conversation
…s, not /v1/models
Bug report: "now it is not even taking the api" — after the previous fix
(verify-before-save), a real API key could no longer be saved at all.
## Root cause (evidence, not assumption)
The prior verification called GET /v1/models to confirm a key works
before persisting it. Anthropic supports scoped/restricted API keys
(created with limited permissions in the Console) that can work perfectly
for chat completions while lacking permission to list models. A fully
valid, working key could therefore get a 403 on /v1/models specifically —
which the verification code treated as "confirmed invalid" and rejected —
silently blocking the user from ever saving a real, working key. This is
strictly worse than the bug it was meant to fix: instead of a confusing
error deep in a chat turn, the user now couldn't save a key at all.
## Fix
Verify against POST /v1/messages instead — the exact endpoint the real
chat feature actually uses, eliminating the scope-mismatch class of false
positives by construction (whatever this call proves works IS the
capability the app needs). Sends a deliberately empty/invalid JSON body
(`{}`, missing the required model/messages/max_tokens fields).
Confirmed LIVE against Anthropic's real API (not assumed) that
authentication is validated before the request body: a bad key gets a 401
regardless of body content, so this never reaches the model and never
consumes a token, while exercising the identical auth path a real request
takes.
Also hardened the blocking rule to be maximally conservative: only a
confirmed 401 blocks a save now (was 401 OR 403). 403, 429, 5xx, and
network failures are all treated as "not evidence the key is bad" and the
key still saves — a scoped key returning 403 for an unrelated permission
is no longer misread as "this key is broken."
## Verification
- Rewrote anthropicKeyCheck.test.ts: confirms the call targets
/v1/messages (not /v1/models) with the same auth headers the real
chat feature sends; a 403 (the exact false-positive class that broke
saving) no longer blocks the save and the route test proves the key
actually lands in the DB; 401 still correctly blocks and never persists.
- Live, unmocked smoke test against a running dev server: confirmed a
genuinely bad key is still rejected via a real call to Anthropic's
actual /v1/messages endpoint and never saved.
- Full suite 232/232, tsc --noEmit clean, lint baseline unchanged, next
build clean.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Bug report
"now it is not even taking the api" — after the previous verify-before-save fix (PR #16), a real API key could no longer be saved at all.
Root cause (evidence, not assumption)
The prior verification called
GET /v1/modelsto confirm a key works before persisting it. Anthropic supports scoped/restricted API keys (created with limited permissions in the Console) that can work perfectly for chat completions while lacking permission to list models. A fully valid, working key could get a 403 on/v1/modelsspecifically — which the old code treated as "confirmed invalid" and rejected. This is strictly worse than the bug it was meant to fix: instead of a confusing error deep in a chat turn, the user could no longer save a key at all.Fix
Verify against
POST /v1/messagesinstead — the exact endpoint the real chat feature actually uses, eliminating the scope-mismatch false-positive class by construction. Sends a deliberately empty/invalid JSON body ({}).Confirmed live against Anthropic's real API (not assumed): authentication is validated before the request body — a bad key gets a 401 regardless of body content, so this never reaches the model and never consumes a token, while exercising the identical auth path a real chat request takes.
Also hardened the blocking rule to be maximally conservative: only a confirmed 401 blocks a save now (was 401 OR 403). 403/429/5xx/network failures are all treated as "not evidence the key is bad" and the key still saves.
Verification
anthropicKeyCheck.test.ts: confirms the call targets/v1/messageswith the real chat feature's exact auth headers; a 403 (the false-positive class that broke saving) no longer blocks the save and the route test proves the key actually lands in the DB; 401 still correctly blocks and never persists./v1/messagesendpoint and never saved.tsc --noEmitclean, lint baseline unchanged,next buildclean.