feat(seer-billing): Check Seer credits before starting autofix steps by srest2021 · Pull Request #130283 · getsentry/sentry

srest2021 · 2026-11-06T18:45:02Z

Relates to CW-3061

Adds check_autofix_billing, called from all autofix entrypoints: trigger_autofix_agent, trigger_autofix_feature, trigger_push_changes and continue_single_run.

Token-billed orgs are checked for credits before every step, before Open PR and before a reply to a single run, and get SeerOutOfCreditsException if they can't pay. All credit checks are also separately feature-flagged.
Other orgs keep the same quota check.

Autofix single run replies get their own billable operation with a placeholder estimate.

WIP followup to handle the out of credits exceptions: #130284

Return a 402 before sending a chat message when a token-billed org does not have enough Seer credits. Dashboard generation runs are not gated. can_run_seer_operation now passes orgs that are not token-billed. Refs CW-3061

Token-billed orgs are checked for Seer credits before each Autofix step and before opening a PR, and get SeerOutOfCreditsException when they can't pay. Other orgs keep the seat-based quota check. Agentic triage runs are not gated. Refs CW-3061

linear-code · 2026-11-06T18:45:10Z

CW-3061

…tofix-gating

Autofix billing checks the token-billed flag itself, so it only needs the credit check.

…tofix-gating # Conflicts: # src/sentry/seer/credits.py

sentry · 2026-11-06T18:58:32Z

Sentry Snapshot Testing

Name	Added	Removed	Changed	Renamed	Unchanged	Skipped	Status
sentry-frontend `sentry-frontend`	0	0	0	0	798	0	✅ Unchanged

⚙️ sentry-frontend Snapshot Settings

…gent-chat # Conflicts: # src/sentry/seer/endpoints/organization_seer_agent_chat.py

…tofix-gating # Conflicts: # src/sentry/seer/autofix/feature/dispatch.py # tests/sentry/seer/autofix/feature/test_dispatch.py

Chat replies to single runs continue the run through continue_single_run, which skipped billing. The run's own referrer decides, so agentic triage runs stay free.

…gent-chat # Conflicts: # src/sentry/seer/endpoints/organization_seer_agent_chat.py

…tofix-gating

…tofix-gating Check the credit check flag before the token-billed flag. has_enough_seer_credits no longer checks the flag; its callers do.

Cover orgs that aren't token-billed, group the chat credit tests after the access tests, and match the neighboring tests' names and setup.

…tofix-gating

…utofix-gating # Conflicts: # src/sentry/seer/credits.py

Single runs are behind organizations:autofix-single-run and need their own estimate, so they're out of scope here.

Agentic triage runs skip the Seer credit check, and the Autofix endpoint accepted any referrer from the request body. Treat a client-supplied agentic triage referrer as unknown.

Restores the single run reply check, using the placeholder estimate for now.

…utofix-gating

billyvg

🤖 AI-assisted review, posted on behalf of @billyvg.

Co-Authored-By: Sofia Rest <sofia.rest@sentry.io>

sehr-m

okay wait i think i get the operations for estimate gating, but a few qs for changes :))

trigger_autofix_agent checked billing, then called trigger_autofix_feature, which checked it again with the same arguments. Keep only the dispatch check, and test a token-billed org with credit checks off.

…tion (#130463) relates to CW-2332 depends on #129563, #130283, #130284 The chat's credit check only runs when the panel opens or the window regains focus. If credits run out partway through a conversation, the message box stays enabled and the next send fails with a generic error. This rechecks credits when: - Seer stops a reply for credits (`failure_reason: out_of_credits`) - a send comes back 402 so the out-of-credits notice and disabled message box show right away. A 402 no longer shows "There was an error sending your message"; the recheck brings up the notice instead. The recheck invalidates the credit check query, like `useExplorerAutofix` does. That needed `useSeerCreditCheck` to turn off with `enabled` instead of `skipToken`, so the key is the same either way. Example (it would show any intermediate thinking as well): <img width="402" height="615" alt="× Seer Agent I" src="https://github.com/user-attachments/assets/c7e056e8-73a7-4a5a-be32-62bcb80e1609" /> --------- Co-authored-by: Billy Vong <billy@sentry.io> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>

Credit checks no longer skip runs by referrer. Agentic triage runs are checked like any other until a follow-up marks them by feature instead. Credit checks are behind seer-credit-checks, so nothing changes in production. Refs CW-3061

srest2021 · 2026-11-09T23:27:32Z

I'm going to drop agentic triage gating from this PR, wiring this feature through and gating only manual iterations on AT is a bit more complicated than I thought and deserves its own PR

sehr-m

okay wait i think i get the operations for estimate gating, but a few qs for changes :))

An operation only sets the credit estimate, since it can run under more than one feature, so callers pass the feature explicitly. The credit check endpoint now takes a feature param.

cursor

Cursor Bugbot has reviewed your changes and found 1 potential issue.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 5e7ccfc. Configure here.}

The UI can't tell runs agentic triage started apart yet, so the credit check endpoint picks the feature itself instead of requiring the UI to send it.

…rs (#130284) Fixes CW-3061 Handles the `SeerOutOfCreditsException` exception from #130283 at every caller. Quota is only checked when a new run starts, so "not checked" means a continuation. **User triggered** | Caller | Out of credits | Out of quota | |---|---|---| | Issue page / API: start or continue a step | 402 | Start: 402. Continue: not checked | | Issue page / API: open PR | 402 | Not checked | | Issue page: PR feedback | 402 before queueing, so the feedback isn't dropped later (see "PR iteration: queued feedback" below) | Not checked | | Seer chat: reply to a single run | 402, same as a chat message | Not checked | | Slack buttons | Error reply, recorded as a halt | Start: error reply, recorded as a failure. Continue: not checked | | Admin: single run | 400 | 400 | | Admin: retry a failed step | Skipped with a reason | Not checked | **Automatically triggered** | Caller | Out of credits | Out of quota | |---|---|---| | New issue automation | N/A, ignored | N/A, ignored | | Completion hook: automatic next step, automatic open PR | Log and stop | Not checked | | PR iteration: queued feedback | Skip and discard the iteration | Not checked | | PR iteration: push, branch-moved retry | Log, not pushed. The iteration ends as `push_failed`, without pausing iteration | Not checked | | Agentic triage | Not checked, stays free | Free cohort skips it | **Note:** The PR iteration behavior doesn't have good UX yet. When a push or retry is refused for credits, the user sees nothing and the PR isn't updated. When Seer itself stops an iteration for credits, iteration on the PR is paused for good, even after a top-up. See [CW-3066](https://linear.app/getsentry/issue/CW-3066/show-out-of-credits-stops-in-pr-iteration-and-stop-them-pausing) for planned followup work --------- Co-authored-by: sentry-junior[bot] <264270552+sentry-junior[bot]@users.noreply.github.com>

`sentry issue explain` and `sentry issue plan` treat every 402 from the autofix endpoint as "Seer requires a paid plan" and offer a free trial. An org on token-based billing that runs out of Seer credits gets that message too. A run that starts and then stops for credits reads as "Analysis completed but no root causes found" or "Plan creation failed". - A 402 with `code: "seer_out_of_credits"` (added in getsentry/sentry#130284) now raises a new `out_of_credits` Seer error. Any other 402, including `seer_out_of_quota` and the 402 with no code that older Sentry versions send, keeps the current no-budget error. - A run that ends with `failure_reason: "out_of_credits"` raises the same error from both commands. - The error has its own exit code, 43, and is left out of the free trial prompt. What the user sees: ``` Your organization is out of Seer credits. Someone with billing access can adjust Seer spend and auto-reload in Sentry: https://my-org.sentry.io/settings/billing/overview/?product=seer ``` A failed run is still retried on the next invocation, so someone who topped up recovers by running the command again. The 402 comes from getsentry/sentry#130283 and getsentry/sentry#130284, which have both merged. Sentry's credit checks are behind the `organizations:seer-credit-checks` flag, so for an org without it the CLI behaves as it does today. The MCP equivalent is #2031. Fixes CW-3075

srest2021 added 2 commits November 6, 2026 10:42

feat(seer): Gate agent chat messages on Seer credits

cbdde3a

Return a 402 before sending a chat message when a token-billed org does not have enough Seer credits. Dashboard generation runs are not gated. can_run_seer_operation now passes orgs that are not token-billed. Refs CW-3061

srest2021 mentioned this pull request Nov 6, 2026

feat(seer-billing): Handle SeerOutOfCreditsException at Autofix callers #130284

Merged

github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 6, 2026

srest2021 added 5 commits November 6, 2026 10:48

ref(seer): Keep the token-billed check inside can_run_seer_operation

e2855ba

Merge branch 'srest2021/CW-3061-agent-chat' into srest2021/CW-3061-au…

33f5287

…tofix-gating

ref(seer): Split has_enough_seer_credits out of can_run_seer_operation

d70d6cb

Autofix billing checks the token-billed flag itself, so it only needs the credit check.

ref(seer): Combine the credit check flag conditions

58a1b63

Merge branch 'srest2021/CW-3061-agent-chat' into srest2021/CW-3061-au…

8856cd4

…tofix-gating # Conflicts: # src/sentry/seer/credits.py

srest2021 added 9 commits November 6, 2026 11:06

Merge remote-tracking branch 'origin/master' into srest2021/CW-3061-a…

c49f7ed

…gent-chat # Conflicts: # src/sentry/seer/endpoints/organization_seer_agent_chat.py

Merge branch 'srest2021/CW-3061-agent-chat' into srest2021/CW-3061-au…

8e5ca8f

…tofix-gating # Conflicts: # src/sentry/seer/autofix/feature/dispatch.py # tests/sentry/seer/autofix/feature/test_dispatch.py

feat(seer): Check Seer credits before replying to an Autofix single run

0ab3b10

Chat replies to single runs continue the run through continue_single_run, which skipped billing. The run's own referrer decides, so agentic triage runs stay free.

Merge remote-tracking branch 'origin/master' into srest2021/CW-3061-a…

69f5488

…gent-chat # Conflicts: # src/sentry/seer/endpoints/organization_seer_agent_chat.py

Merge branch 'srest2021/CW-3061-agent-chat' into srest2021/CW-3061-au…

0f648e8

…tofix-gating

ref(seer): Check the credit check flag first

a3dae0d

Merge branch 'srest2021/CW-3061-agent-chat' into srest2021/CW-3061-au…

f738ddf

…tofix-gating Check the credit check flag before the token-billed flag. has_enough_seer_credits no longer checks the flag; its callers do.

test(seer): Align credit check tests with their files

94b979c

Cover orgs that aren't token-billed, group the chat credit tests after the access tests, and match the neighboring tests' names and setup.

Merge branch 'srest2021/CW-3061-agent-chat' into srest2021/CW-3061-au…

302de4f

…tofix-gating

Base automatically changed from srest2021/CW-3061-agent-chat to master November 6, 2026 20:03

Merge remote-tracking branch 'origin/master' into srest2021/CW-3061-a…

0ec42da

…utofix-gating # Conflicts: # src/sentry/seer/credits.py

vercel Bot deployed to Preview November 6, 2026 20:07

View deployment

ref(seer): Leave single run replies out of credit gating

6f7cb6f

Single runs are behind organizations:autofix-single-run and need their own estimate, so they're out of scope here.

sentry-warden Bot reviewed Nov 6, 2026

View reviewed changes

Comment thread src/sentry/seer/autofix/autofix_agent.py Resolved

srest2021 added 5 commits November 6, 2026 12:43

fix(seer): Don't let API callers claim the agentic triage referrer

99eb2ac

Agentic triage runs skip the Seer credit check, and the Autofix endpoint accepted any referrer from the request body. Treat a client-supplied agentic triage referrer as unknown.

ref(seer): Clarify why the endpoint ignores the agentic triage referrer

e8f415e

feat(seer): Check Seer credits before replying to an Autofix single run

396779d

Restores the single run reply check, using the placeholder estimate for now.

cleaning up diff

90c5db5

ref(seer): Restore the billing comment in trigger_autofix_agent

ff4faae

srest2021 marked this pull request as ready for review November 6, 2026 21:54

srest2021 requested a review from a team as a code owner November 6, 2026 21:54

srest2021 requested a review from a team November 6, 2026 21:54

Merge remote-tracking branch 'origin/master' into srest2021/CW-3061-a…

4e289d3

…utofix-gating

vercel Bot deployed to Preview November 7, 2026 00:15

View deployment

This was referenced Nov 9, 2026

feat(seer-billing): Show out-of-credits Seer failures in MCP getsentry/toolkit#2031

Merged

feat(seer): Show the credits notice when a chat runs out mid-conversation #130463

Merged

billyvg reviewed Nov 9, 2026

View reviewed changes

Comment thread src/sentry/seer/autofix/feature/dispatch.py Resolved

billyvg reviewed Nov 9, 2026

View reviewed changes

Comment thread src/sentry/seer/autofix/autofix_agent.py Resolved

Comment thread src/sentry/seer/autofix/feature/dispatch.py Resolved

Comment thread src/sentry/seer/autofix/autofix_agent.py Resolved

Comment thread src/sentry/seer/autofix/feature/dispatch.py Resolved

ref(seer-billing): Pass organization to check_autofix_billing

451f3bd

Co-Authored-By: Sofia Rest <sofia.rest@sentry.io>

sehr-m reviewed Nov 9, 2026

View reviewed changes

Comment thread src/sentry/seer/autofix/feature/dispatch.py Resolved

Comment thread src/sentry/seer/autofix/feature/dispatch.py Resolved

Comment thread src/sentry/seer/autofix/feature/dispatch.py Resolved

Comment thread src/sentry/seer/autofix/autofix_agent.py Resolved

ref(seer-billing): Check Autofix billing only in dispatch

ab0d0aa

trigger_autofix_agent checked billing, then called trigger_autofix_feature, which checked it again with the same arguments. Keep only the dispatch check, and test a token-billed org with credit checks off.

srest2021 mentioned this pull request Nov 9, 2026

feat(slack): Hide Fix with Seer on issue alerts when Seer credits run out #130271

Open

srest2021 mentioned this pull request Nov 10, 2026

feat(seer-billing): Skip credit checks on runs agentic triage starts #130552

Draft

srest2021 added this pull request to stack #130567 November 10, 2026 01:16

alexsohn1126 mentioned this pull request Nov 10, 2026

feat(slack): Reply in the thread when an autofix run runs out of Seer credits #129975

Draft

sehr-m approved these changes Nov 10, 2026

View reviewed changes

Comment thread src/sentry/seer/credits.py Resolved

ref(seer-billing): Pass the Seer feature to credit checks

5e7ccfc

An operation only sets the credit estimate, since it can run under more than one feature, so callers pass the feature explicitly. The credit check endpoint now takes a feature param.

cursor Bot reviewed Nov 10, 2026

View reviewed changes

Comment thread src/sentry/seer/endpoints/organization_seer_credit_check.py Resolved

ref(seer-billing): Pick the credit check feature from the operation

1a78b12

The UI can't tell runs agentic triage started apart yet, so the credit check endpoint picks the feature itself instead of requiring the UI to send it.

srest2021 removed this pull request from stack #130567 November 10, 2026 19:35

jaydgoss mentioned this pull request Nov 10, 2026

feat(cli): Tell users when Seer is out of credits getsentry/toolkit#2150

Merged

srest2021 merged commit a2fff8f into master Nov 10, 2026
80 checks passed

srest2021 deleted the srest2021/CW-3061-autofix-gating branch November 10, 2026 19:42

sehr-m mentioned this pull request Nov 10, 2026

feat(billing): share Seer billing feature identifiers getsentry/sentry-protos#521

Merged

Conversation

srest2021 commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

linear-code Bot commented Nov 6, 2026

Uh oh!

sentry Bot commented Nov 6, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Sentry Snapshot Testing

Uh oh!

Uh oh!

Uh oh!

billyvg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

sehr-m left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

srest2021 commented Nov 9, 2026

Uh oh!

sehr-m left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

cursor Bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Reviewers

Uh oh!

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

srest2021 commented Nov 6, 2026 •

edited

Loading

sentry Bot commented Nov 6, 2026 •

edited

Loading