Skip to content

feat(seer-billing): Skip credit checks on runs agentic triage starts - #130552

Draft
srest2021 wants to merge 73 commits into
sehrm/attribute-slack-seer-agent-chatsfrom
srest2021/CW-3061-agentic-triage-feature
Draft

srest2021 wants to merge 73 commits into
sehrm/attribute-slack-seer-agent-chatsfrom
srest2021/CW-3061-agentic-triage-feature

Conversation

@srest2021

@srest2021 srest2021 commented Nov 10, 2026 •

Copy link
Copy Markdown
Member

Stacked on #130729, which sends the billing feature on new runs. Depends on getsentry/seer#9515, which must deploy first.

Agentic triage isn't billed yet, so nothing on an Autofix run it starts should be charged or blocked for credits. To do that, each Autofix run gets a billing feature that's set when it starts and kept for its whole life, credit checks use it, and the autofix response returns it for the UI.

Billing features are the shared SeerBillingFeature proto enum (getsentry/sentry-protos#521). New runs are checked under the feature #130729 resolves for them (agentic triage starts runs with AUTOFIX_AGENTIC_TRIAGE), and continuations under the one Seer stored on the run (getsentry/seer#9515). Runs without one are unattributed.

The post-process skip for token-billed orgs moved to #130825.

Refs CW-3061

via sofia.

--

View Session in Sentry

Return a 402 before sending a chat message when a token-billed org does not have enough Seer credits. Dashboard generation runs are not gated. can_run_seer_operation now passes orgs that are not token-billed.

Refs CW-3061
Token-billed orgs are checked for Seer credits before each Autofix step and before opening a PR, and get SeerOutOfCreditsException when they can't pay. Other orgs keep the seat-based quota check. Agentic triage runs are not gated.

Refs CW-3061
The issue page returns a 402, Slack shows an error, the admin endpoints report it, and automatic paths (new issue automation, completion hook, PR iteration) log and stop.

Refs CW-3061
Autofix billing checks the token-billed flag itself, so it only needs the credit check.
…tofix-gating

# Conflicts:
#	src/sentry/seer/credits.py
…gent-chat

# Conflicts:
#	src/sentry/seer/endpoints/organization_seer_agent_chat.py
…tofix-gating

# Conflicts:
#	src/sentry/seer/autofix/feature/dispatch.py
#	tests/sentry/seer/autofix/feature/test_dispatch.py
…1-autofix-out-of-credits

# Conflicts:
#	src/sentry/seer/endpoints/group_ai_autofix.py
Chat replies to single runs continue the run through continue_single_run, which skipped billing. The run's own referrer decides, so agentic triage runs stay free.
…gent-chat

# Conflicts:
#	src/sentry/seer/endpoints/organization_seer_agent_chat.py
…tofix-gating

Check the credit check flag before the token-billed flag. has_enough_seer_credits no longer checks the flag; its callers do.
…1-autofix-out-of-credits

# Conflicts:
#	src/sentry/seer/entrypoints/operator.py
#	tests/sentry/seer/endpoints/test_group_ai_autofix.py
Cover orgs that aren't token-billed, group the chat credit tests after the access tests, and match the neighboring tests' names and setup.
…utofix-gating

# Conflicts:
#	src/sentry/seer/credits.py
Single runs are behind organizations:autofix-single-run and need their own estimate, so they're out of scope here.
…its"

Single run replies are out of scope for these PRs.
Agentic triage runs skip the Seer credit check, and the Autofix endpoint accepted any referrer from the request body. Treat a client-supplied agentic triage referrer as unknown.
Checking existing_run_id directly lets mypy narrow it, and is_new_run can't drift from the lookup.

Refs CW-3061
…' into srest2021/CW-3061-autofix-out-of-credits
…-credits' into srest2021/CW-3061-agentic-triage-feature
…ge runs

Feedback typed in the UI is credit-checked before it's queued. The run's feature is only read when the check refuses, so most requests make no extra query.

Refs CW-3061
@srest2021
srest2021 marked this pull request as ready for review November 10, 2026 00:54
@srest2021
srest2021 requested review from a team as code owners November 10, 2026 00:54
@srest2021
srest2021 changed the base branch from srest2021/CW-3061-autofix-gating to srest2021/CW-3061-autofix-out-of-credits November 10, 2026 00:54

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit d15d1c0. Configure here.

Comment thread src/sentry/tasks/post_process.py Resolved
Comment thread src/sentry/seer/credits.py Outdated
Comment on lines 25 to 30
class SeerFeature(StrEnum):
"""Seer features as tagged on LLM proxy calls, which billing maps to data categories."""

AGENTIC_TRIAGE = "agentic_triage" # Not billed yet. Marks the Autofix runs it starts.
AUTOFIX = "autofix"
EXPLORER = "explorer"

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Alternative: make this a protobuf and keep it in sync between getsentry, sentry, and seer

@sehr-m sehr-m left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you go with the changes I suggest the seer side handling will need to be made before this can be merged, but lmk what you think!!

@srest2021
srest2021 removed this pull request from stack #130567 November 10, 2026 19:35
Base automatically changed from srest2021/CW-3061-autofix-out-of-credits to master November 10, 2026 21:08
…gentic-triage-feature

# Conflicts:
#	src/sentry/seer/autofix/autofix_agent.py
#	src/sentry/seer/autofix/feature/dispatch.py
#	src/sentry/seer/autofix/pr_iteration/manual_trigger.py
#	src/sentry/seer/credits.py
#	src/sentry/seer/endpoints/group_ai_autofix.py
#	src/sentry/seer/endpoints/organization_seer_agent_chat.py
#	tests/sentry/seer/autofix/feature/test_dispatch.py
#	tests/sentry/seer/autofix/test_autofix_agent.py
#	tests/sentry/seer/endpoints/test_group_ai_autofix.py
#	tests/sentry/seer/endpoints/test_organization_seer_agent_chat.py
Send billing_feature to Seer when an Autofix run starts, and read it back from the run state for later steps instead of storing it in extras. Rename SeerFeature to SeerBillingFeature to match the billing protos, and require billing_feature wherever Autofix billing is checked so a caller can't leave it out. Expose it in the autofix response and accept it on the credit check endpoint.
…-chats' into srest2021/CW-3061-agentic-triage-feature

# Conflicts:
#	tests/sentry/seer/endpoints/test_group_ai_autofix.py
Use the billing feature sent by the shared dispatch for new runs and the stored feature for continuations, so credit checks and Seer agree.
@srest2021
srest2021 changed the base branch from master to sehrm/attribute-slack-seer-agent-chats November 11, 2026 18:25

This branch was successfully deployed

1 active (outdated) deployment
Preview — 40643f08 Deployed Nov 10, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants