Skip to content

perf(devserver): Start the web without loading the app or waiting for Kafka - #130860

Merged
mchen-sentry merged 3 commits into
masterfrom
perf/devserver-settings-only-start
Nov 12, 2026
Merged

mchen-sentry merged 3 commits into
masterfrom
perf/devserver-settings-only-start

Conversation

@sentry-junior

@sentry-junior sentry-junior Bot commented Nov 12, 2026

Copy link
Copy Markdown
Contributor

sentry devserver loaded all of Sentry (~7–10 s) only to read settings, pick its daemons, and create Kafka topics. Then the web server started and loaded all of Sentry again. On a fresh boot, Kafka usually answers only at about that moment, so the web also waited for Kafka.

Now the devserver:

  • Loads settings only, through a new settings_configuration decorator (configure(settings_only=True)). The paths that serve from this process (--debug-server, and no daemons) still run the full configure().
  • Decides on the post-process forwarder from SENTRY_EVENTSTREAM, because asking the backend imports models. A test checks the list against every EventStream subclass's requires_post_process_forwarder().
  • Creates Kafka topics in a background thread, so the web starts right away. The thread probes Kafka with a fresh client until Kafka answers. A client that can't reach Kafka backs off for up to 10 s, which cost ~9 s when settings loaded before Kafka was up.
  • Holds back the consumers until the topics exist. A consumer that subscribes to a missing topic crashes with UNKNOWN_TOPIC_OR_PART, and that stops everything. honcho runs commands through sh, so each consumer command first waits for .artifacts/devserver-kafka-topics-ready.
  • Prints the traceback and stops the whole devserver if creating topics fails.
  • Lets child processes validate the backends. Before, the parent validated them and then set SENTRY_SKIP_BACKEND_VALIDATION for its children.

Time from launch to the first 200 on /auth/login/ through rsbuild on :8000, measured in a sandbox with devservices up (3 starts each):

  • Kafka already up: 12.7 / 13.5 / 13.3 s before, 9.3 / 8.4 / 8.8 s after.
  • Kafka container just started: 18.6 / 19.2 / 18.2 s before, 8.3 / 10.5 / 9.7 s after.

With every topic deleted and --workers --occurrence-ingest, the web answered in 10.3 s. All topics existed 19 s after launch, and only then did the consumer start and get its partitions. A bad cluster mapping printed its KeyError traceback to the terminal and dev.log, then stopped every process.

via Ming Chen.

--

View Session [Sentry]

… Kafka

The devserver loaded all of Sentry only to read settings and ask the eventstream backend
whether it needs the post-process forwarder, then created Kafka topics before starting
anything. It now loads settings only, decides the forwarder from SENTRY_EVENTSTREAM, and
creates topics in a background thread that waits for Kafka. Consumers wait for the
topics, since one that subscribes to a missing topic crashes. A topic failure prints its
traceback and stops the devserver. The children validate the backends, since this process
no longer does.

Co-Authored-By: Ming Chen <ming.chen@sentry.io>
@github-actions github-actions Bot added the Scope: Backend Automatically applied to PRs that change backend components label Nov 12, 2026

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit b0728dc. Configure here.

Comment thread src/sentry/runner/commands/devserver.py Resolved
Comment thread src/sentry/runner/commands/devserver.py Resolved
sentry-junior Bot and others added 2 commits November 12, 2026 00:29
Granian forks its workers, so the in-process path creates topics synchronously instead of leaving Kafka client threads alive across the fork. Exit after the in-process server stops, rather than falling through into honcho and starting a second server.
Topic creation now prints one "waiting for kafka at <host>" line,
silences librdkafka's per-attempt connect errors, and gives up after
60s with the traceback, which stops the devserver.

The --no-watchers path creates topics in a forked child started before
Granian forks its workers, so no worker copies Kafka client threads and
the web no longer waits for topics.

The tests import from sentry.runner.commands.devserver directly, so
selective testing picks them for any change to devserver.py.

Co-Authored-By: Ming Chen <ming.chen@sentry.io>
@mchen-sentry
mchen-sentry merged commit 8dda936 into master Nov 12, 2026
72 checks passed
@mchen-sentry
mchen-sentry deleted the perf/devserver-settings-only-start branch November 12, 2026 02:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Scope: Backend Automatically applied to PRs that change backend components

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant