feat: SLS-494 run users' worker initialization code - #567
Draft
jasonwang-runpod wants to merge 4 commits into
Draft
feat: SLS-494 run users' worker initialization code#567jasonwang-runpod wants to merge 4 commits into
jasonwang-runpod wants to merge 4 commits into
Conversation
Add optional `initializer` and `init_timeout` to the serverless config. The worker runs the initializer (e.g. model load / engine start) before it starts taking jobs. On failure or timeout it emits a structured `init_failed` log line, best-effort reports the reason to the platform, and exits non-zero so the existing backoff respawns it — instead of a broken worker sitting silently IN_QUEUE until the request times out. Opt-in and a no-op when no initializer is configured.
Sync and async initializers, InitializerError wrapping, init_timeout via SIGALRM, loop-gating (the job loop starts only after init succeeds), the init_failed payload shape, and the best-effort POST URL derivation.
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull request overview
This PR introduces a supervised startup phase for serverless workers by adding an optional initializer callable and init_timeout to the serverless worker config, ensuring model/engine startup failures (or hangs) are surfaced quickly and the worker exits before taking jobs.
Changes:
- Added
rp_initializermodule to run an initializer with timeout handling, structuredinit_failedlogging, and best-effort platform reporting. - Updated the worker startup sequence to run initialization before starting the job loop.
- Added unit tests covering sync/async initializers, error wrapping, timeouts, reporting, and job-loop gating.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.
| File | Description |
|---|---|
| tests/test_serverless/test_initializer.py | Adds unit tests for initializer execution, failure/timeout behavior, reporting, and gating before the job loop. |
| runpod/serverless/worker.py | Runs supervised initialization before starting the JobScaler/job loop. |
| runpod/serverless/modules/rp_initializer.py | Implements supervised initializer execution, timeout/error wrapping, structured logging, and best-effort reporting. |
| runpod/serverless/init.py | Documents new initializer and init_timeout config options in the serverless start() docstring. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
Author
|
bugbot run |
- detect functools.partial and async __call__ initializers (were run on the sync path and never awaited, silently skipping init) - catch Exception, not BaseException, so KeyboardInterrupt/SystemExit from user code propagate instead of being reported as init failures - use a single unittest import style (CodeQL)
jasonwang-runpod
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
A worker whose model load or engine start fails or hangs should fail fast and report the error.
This PR adds an optional
initializerandinit_timeoutto the serverless config. The SDK runs it before the worker takes any job. On failure or timeout it:init_failedlogTesting