Skip to content

task: drain jobs and proxy streams on graceful shutdown#351

Merged
greatest0fallt1me merged 1 commit into
CalloraOrg:mainfrom
Bug-Hunter-X:task/graceful-shutdown-draining
May 27, 2026
Merged

task: drain jobs and proxy streams on graceful shutdown#351
greatest0fallt1me merged 1 commit into
CalloraOrg:mainfrom
Bug-Hunter-X:task/graceful-shutdown-draining

Conversation

@Bug-Hunter-X

@Bug-Hunter-X Bug-Hunter-X commented May 27, 2026

Copy link
Copy Markdown
Contributor

Overview

This PR improves graceful shutdown handling in src/index.ts so the backend drains in-flight gateway proxy requests and background webhook work before database resources are closed. It introduces coordinated shutdown hooks for drainable subsystems, preserves the existing single-execution process.once(...) behavior, and keeps termination bounded by the configured drain window.

Related Issue

Closes #334

Changes

⚙️ Graceful Shutdown Coordination

  • [MODIFY] src/index.ts
  • Added a drainable subsystem contract for graceful shutdown.
  • Coordinated shutdown so subsystems stop accepting new work and wait for in-flight work to finish before DB teardown.
  • Preserved single in-flight shutdown handling to avoid duplicate execution on rapid repeated signals.
  • Added an in-flight /v1/call tracker so active gateway proxy requests can finish within the drain window.

🌐 Proxy Drain Handling

  • [MODIFY] src/index.ts
  • Mounted a request tracker ahead of the gateway proxy router.
  • During shutdown, active proxy responses are awaited instead of being abandoned immediately.
  • Lingering sockets are still destroyed once the timeout expires, preserving bounded shutdown behavior.

🪝 Webhook Dispatcher Draining

  • [MODIFY] src/webhooks/webhook.dispatcher.ts
  • Added hooks to stop accepting new webhook deliveries once shutdown begins.
  • Tracked in-flight webhook deliveries so graceful shutdown can await pending sends.
  • Added a small test reset helper to keep dispatcher state isolated across tests.

🧪 Tests

  • [MODIFY] src/index.test.ts

  • Added coverage for subsystem drain ordering.

  • Added coverage for socket destruction after timeout.

  • Added coverage that repeated signals reuse the same shutdown flow.

  • Added coverage for active proxy request draining.

  • [MODIFY] src/webhooks/webhook.dispatcher.test.ts

  • Added coverage that new webhook deliveries are skipped after shutdown begins.

📘 Documentation

  • [MODIFY] README.md
  • Documented that graceful shutdown now drains in-flight proxy work and webhook deliveries before closing data resources.

Verification Results

Acceptance Criteria Status
Background jobs stop scheduling and finish in-flight runs on shutdown
Active proxy streams complete within the drain window
No duplicate shutdown execution on rapid restarts
Database resources close only after draining
Focused shutdown tests pass successfully
Typecheck passes successfully

Screenshots

✅ Graceful shutdown tests pass

A screenshot of:

npm test -- src/index.test.ts
image ### ✅ Webhook dispatcher drain tests pass A screenshot of:
npm test -- src/webhooks/webhook.dispatcher.test.ts
image

@drips-wave

drips-wave Bot commented May 27, 2026

Copy link
Copy Markdown

@Bug-Hunter-X Great news! 🎉 Based on an automated assessment of this PR, the linked Wave issue(s) no longer count against your application limits.

You can now already apply to more issues while waiting for a review of this PR. Keep up the great work! 🚀

Learn more about application limits

@greatest0fallt1me greatest0fallt1me merged commit 4e7ca39 into CalloraOrg:main May 27, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Wire graceful-shutdown draining of in-flight gateway proxy and background jobs in index.ts

3 participants