Skip to content

opt-in ping_interval_secs keepalive on the listener#1942

Open
Atlas1225 wants to merge 1 commit into
iii-hq:mainfrom
Atlas1225:feat/worker-manager-ping-interval
Open

opt-in ping_interval_secs keepalive on the listener#1942
Atlas1225 wants to merge 1 commit into
iii-hq:mainfrom
Atlas1225:feat/worker-manager-ping-interval

Conversation

@Atlas1225

@Atlas1225 Atlas1225 commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

The worker-manager listener has no active liveness mechanism: the engine
only answers Ping frames, never initiates them, and a connection whose
peer died without FIN/RST (suspended device, NAT-dropped path) lingers in
the worker and trigger registries until a write to it exhausts TCP
retransmissions.

Add an opt-in ping_interval_secs to WorkerManagerConfig. When set, the
listener sends a WebSocket Ping frame to each connection every interval
and closes connections that produce no inbound frame (Pong included —
browsers and WS libraries auto-reply per RFC 6455) for two consecutive
intervals. Unset keeps the previous behavior unchanged.

  • I license my contributions to this repository under Apache 2.0, and I have all necessary rights over the code I am contributing.

Summary by CodeRabbit

  • New Features

    • Added optional WebSocket keepalive checks for worker connections, with configurable ping intervals.
    • Connections now stay alive during normal activity and are closed only after extended inactivity.
  • Bug Fixes

    • Improved handling of unresponsive worker connections to prevent stale sessions from lingering.
    • Preserved existing behavior when keepalive checks are not configured.
  • Tests

    • Added end-to-end coverage for silent, responsive, and disabled keepalive scenarios.

@vercel

vercel Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Someone is attempting to deploy a commit to the motia Team on Vercel.

A member of the Team first needs to authorize it.

@iii-hq-ci

iii-hq-ci Bot commented Jul 8, 2026

Copy link
Copy Markdown

License agreement recorded

@Atlas1225, your agreement has been recorded and you have been added to contributors.md. All future PRs from your account will pass this check automatically.

@coderabbitai

coderabbitai Bot commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Review Change Stack

📝 Walkthrough

Walkthrough

Adds an opt-in ping_interval_secs config option to WorkerManagerConfig, implements periodic WebSocket ping and inactivity-based connection reaping in Engine::handle_worker, updates test wiring accordingly, and adds a new E2E test suite covering reaping, keepalive, and disabled behaviors.

Changes

Ping Keepalive Feature

Layer / File(s) Summary
Config field for ping interval
engine/src/workers/worker/mod.rs, engine/tests/rbac_infrastructure_e2e.rs
WorkerManagerConfig adds optional ping_interval_secs: Option<u64> with serde default, Default impl sets it to None, and test session config explicitly sets it to None.
Ping/health-check logic in handle_worker
engine/src/engine/mod.rs
Computes optional Duration from config, tracks last_inbound, drives a Tokio interval timer, closes connections idle beyond two ping intervals, and otherwise sends periodic Ping frames.
E2E tests for ping keepalive
engine/tests/worker_manager_ping_keepalive_e2e.rs
New test file with spawn_engine and wait_for_workers helpers and three tests validating reaping of silent connections, persistence of responsive connections, and no reaping when ping is disabled.

Estimated code review effort: 3 (Moderate) | ~25 minutes

Sequence Diagram(s)

sequenceDiagram
  participant Client
  participant Engine
  participant WorkerRegistry

  Engine->>Engine: start ping interval timer
  loop on each tick
    alt inbound inactive beyond 2x interval
      Engine->>WorkerRegistry: close/reap connection
    else
      Engine->>Client: send WebSocket Ping frame
    end
  end
  Client-->>Engine: inbound frame received
  Engine->>Engine: update last_inbound timestamp
Loading

Poem

A rabbit taps the wire, tick-tock, tick-tock,
"Are you there, dear worker?" — knock, knock, knock! 🐇
Silent burrows get swept away,
while chatty ones are here to stay.
Ping, pong, ping — my whiskers cheer,
a tidy warren, year to year! 🥕

🚥 Pre-merge checks | ✅ 5
✅ Passed checks (5 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Title check ✅ Passed The title is concise and accurately describes the main change: opt-in ping_interval_secs keepalive support on the listener.
Description check ✅ Passed The description clearly covers what changed and why, and includes reviewer notes, though it does not use the template headings exactly.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
engine/src/engine/mod.rs (1)

1542-1607: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Document ping_interval_secs in worker-manager docs and SDK examples. The engine accepts the new option, but the worker-manager docs and SDK-facing config surface don’t expose it yet.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/engine/mod.rs` around lines 1542 - 1607, The worker ping interval
option is implemented in the engine loop, but it is not exposed in the
worker-manager documentation or SDK-facing configuration examples. Update the
relevant docs and examples to include ping_interval_secs, and make sure the
worker-manager config surface and any SDK config structs or builders that feed
into this code path reference the new option consistently so users can discover
and set it.

Source: Path instructions

🧹 Nitpick comments (2)
engine/src/workers/worker/mod.rs (1)

275-282: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Extend default-value test to cover the new field.

worker_config_default_values asserts port/host/middleware/rbac defaults but doesn't assert ping_interval_secs defaults to None, leaving the new field's default behavior unverified by this test.

✅ Proposed test addition
         assert!(config.middleware_function_id.is_none());
         assert!(config.rbac.is_none());
+        assert!(config.ping_interval_secs.is_none());
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/workers/worker/mod.rs` around lines 275 - 282, The
worker_config_default_values test in WorkerManagerConfig should also verify the
new ping_interval_secs field defaults to None. Update the existing assertion set
in worker_config_default_values to include an assertion for ping_interval_secs
alongside the current port, host, middleware_function_id, and rbac checks so the
default behavior of WorkerManagerConfig remains covered.
engine/src/engine/mod.rs (1)

1542-1607: 🩺 Stability & Availability | 🔵 Trivial

Consider whether /otel connections need the same liveness check.

handle_otel (unchanged in this PR) shares the same WebSocket transport and can equally hang on a peer that disappears without FIN/RST (e.g., device suspension, NAT path loss), but it has no ping/reap mechanism. This is out of scope for this PR's stated goal (worker-manager listener), but may be worth a follow-up for consistency.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@engine/src/engine/mod.rs` around lines 1542 - 1607, The `/otel` WebSocket
path currently lacks the same ping-based liveness handling used in the worker
connection loop, so `handle_otel` can hang indefinitely on dead peers. Update
`handle_otel` to share the same ping/reap behavior as the existing
`ping_timer`/`ws_rx.next()` loop in `engine::mod`, or refactor the shared
connection-handling logic so both `handle_otel` and the worker listener use the
same keepalive mechanism.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Outside diff comments:
In `@engine/src/engine/mod.rs`:
- Around line 1542-1607: The worker ping interval option is implemented in the
engine loop, but it is not exposed in the worker-manager documentation or
SDK-facing configuration examples. Update the relevant docs and examples to
include ping_interval_secs, and make sure the worker-manager config surface and
any SDK config structs or builders that feed into this code path reference the
new option consistently so users can discover and set it.

---

Nitpick comments:
In `@engine/src/engine/mod.rs`:
- Around line 1542-1607: The `/otel` WebSocket path currently lacks the same
ping-based liveness handling used in the worker connection loop, so
`handle_otel` can hang indefinitely on dead peers. Update `handle_otel` to share
the same ping/reap behavior as the existing `ping_timer`/`ws_rx.next()` loop in
`engine::mod`, or refactor the shared connection-handling logic so both
`handle_otel` and the worker listener use the same keepalive mechanism.

In `@engine/src/workers/worker/mod.rs`:
- Around line 275-282: The worker_config_default_values test in
WorkerManagerConfig should also verify the new ping_interval_secs field defaults
to None. Update the existing assertion set in worker_config_default_values to
include an assertion for ping_interval_secs alongside the current port, host,
middleware_function_id, and rbac checks so the default behavior of
WorkerManagerConfig remains covered.

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository UI

Review profile: CHILL

Plan: Pro

Run ID: 3c8ef60b-e861-4b9c-8fa7-b9735c0c324c

📥 Commits

Reviewing files that changed from the base of the PR and between 1cb3feb and ae10d96.

📒 Files selected for processing (4)
  • engine/src/engine/mod.rs
  • engine/src/workers/worker/mod.rs
  • engine/tests/rbac_infrastructure_e2e.rs
  • engine/tests/worker_manager_ping_keepalive_e2e.rs

@Atlas1225

Copy link
Copy Markdown
Contributor Author

Do I need to do any change?

@guibeira guibeira self-requested a review July 8, 2026 23:13
@guibeira

guibeira commented Jul 8, 2026

Copy link
Copy Markdown
Contributor

Hey @Atlas1225 thanks for your contribution! This really makes sense. Before I merge it, do you mind fixing the linter issue?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants