Skip to content

feat(router): parent-aware placement for subagent sessions (agentic workloads) #11343

Description

@YAMY1234

Summary

Feature request: an opt-in router placement policy that uses parent_session_id (or the lineage root) to place new subagent sessions near the prefix they share — turning the already-parsed agentic identity hints into a routing signal.

Background

Dynamo today has, at two separate layers:

  • Identity/tracing layer: X-Dynamo-Session-ID / X-Dynamo-Parent-Session-ID (and nvext equivalents) are parsed into internal session identity and request traces. Per docs/agents/session-ids.md, the parent id is passive metadata — "traces and replay tools can rebuild the tree"; it does not affect placement.
  • Routing layer: session affinity keeps an individual session's turns on one worker once bound, and feat(router): coordinate session affinity across replicas [DYN-3249] #11079 coordinates that affinity across frontend replicas.

What is missing is the step in between: a child session's first request is unbound, and unless KV-event prefix indexing is deployed and warm, the router places it with no knowledge that a worker already holds most of its prompt.

Problem

Agentic workloads are trees: a root agent spawns subagents, and each subagent is its own session. In public agentic coding trace corpora (e.g. the semianalysisai/cc-traces-weka-* datasets), the majority of requests come from subagents, and sibling subagents share a large common prefix — mostly with each other, and to a lesser degree with the root agent.

With per-session affinity only, every spawned subagent's first request can land on a different worker (and, for data-parallel attention deployments, a different DP rank), re-prefilling the shared prefix once per sibling. The KV/prefix cost scales with fan-out.

Proposal

An opt-in router policy, roughly:

  1. When a request carries parent_session_id and its own session has no affinity yet, prefer the worker (and DP rank) currently bound to the parent's session (or to a sibling-group key derived from the parent), subject to load thresholds.
  2. Keep it soft: this is an initial-placement preference, not a hard pin — the router remains free to rebalance or to spread siblings once the shared prefix is resident (e.g. future D2D/host-pool distribution).
  3. Make the grouping key configurable (parent vs root), since sibling↔sibling sharing is typically stronger than child↔root sharing.

A complementary engine-side idea (out of scope here, backend territory): scheduler-level common-prefix gating — schedule one sibling first and release the rest once the shared prefix is cached, so co-located siblings don't all compute the same prefix concurrently.

Workaround today (and why it isn't the right long-term shape)

Clients can alias each child's session id to the lineage root so the whole tree rides one session-affinity entry (client-side only; see SemiAnalysisAI/aiperf#14 for a benchmark-client implementation). It works, but it overloads session identity, hard-pins the entire tree to one worker/rank, and removes the router's freedom to rebalance or fan out — all things a first-class parent-aware policy would keep.

References

Metadata

Metadata

Assignees

Labels

dynamo-runtimeRelates to the dynamo-runtime componentenhancementNew feature or requestrouterRelates to routing, KV-aware routing, etc.

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions