You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Feature request: an opt-in router placement policy that usesparent_session_id(or the lineage root) to place new subagent sessions near the prefix they share — turning the already-parsed agentic identity hints into a routing signal.
Background
Dynamo today has, at two separate layers:
Identity/tracing layer: X-Dynamo-Session-ID / X-Dynamo-Parent-Session-ID (and nvext equivalents) are parsed into internal session identity and request traces. Per docs/agents/session-ids.md, the parent id is passive metadata — "traces and replay tools can rebuild the tree"; it does not affect placement.
What is missing is the step in between: a child session's first request is unbound, and unless KV-event prefix indexing is deployed and warm, the router places it with no knowledge that a worker already holds most of its prompt.
Problem
Agentic workloads are trees: a root agent spawns subagents, and each subagent is its own session. In public agentic coding trace corpora (e.g. the semianalysisai/cc-traces-weka-* datasets), the majority of requests come from subagents, and sibling subagents share a large common prefix — mostly with each other, and to a lesser degree with the root agent.
With per-session affinity only, every spawned subagent's first request can land on a different worker (and, for data-parallel attention deployments, a different DP rank), re-prefilling the shared prefix once per sibling. The KV/prefix cost scales with fan-out.
Proposal
An opt-in router policy, roughly:
When a request carries parent_session_id and its own session has no affinity yet, prefer the worker (and DP rank) currently bound to the parent's session (or to a sibling-group key derived from the parent), subject to load thresholds.
Keep it soft: this is an initial-placement preference, not a hard pin — the router remains free to rebalance or to spread siblings once the shared prefix is resident (e.g. future D2D/host-pool distribution).
Make the grouping key configurable (parent vs root), since sibling↔sibling sharing is typically stronger than child↔root sharing.
A complementary engine-side idea (out of scope here, backend territory): scheduler-level common-prefix gating — schedule one sibling first and release the rest once the shared prefix is cached, so co-located siblings don't all compute the same prefix concurrently.
Workaround today (and why it isn't the right long-term shape)
Clients can alias each child's session id to the lineage root so the whole tree rides one session-affinity entry (client-side only; see SemiAnalysisAI/aiperf#14 for a benchmark-client implementation). It works, but it overloads session identity, hard-pins the entire tree to one worker/rank, and removes the router's freedom to rebalance or fan out — all things a first-class parent-aware policy would keep.
Summary
Feature request: an opt-in router placement policy that uses
parent_session_id(or the lineage root) to place new subagent sessions near the prefix they share — turning the already-parsed agentic identity hints into a routing signal.Background
Dynamo today has, at two separate layers:
X-Dynamo-Session-ID/X-Dynamo-Parent-Session-ID(andnvextequivalents) are parsed into internal session identity and request traces. Perdocs/agents/session-ids.md, the parent id is passive metadata — "traces and replay tools can rebuild the tree"; it does not affect placement.What is missing is the step in between: a child session's first request is unbound, and unless KV-event prefix indexing is deployed and warm, the router places it with no knowledge that a worker already holds most of its prompt.
Problem
Agentic workloads are trees: a root agent spawns subagents, and each subagent is its own session. In public agentic coding trace corpora (e.g. the
semianalysisai/cc-traces-weka-*datasets), the majority of requests come from subagents, and sibling subagents share a large common prefix — mostly with each other, and to a lesser degree with the root agent.With per-session affinity only, every spawned subagent's first request can land on a different worker (and, for data-parallel attention deployments, a different DP rank), re-prefilling the shared prefix once per sibling. The KV/prefix cost scales with fan-out.
Proposal
An opt-in router policy, roughly:
parent_session_idand its own session has no affinity yet, prefer the worker (and DP rank) currently bound to the parent's session (or to a sibling-group key derived from the parent), subject to load thresholds.parentvsroot), since sibling↔sibling sharing is typically stronger than child↔root sharing.A complementary engine-side idea (out of scope here, backend territory): scheduler-level common-prefix gating — schedule one sibling first and release the rest once the shared prefix is cached, so co-located siblings don't all compute the same prefix concurrently.
Workaround today (and why it isn't the right long-term shape)
Clients can alias each child's session id to the lineage root so the whole tree rides one session-affinity entry (client-side only; see SemiAnalysisAI/aiperf#14 for a benchmark-client implementation). It works, but it overloads session identity, hard-pins the entire tree to one worker/rank, and removes the router's freedom to rebalance or fan out — all things a first-class parent-aware policy would keep.
References
docs/agents/session-ids.md,docs/agents/agent-replay.md— parent id defined as passive metadata