Skip to content

Commit 9836bff

Browse files
pimlockdrew
andauthored
fix(kubernetes): stop supervisor before workload (#3424)
* fix(kubernetes): stop supervisor before workload Make Kubernetes sandbox stop crash-resumable and give supervisor Pod deletion its configured grace period plus API observation headroom.\n\nCloses #3421 Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(kubernetes): harden resumed sandbox stops Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * test(kubernetes): simplify resumed stop regression Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> * fix(kubernetes): generalize runtime control release Signed-off-by: Drew Newberry <anewberry@nvidia.com> * refactor(kubernetes): clarify stop phase names Signed-off-by: Drew Newberry <anewberry@nvidia.com> --------- Signed-off-by: Piotr Mlocek <pmlocek@nvidia.com> Signed-off-by: Drew Newberry <anewberry@nvidia.com> Co-authored-by: Drew Newberry <anewberry@nvidia.com>
1 parent 50c5cf8 commit 9836bff

4 files changed

Lines changed: 596 additions & 73 deletions

File tree

architecture/compute-runtimes.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -186,6 +186,15 @@ A driver stop operation does not complete while its backend still reports an
186186
in-progress stop. This prevents an immediate start from racing the previous
187187
run's delayed exit event and regressing the new run to `Error`.
188188

189+
The Kubernetes driver records stop as a durable two-phase transition. The
190+
`releasing` phase releases the sandbox's runtime-control relationship while the
191+
workload boundary remains reachable. The `suspending` phase then suspends the
192+
Agent Sandbox workload and cleans generation bootstrap material. The current
193+
dedicated-supervisor implementation releases control by deleting the supervisor
194+
Pod. Periodic reconciliation resumes either phase after a gateway restart. Pod
195+
deletion waits include the configured termination grace period plus Kubernetes
196+
API observation headroom.
197+
189198
Persisted `Stopping` and `Starting` rows are retried at startup. Stable
190199
`Stopped` rows remain stopped. Docker and Podman retain the stopped container
191200
and attached storage, Kubernetes retains the Sandbox CR and PVC while scaling

0 commit comments

Comments
 (0)