Skip to content

[DGX Spark][Sandbox] serving profile provenance is never recorded, so status cannot report it before or after rebuild #12289

Description

@hulynn

Warning

Regression — worked in: never confirmed working; broken in v0.0.128 (and in v0.0.124 / v0.0.125 before it). Two successive fixes have been merged and closed against this symptom without it being resolved.

Note

Re-verified reopening of #11896 — closed via #11924 (merge commit e1907e40b, confirmed an ancestor of v0.0.128). Re-verification on v0.0.128 still shows no serving-profile provenance. #11896 was itself the re-filing of #11417 after the same symptom survived that issue's fix. See the QA verify comment on the linked NVBug for the full repro transcript.

Description

A sandbox onboarded with an explicit managed serving profile does not report its serving profile, serving recipe or catalog digest in nemoclaw status, and the underlying servingProfileProvenance value is absent from the sandbox registry record entirely.

The original report framed this as a rebuild defect — provenance present before rebuild, gone after. Re-verification on v0.0.124/125 and again on v0.0.128 shows the value is missing before any rebuild as well: it is never written at onboard time. Fixes that change how rebuild handles an already-populated provenance value therefore cannot surface it.

The profile itself is honored — the registry's encoded profile decodes to the requested serving preset, and the sandbox serves the right model. Only the provenance reporting is missing.

Platform scope: Reproduced on DGX Spark GB10 aarch64 only; other platforms not tested in this pass.
Regression: Yes — the symptom has survived two merged fixes (#11417 closed by its fix, re-filed as #11896, closed by #11924) and still reproduces on v0.0.128.
OpenShell issue: No

Environment

Device:        DGX Spark GB10
OS:            Ubuntu 24.04.3 LTS
Architecture:  aarch64
NemoClaw:      nemoclaw v0.0.128
OpenClaw:      2026.9.1

Steps to Reproduce

  1. Pick a managed serving profile reported as compatible on the host:
nemoclaw profiles list
  1. Onboard with that profile explicitly:
nemoclaw onboard --name <sandbox> \
  --profile vllm.dgx-spark-gb10.single.nemotron-3-nano-4b-fp8 \
  --fresh --non-interactive --yes --yes-i-accept-third-party-software
  1. Check provenance before any rebuild, both ways:
nemoclaw <sandbox> status | grep -E 'Serving profile|Serving recipe|Catalog digest'
grep -o 'servingProfile[A-Za-z]*' ~/.nemoclaw/sandboxes.json
nemoclaw <sandbox> status --json
  1. Rebuild and confirm it succeeds:
nemoclaw <sandbox> rebuild --yes
  1. Repeat step 3.

Expected Result

After onboarding with an explicit managed serving profile, nemoclaw status reports that sandbox's serving profile, serving recipe and catalog digest, and those values are recorded in the sandbox registry. They remain reported after a successful rebuild.

A user who onboarded with a specific profile can confirm from the CLI which profile their sandbox is actually running, at any point in the sandbox's life.

Actual Result

Step 3, before any rebuild — provenance absent.

nemoclaw <sandbox> status prints no Serving profile:, Serving recipe: or Catalog digest: line at all. The status block shows only:

Model:    nvidia/NVIDIA-Nemotron-3-Nano-4B-FP8
Provider: vllm-local
Inference: healthy (https://inference.local/v1/models)
Agent:    OpenClaw v2026.9.1

In the registry, grep -o 'servingProfile[A-Za-z]*' ~/.nemoclaw/sandboxes.json returns zero matches for the entire file — the field is not present on the record in any form.

nemoclaw <sandbox> status --json explicitly emits:

"servingProfileProvenance": null

so the field is part of this build's status contract but carries no value.

~/.nemoclaw/onboard-session.json likewise records "servingProfileProvenance": null — the value is already null at onboard time, before any rebuild is involved.

The profile was genuinely applied: the registry's workload.encodedProfile decodes to "servingPreset": "vllm.dgx-spark-gb10.single.nemotron-3-nano-4b-fp8".

Step 4 — rebuild succeeds:

✓ Sandbox '<sandbox>' rebuild completed
  Now running: OpenClaw v2026.9.1
exit code 0

Step 5, after rebuild — identical. No provenance lines in status; grep for servingProfile in the registry still returns zero matches; status --json still emits null.

A second sandbox already present on the host, created earlier the same day on v0.0.128 with the same preset, had no servingProfileProvenance on its record either — consistent with the above.

Logs

Not captured beyond the terminal output quoted in Actual Result.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    NV QABugs found by the NVIDIA QA Team

    Type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions