Skip to content

feat(network): support additional destination CAs - #3292

Open
jhjaggars wants to merge 5 commits into
NVIDIA:mainfrom
jhjaggars:feat/network-supervisor-additional-ca
Open

jhjaggars wants to merge 5 commits into
NVIDIA:mainfrom
jhjaggars:feat/network-supervisor-additional-ca

Conversation

@jhjaggars

@jhjaggars jhjaggars commented Sep 11, 2026

Copy link
Copy Markdown
Contributor

Summary

Add a global network-supervisor configuration for additional destination CA certificates so sandbox egress can trust private PKI without replacing default roots or changing gateway control-plane trust. Deliver the normalized trust bundle consistently through Docker, Podman, Kubernetes combined/sidecar, and VM compute paths, including lifecycle-safe certificate rollover for stopped sandboxes.

Related Issue

No linked accepted issue (process discrepancy): this feature was implemented and published by direct user request. An accepted issue is still required before this PR is ready to merge.

Changes

  • Add [openshell.supervisor.network].additional_ca_cert_paths with strict, bounded startup validation, canonicalization, redacted metadata, and fail-closed capability-based driver handling.
  • Keep effective-config preflight side-effect-free; stage gateway-owned trust material only after the selected driver passes compatibility checks.
  • Extend the shared network supervisor with additive rustls and child-process trust while preserving user TLS variables in direct mode and keeping proxy CA and gateway mTLS trust separate.
  • Stage the normalized bundle through read-only Docker/Podman mounts, Kubernetes immutable content-addressed ConfigMaps, and VM overlays.
  • Record and validate the expected trust generation at the supervisor boundary so mutable or stale runtime material fails closed.
  • Reconcile stopped sandboxes whose recorded trust generation differs from the restarted gateway snapshot while retaining sandbox identity and supported durable workspace contents.
  • Restrict Kubernetes ConfigMap RBAC to the operations required for immutable generation creation and validation; do not grant list, watch, patch, update, or delete for this feature.
  • Enforce one deployable, bounded trust-bundle limit across gateway, Kubernetes, VM, and supervisor consumers.
  • Add shared cross-driver additional-CA coverage for private/public trust, hostname enforcement, invalid staged material, callback isolation, and removal/restart behavior.
  • Update gateway reference, architecture, RFC, Helm, compute-driver, and cluster-debugging documentation.

Why lifecycle reconciliation is included

Additional CA sources are startup-only gateway configuration. Updating a source file or Helm source ConfigMap does not hot-reload running supervisors. To roll certificates, an operator updates the source, restarts or redeploys the gateway, and explicitly stop/starts each affected sandbox.

Each runtime records the immutable trust generation it was created with. Without reconciliation, ordinary stop/start cannot reliably move an existing sandbox to the new generation: Docker and Podman retain old container metadata, Kubernetes retains the old content-addressed ConfigMap mount, and VM retains the old writable overlay. Replacing only the staged file would conflict with the supervisor's recorded digest and correctly fail closed.

The only existing manual alternative is to delete and recreate each affected sandbox, or create a replacement sandbox and move work to it. Sandbox deletion normally removes its driver-owned mutable filesystem storage—Docker/Podman workspace volumes, the Kubernetes sandbox PVC, or the VM writable overlay. Operators would therefore need to back up and restore required /sandbox contents, use external persistent storage, and re-establish any sandbox-specific state. The reconciliation implemented here avoids that migration by rebuilding stale runtime resources during stop/start while preserving the sandbox record and supported durable workspace data.

This lifecycle work can be split into a follow-up only if destructive replacement plus explicit data migration is an acceptable interim certificate-rotation procedure.

Testing

  • mise run pre-commit passes
  • Workspace formatting and Clippy with warnings denied pass
  • Targeted Rust tests pass across core, sandbox, supervisor-network, Docker, Kubernetes, Podman, VM, server, and gateway packages
  • Helm tests pass: 164 tests across 15 main-chart suites plus 4 workspace-chart tests
  • Unit and process-level tests added/updated
  • Cross-driver E2E coverage added and feature combinations compile
  • Rootless Podman manually verified against a private-PKI GitLab endpoint: TLS failed without the additional CA and succeeded with it while default public trust remained enabled
  • Full mise run test task run as a single validation command
  • Docker, Kubernetes combined/sidecar, and VM E2E runtime lanes (configured for CI; not run locally)

Checklist

  • Follows Conventional Commits
  • Commits are signed off (DCO)
  • Architecture and user-facing configuration documentation updated

@copy-pr-bot

copy-pr-bot Bot commented Sep 11, 2026

Copy link
Copy Markdown

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@jhjaggars

Copy link
Copy Markdown
Contributor Author
 [openshell.supervisor.network]
   additional_ca_cert_paths = [
     "/tmp/openshell-e2e-podman.McovxM/additional-ca/ca.crt",
   ]

@jhjaggars
jhjaggars force-pushed the feat/network-supervisor-additional-ca branch from a65ec41 to 76279f5 Compare September 14, 2026 19:04
@jhjaggars
jhjaggars marked this pull request as ready for review September 14, 2026 19:05
@jhjaggars
jhjaggars force-pushed the feat/network-supervisor-additional-ca branch 8 times, most recently from 1750116 to 24b9186 Compare September 18, 2026 22:36
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
Signed-off-by: Jesse Jaggars <jjaggars@redhat.com>
@jhjaggars
jhjaggars force-pushed the feat/network-supervisor-additional-ca branch from 24b9186 to 46692e8 Compare September 19, 2026 12:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant