Skip to content

Commit 52cb8ec

Browse files
authored
fix(kubernetes): remove NetworkPolicy acknowledgement (#3677)
Signed-off-by: Drew Newberry <anewberry@nvidia.com>
1 parent 4cd1351 commit 52cb8ec

29 files changed

Lines changed: 64 additions & 116 deletions

‎.agents/skills/test-release-canary/SKILL.md‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,7 +109,6 @@ helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
109109
--namespace openshell --create-namespace \
110110
--set server.disableTls=true \
111111
--set server.telemetryEnabled=false \
112-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
113112
--wait --timeout 5m
114113
115114
kubectl wait --namespace openshell \

‎.github/workflows/release-canary.yml‎

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -359,7 +359,6 @@ jobs:
359359
--set server.disableTls=true \
360360
--set server.auth.allowUnauthenticatedUsers=true \
361361
--set "server.telemetryEnabled=${OPENSHELL_TELEMETRY_ENABLED}" \
362-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
363362
--wait --timeout 5m
364363
365364
- name: Verify gateway pod is Ready

‎README.md‎

Lines changed: 4 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -41,12 +41,14 @@ The installer installs the latest stable release by default. See [Prerelease and
4141
**Kubernetes installation:**
4242

4343
> **Experimental** — the Kubernetes deployment path is under active development. Expect rough edges and breaking changes.
44+
> **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for
45+
> ingress and egress in every sandbox namespace. OpenShell creates the policies,
46+
> but Kubernetes does not verify that the CNI applies them.
4447
4548
Deploy the OpenShell gateway into a Kubernetes cluster from the OCI chart published to GHCR:
4649

4750
```bash
48-
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart \
49-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true
51+
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart
5052
```
5153

5254
See [`deploy/helm/openshell/README.md`](deploy/helm/openshell/README.md) for available versions, dev tag conventions, and configuration.

‎architecture/compute-runtimes.md‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -301,7 +301,7 @@ delete, reconciliation removes the row; otherwise it can remain `Deleting`.
301301
|---|---|---|---|
302302
| Docker | Local development with Docker available. | Capability-free workload container. | Uses `network_mode=none`; a separate capability-free supervisor container mediates egress and access over a private daemon-local Unix socket volume. |
303303
| Podman | Existing rootless driver. | Container. | Not converted by this isolation stack. |
304-
| Kubernetes | Cluster deployment through Helm. | Capability-free sandbox Pod. | Uses one namespace-wide empty-egress workload NetworkPolicy and a separate capability-free supervisor Pod over mutually authenticated TLS. It requires an enforcing CNI and trusted sandbox namespace. |
304+
| Kubernetes | Cluster deployment through Helm. | Capability-free sandbox Pod. | Always creates a namespace-wide empty-egress workload NetworkPolicy and a separate capability-free supervisor Pod over mutually authenticated TLS. It requires an enforcing CNI and trusted sandbox namespace; the Kubernetes API does not attest policy enforcement. |
305305
| VM | Experimental microVM isolation. | Per-sandbox libkrun or QEMU VM. | The NIC-less guest runs `openshell-sandbox` as PID 1; host `openshell-supervisor` owns gateway networking and reaches the guest over vsock. |
306306
| Extension | Out-of-tree drivers operated alongside the gateway. | Whatever boundary the driver implements. | Selected by a custom `compute_drivers = ["<name>"]` entry with `[openshell.drivers.<name>].socket_path`, or at launch time by pairing `--drivers <name>` with `--compute-driver-socket=<path>`. A launch-time endpoint may use a canonical built-in name to preserve its driver-config key while replacing in-process construction. The gateway connects to an operator-provisioned UDS, snapshots `GetCapabilities`, and dispatches all sandbox lifecycle calls through `compute_driver.proto`. The driver process and socket lifecycle are operator-owned; the gateway does not spawn, supervise, or remove unmanaged extension drivers. The trust boundary is the socket's filesystem permissions: the operator must ensure only the gateway uid can read/write it. |
307307

‎crates/openshell-driver-kubernetes/README.md‎

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -81,9 +81,9 @@ and permits OpenShell supervisor Pods to reach the sandbox TLS port. The
8181
authenticated Sandbox Protocol binds each connection to the exact sandbox and
8282
supervisor Pod identities. Supervisors have normal egress for gateway, DNS,
8383
and policy-approved upstream connections unless an operator policy restricts
84-
them. Set
85-
`sandbox_runtime.network_policy_enforced = true` only after verifying that the cluster
86-
CNI enforces ingress and egress `NetworkPolicy` for sandbox namespaces.
84+
them. The cluster CNI must enforce ingress and egress `NetworkPolicy` for every
85+
sandbox namespace. Kubernetes accepts policy objects without confirming
86+
enforcement, so operators must verify CNI support before running sandboxes.
8787

8888
Each sandbox generation uses two immutable bootstrap Secrets. A trusted init
8989
container stages the sandbox bootstrap into memory, and the sandbox removes it

‎crates/openshell-driver-kubernetes/src/config.rs‎

Lines changed: 0 additions & 23 deletions
Original file line numberDiff line numberDiff line change
@@ -78,30 +78,20 @@ pub const DEFAULT_WORKSPACE_STORAGE_SIZE: &str = "2Gi";
7878
#[derive(Debug, Clone, Serialize, Deserialize)]
7979
#[serde(default, deny_unknown_fields)]
8080
pub struct KubernetesSandboxRuntimeConfig {
81-
/// Explicit operator assertion that the cluster CNI enforces
82-
/// `networking.k8s.io/v1` `NetworkPolicy` for the sandbox namespaces.
83-
pub network_policy_enforced: bool,
8481
/// TCP port exposed by the workload boundary to its paired control pod.
8582
pub boundary_port: u16,
8683
}
8784

8885
impl Default for KubernetesSandboxRuntimeConfig {
8986
fn default() -> Self {
9087
Self {
91-
network_policy_enforced: false,
9288
boundary_port: 5500,
9389
}
9490
}
9591
}
9692

9793
impl KubernetesSandboxRuntimeConfig {
9894
pub fn validate(&self) -> Result<(), String> {
99-
if !self.network_policy_enforced {
100-
return Err(
101-
"sandbox_runtime.network_policy_enforced must be true after the operator has verified CNI NetworkPolicy enforcement"
102-
.to_string(),
103-
);
104-
}
10595
if self.boundary_port < 1024 {
10696
return Err("sandbox_runtime.boundary_port must be at least 1024".to_string());
10797
}
@@ -887,19 +877,6 @@ mod tests {
887877
assert!(cfg.workspace_storage_class.is_empty());
888878
}
889879

890-
#[test]
891-
fn sandbox_runtime_requires_network_policy_enforcement_acknowledgement() {
892-
let mut cfg = KubernetesComputeConfig::default();
893-
assert!(
894-
cfg.validate_proxy_uid()
895-
.unwrap_err()
896-
.contains("network_policy_enforced")
897-
);
898-
899-
cfg.sandbox_runtime.network_policy_enforced = true;
900-
cfg.validate_proxy_uid().unwrap();
901-
}
902-
903880
#[test]
904881
fn serde_rejects_sidecar_binary_identity_field() {
905882
let json = serde_json::json!({

‎crates/openshell-driver-kubernetes/src/main.rs‎

Lines changed: 0 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -128,13 +128,6 @@ struct Args {
128128
#[arg(long, env = "OPENSHELL_SUPERVISOR_IMAGE_PULL_POLICY")]
129129
supervisor_image_pull_policy: Option<KubernetesImagePullPolicy>,
130130

131-
#[arg(
132-
long,
133-
env = "OPENSHELL_K8S_SANDBOX_RUNTIME_NETWORK_POLICY_ENFORCED",
134-
default_value_t = false
135-
)]
136-
sandbox_runtime_network_policy_enforced: bool,
137-
138131
#[arg(
139132
long,
140133
env = "OPENSHELL_K8S_SANDBOX_RUNTIME_BOUNDARY_PORT",
@@ -270,7 +263,6 @@ async fn main() -> Result<()> {
270263
.unwrap_or_else(openshell_core::config::default_supervisor_image),
271264
supervisor_image_pull_policy: args.supervisor_image_pull_policy,
272265
sandbox_runtime: KubernetesSandboxRuntimeConfig {
273-
network_policy_enforced: args.sandbox_runtime_network_policy_enforced,
274266
boundary_port: args.sandbox_runtime_boundary_port,
275267
},
276268
https_proxy: args.https_proxy,

‎deploy/helm/openshell/README.md‎

Lines changed: 7 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,12 @@ multiple pre-provisioned workspace namespaces.
1919

2020
## Prerequisites
2121

22+
> **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for
23+
> ingress and egress in every sandbox namespace. OpenShell creates the policies,
24+
> but Kubernetes accepts them even if no CNI enforces them. Without enforcement,
25+
> sandbox workloads may connect directly and bypass supervisor network policy.
26+
> Verify CNI support before installing OpenShell.
27+
2228
The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with:
2329

2430
```shell
@@ -35,8 +41,7 @@ where Helm cannot discover cluster APIs.
3541
## Install on Kubernetes
3642

3743
```shell
38-
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
39-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true
44+
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version>
4045
```
4146

4247
## Install on OpenShift
@@ -49,7 +54,6 @@ oc create ns openshell
4954

5055
# Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway
5156
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> -n openshell \
52-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
5357
--set server.disableTls=true \
5458
--set podSecurityContext.fsGroup=null \
5559
--set securityContext.runAsUser=null
@@ -110,7 +114,6 @@ Then install the chart pointing at that Secret:
110114
```bash
111115
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
112116
-n openshell \
113-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
114117
--set workload.kind=deployment \
115118
--set server.externalDbSecret=my-pg-credentials
116119
```
@@ -352,7 +355,6 @@ discovery endpoint or its TLS CA.
352355
| supervisor.image.repository | string | `"openshell/supervisor"` | Supervisor image repository. |
353356
| supervisor.image.tag | string | `""` | Supervisor image tag. Defaults to the chart appVersion when empty. |
354357
| supervisor.sandboxRuntime.boundaryPort | int | `5500` | Workload boundary TLS listener port. |
355-
| supervisor.sandboxRuntime.networkPolicyEnforced | bool | `false` | Required operator acknowledgement that the cluster CNI enforces NetworkPolicy. |
356358
| tolerations | list | `[]` | Tolerations for the gateway pod. |
357359
| upstreamProxy | object | `{"authAllowInsecure":false,"authSecret":{"key":"","name":""},"caBundle":{"configMapName":"","key":"ca.crt"},"connectByHostname":false,"noProxy":"","url":""}` | Operator-owned corporate forward proxy for policy-approved TLS egress from Kubernetes sandboxes. The workload cannot select or override it. |
358360
| upstreamProxy.authAllowInsecure | bool | `false` | Required when authSecret is configured because Basic auth to an HTTP proxy is cleartext. |

‎deploy/helm/openshell/README.md.gotmpl‎

Lines changed: 7 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -19,6 +19,12 @@ multiple pre-provisioned workspace namespaces.
1919

2020
## Prerequisites
2121

22+
> **Required:** Your cluster CNI MUST enforce Kubernetes `NetworkPolicy` for
23+
> ingress and egress in every sandbox namespace. OpenShell creates the policies,
24+
> but Kubernetes accepts them even if no CNI enforces them. Without enforcement,
25+
> sandbox workloads may connect directly and bypass supervisor network policy.
26+
> Verify CNI support before installing OpenShell.
27+
2228
The Kubernetes Agent Sandbox CRDs and controller must be installed on the cluster before deploying OpenShell. Install them with:
2329

2430
```shell
@@ -35,8 +41,7 @@ where Helm cannot discover cluster APIs.
3541
## Install on Kubernetes
3642

3743
```shell
38-
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
39-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true
44+
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version>
4045
```
4146

4247
## Install on OpenShift
@@ -49,7 +54,6 @@ oc create ns openshell
4954
5055
# Deploy openshell with overrides to allow SCC assignment of fsGroup and runAsUser for the gateway
5156
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> -n openshell \
52-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
5357
--set server.disableTls=true \
5458
--set podSecurityContext.fsGroup=null \
5559
--set securityContext.runAsUser=null
@@ -110,7 +114,6 @@ Then install the chart pointing at that Secret:
110114
```bash
111115
helm install openshell oci://ghcr.io/nvidia/openshell/helm-chart --version <version> \
112116
-n openshell \
113-
--set supervisor.sandboxRuntime.networkPolicyEnforced=true \
114117
--set workload.kind=deployment \
115118
--set server.externalDbSecret=my-pg-credentials
116119
```

‎deploy/helm/openshell/ci/values-keycloak.yaml‎

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,8 +8,7 @@
88
#
99
# Then layer this file on top of values.yaml when deploying:
1010
# helm upgrade --install openshell . \
11-
# -f values.yaml -f ci/values-skaffold.yaml -f ci/values-keycloak.yaml \
12-
# --set supervisor.sandboxRuntime.networkPolicyEnforced=true
11+
# -f values.yaml -f ci/values-skaffold.yaml -f ci/values-keycloak.yaml
1312
#
1413
# Or add this file to skaffold.yaml valuesFiles for iterative dev.
1514
#

0 commit comments

Comments
 (0)