Add ability to configure custom init containers #1532

akurinnoy · 2025-10-30T16:18:16Z

What does this PR do?

This PR adds the ability for cluster administrators to configure custom init containers that run in all workspace pods via the DWOC.
So, now administrators can:

Inject arbitrary init containers into all workspaces via config.workspace.initContainers.
Override the built-in init-persistent-home logic by providing a custom container with the same name.

What issues does this PR fix or reference?

https://issues.redhat.com/browse/CRW-9373
https://issues.redhat.com/browse/CRW-9367

Is it tested? How?

Deploy the DevWorkspace Operator with these changes

Test 1: Custom init-persistent-home script execution

Apply a DWOC with a custom init container:

kubectl apply -f - <<EOF
apiVersion: controller.devfile.io/v1alpha1
kind: DevWorkspaceOperatorConfig
metadata:
  name: devworkspace-operator-config
  namespace: $OPERATOR_NAMESPACE
config:
  workspace:
    persistUserHome:
      enabled: true
    initContainers:
      - name: init-persistent-home
        args:
          - |
            echo "Custom home init executed" > /home/user/.custom_init_marker
            mkdir -p /home/user/custom-config
            echo "enterprise-config" > /home/user/custom-config/settings.txt
EOF

Create a test workspace and wait for it to start

kubectl apply -f - <<EOF
apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  name: test1-custom-home
  namespace: $WORKSPACE_NAMESPACE
spec:
  started: true
  template:
    attributes:
      controller.devfile.io/storage-type: per-user
    components:
      - name: tooling
        container:
          image: quay.io/devfile/universal-developer-image:latest
          memoryLimit: 2Gi
          memoryRequest: 256Mi
EOF

Wait for workspace to run:

kubectl wait --for=condition=Ready devworkspace/test1-custom-home -n $WORKSPACE_NAMESPACE --timeout=5m

Verify the init container was injected:

POD_NAME=$(kubectl get pods -n $WORKSPACE_NAMESPACE -l controller.devfile.io/devworkspace_name=test1-custom-home -o jsonpath='{.items[0].metadata.name}')

Verify marker file

kubectl exec -n $WORKSPACE_NAMESPACE $POD_NAME -c tooling -- cat /home/user/.custom_init_marker

Expected output

Custom home init executed

Verify config file

kubectl exec -n $WORKSPACE_NAMESPACE $POD_NAME -c tooling -- cat /home/user/custom-config/settings.txt

Expected output

enterprise-config

Cleanup

kubectl delete devworkspace test1-custom-home -n $WORKSPACE_NAMESPACE

Test 2: Default init-persistent-home behavior

Reset DWOC to default (no custom init)

kubectl apply -f - <<EOF
apiVersion: controller.devfile.io/v1alpha1
kind: DevWorkspaceOperatorConfig
metadata:
  name: devworkspace-operator-config
  namespace: $OPERATOR_NAMESPACE
config:
  workspace:
    persistUserHome:
      enabled: true
EOF

Create workspace with persistent home enabled

kubectl apply -f - <<EOF
apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  name: test2-backward-compat
  namespace: $WORKSPACE_NAMESPACE
spec:
  started: true
  template:
    attributes:
      controller.devfile.io/storage-type: per-user
    components:
      - name: tooling
        container:
          image: quay.io/devfile/universal-developer-image:latest
          memoryLimit: 2Gi
          memoryRequest: 256Mi
EOF

Wait for workspace to run:

kubectl wait --for=condition=Ready devworkspace/test2-backward-compat -n $WORKSPACE_NAMESPACE --timeout=5m

Verify default stow logic still works

kubectl logs -n $WORKSPACE_NAMESPACE -l controller.devfile.io/devworkspace_name=test2-backward-compat -c init-persistent-home

Expected output

Checking for stow command
Running stow command

Cleanup

kubectl delete devworkspace test2-backward-compat -n $WORKSPACE_NAMESPACE

Test 3: Validation of init-persistent-home configuration

Apply DWOC with invalid command

kubectl apply -f - <<EOF
apiVersion: controller.devfile.io/v1alpha1
kind: DevWorkspaceOperatorConfig
metadata:
  name: devworkspace-operator-config
  namespace: $OPERATOR_NAMESPACE
config:
  workspace:
    persistUserHome:
      enabled: true
    initContainers:
      - name: init-persistent-home
        image: busybox:latest
        command: ["/bin/bash"]  # Invalid - must be ["/bin/sh", "-c"]
        args:
          - echo "test"
EOF

Create workspace

kubectl apply -f - <<EOF
apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  name: test3-invalid-command
  namespace: $WORKSPACE_NAMESPACE
spec:
  started: true
  template:
    attributes:
      controller.devfile.io/storage-type: per-user
    components:
      - name: tooling
        container:
          image: quay.io/devfile/universal-developer-image:latest
          memoryLimit: 2Gi
          memoryRequest: 256Mi
EOF

Wait a few seconds for reconciliation

Verify workspace fails with validation error

kubectl get devworkspace test3-invalid-command -n $WORKSPACE_NAMESPACE -o jsonpath='{.status.phase}'

Expected output

Failed

kubectl get devworkspace test3-invalid-command -n $WORKSPACE_NAMESPACE -o jsonpath='{.status.message}'

Expected output

Invalid init-persistent-home container: command must be exactly [/bin/sh, -c]

Cleanup

kubectl delete devworkspace test3-invalid-command -n $WORKSPACE_NAMESPACE

Test 4: Multiple Custom Init Containers

Apply DWOC with multiple init containers

kubectl apply -f - <<EOF
apiVersion: controller.devfile.io/v1alpha1
kind: DevWorkspaceOperatorConfig
metadata:
  name: devworkspace-operator-config
  namespace: $OPERATOR_NAMESPACE
config:
  workspace:
    persistUserHome:
      enabled: true
    initContainers:
      - name: init-persistent-home
        args:
          - |
            echo "Step 1: Home init" > /home/user/init-sequence.log
            date >> /home/user/init-sequence.log
            echo "Home directory initialized" >> /home/user/init-sequence.log
      - name: install-tools
        image: quay.io/devfile/universal-developer-image:latest
        command: ["/bin/sh", "-c"]
        args:
          - |
            echo "Step 2: Installing tools" >> /home/user/init-sequence.log
            date >> /home/user/init-sequence.log
            echo "Tools setup completed" > /home/user/tools-installed.txt
            echo "wget simulation completed" >> /home/user/init-sequence.log
        volumeMounts:
          - name: persistent-home
            mountPath: /home/user/
      - name: config-setup
        image: quay.io/devfile/universal-developer-image:latest
        command: ["/bin/sh", "-c"]
        args:
          - |
            echo "Step 3: Final config" >> /home/user/init-sequence.log
            date >> /home/user/init-sequence.log
            echo "Configuration setup completed" > /home/user/config-complete.txt
        volumeMounts:
          - name: persistent-home
            mountPath: /home/user/
EOF

Create workspace

kubectl apply -f - <<EOF
apiVersion: workspace.devfile.io/v1alpha2
kind: DevWorkspace
metadata:
  name: test4-multiple-init
  namespace: $WORKSPACE_NAMESPACE
spec:
  started: true
  template:
    attributes:
      controller.devfile.io/storage-type: per-user
    components:
      - name: tooling
        container:
          image: quay.io/devfile/universal-developer-image:latest
          memoryLimit: 2Gi
          memoryRequest: 256Mi
EOF

Wait for workspace to run:

kubectl wait --for=condition=Ready devworkspace/test4-multiple-init -n $WORKSPACE_NAMESPACE --timeout=5m

Verify all init containers were injected and ran in order

POD_NAME=$(kubectl get pods -n $WORKSPACE_NAMESPACE -l controller.devfile.io/devworkspace_name=test4-multiple-init -o jsonpath='{.items[0].metadata.name}')

Verify all 3 init containers are present

kubectl get pod -n $WORKSPACE_NAMESPACE $POD_NAME -o jsonpath='{.spec.initContainers[*].name}'

Expected output

init-persistent-home install-tools config-setup

Verify the sequence log shows all 3 steps

kubectl exec -n $WORKSPACE_NAMESPACE $POD_NAME -c tooling -- cat /home/user/init-sequence.log

Expected output

Step 1: Home init
[timestamp]
Home directory initialized
Step 2: Installing tools
[timestamp]
wget simulation completed
Step 3: Final config
[timestamp]

Verify marker files from each init container

kubectl exec -n $WORKSPACE_NAMESPACE $POD_NAME -c tooling -- cat /home/user/tools-installed.txt

Expected output

Tools setup completed

kubectl exec -n $WORKSPACE_NAMESPACE $POD_NAME -c tooling -- cat /home/user/config-complete.txt

Expected output

Configuration setup completed

Cleanup

kubectl delete devworkspace test4-multiple-init -n $WORKSPACE_NAMESPACE

PR Checklist

E2E tests pass (when PR is ready, comment /test v8-devworkspace-operator-e2e, v8-che-happy-path to trigger)
- v8-devworkspace-operator-e2e: DevWorkspace e2e test
- v8-che-happy-path: Happy path for verification integration with Che

Signed-off-by: Oleksii Kurinnyi <[email protected]>

…erate_all Signed-off-by: Oleksii Kurinnyi <[email protected]>

openshift-ci · 2025-10-30T16:18:20Z

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

openshift-ci · 2025-10-30T16:18:24Z

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: akurinnoy
Once this PR has been reviewed and has the lgtm label, please assign dkwon17 for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

apis/controller/v1alpha1/devworkspaceoperatorconfig_types.go

controllers/workspace/devworkspace_controller.go

test/resources/disabled-init-test-workspace.yaml

pkg/library/home/util.go

controllers/workspace/devworkspace_controller.go

rohanKanojia · 2025-10-31T11:27:23Z

I tested with the steps provided in the PR description and can confirm it works as expected ✔️

Signed-off-by: Oleksii Kurinnyi <[email protected]>

Co-authored-by: Rohan Kumar <[email protected]> Signed-off-by: Oleksii Kurinnyi <[email protected]>

Signed-off-by: Oleksii Kurinnyi <[email protected]>

tolusha · 2025-10-31T11:42:15Z

apis/controller/v1alpha1/devworkspaceoperatorconfig_types.go

 	// If the feature is disabled, setting this field may cause an endless workspace start loop.
 	// +kubebuilder:validation:Optional
 	HostUsers *bool `json:"hostUsers,omitempty"`
+	// InitContainers defines a list of Kubernetes init containers that are automatically injected into all workspace pods.


Suggested change

// InitContainers defines a list of Kubernetes init containers that are automatically injected into all workspace pods.

// InitContainers defines a list of Kubernetes init containers that are automatically added into all workspace pods.

tolusha · 2025-11-03T08:45:49Z

apis/controller/v1alpha1/devworkspaceoperatorconfig_types.go

 	HostUsers *bool `json:"hostUsers,omitempty"`
+	// InitContainers defines a list of Kubernetes init containers that are automatically injected into all workspace pods.
+	// Typical uses: injecting organization tools/configs, initializing persistent home, etc.
+	// Note: Only trusted administrators should be allowed to edit the DevWorkspaceOperatorConfig.


Suggested change

// Note: Only trusted administrators should be allowed to edit the DevWorkspaceOperatorConfig.

// Note: Only administrators should be allowed to edit the DevWorkspaceOperatorConfig.

akurinnoy · 2025-11-03T11:16:46Z

/retest

openshift-ci · 2025-11-03T13:40:58Z

@akurinnoy: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name	Commit	Details	Required	Rerun command
ci/prow/v14-devworkspace-operator-e2e	`81727ce`	link	true	`/test v14-devworkspace-operator-e2e`

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

tolusha · 2025-11-04T11:16:26Z

controllers/workspace/devworkspace_controller.go

+	if len(c.Command) != 2 || c.Command[0] != "/bin/sh" || c.Command[1] != "-c" {
+		return fmt.Errorf("command must be exactly [/bin/sh, -c] for %s", constants.HomeInitComponentName)
+	}
+
+	if len(c.Args) != 1 {
+		return fmt.Errorf("args must contain exactly one script string for %s", constants.HomeInitComponentName)
+	}


Could you explain why do we need to limit command and args?

tolusha · 2025-11-04T11:17:46Z

controllers/workspace/devworkspace_controller.go

+
+// validateNoAdvancedFields validates that the init-persistent-home container
+// does not use advanced Kubernetes container fields that could make behavior unpredictable.
+func validateNoAdvancedFields(c corev1.Container) error {


Could you share the documentation where those limitations come from?

tolusha · 2025-11-04T11:19:54Z

controllers/workspace/devworkspace_controller.go

+		disableHomeInit := workspace.Config.Workspace.PersistUserHome.DisableInitContainer != nil &&
+			*workspace.Config.Workspace.PersistUserHome.DisableInitContainer


Please use pointer package

tolusha · 2025-11-04T11:26:21Z

controllers/workspace/devworkspace_controller.go

+				if !home.PersistUserHomeEnabled(workspace) {
+					continue
+				}


I think we shouldn't take this clause into account
cc @dkwon17

tolusha · 2025-11-04T11:39:40Z

controllers/workspace/devworkspace_controller.go

+	c.VolumeMounts = []corev1.VolumeMount{{
+		Name:      constants.HomeVolumeName,
+		MountPath: constants.HomeUserDirectory,
+	}}


Could you move this to some utility class since it has been used twice so far.

tolusha · 2025-11-04T11:40:15Z

controllers/workspace/devworkspace_controller.go

+				}
+				// Skip if init container is explicitly disabled
+				if disableHomeInit {
+					continue


Let's log some info here

tolusha · 2025-11-04T11:41:37Z

controllers/workspace/devworkspace_controller.go

+		disableHomeInit := workspace.Config.Workspace.PersistUserHome.DisableInitContainer != nil &&
+			*workspace.Config.Workspace.PersistUserHome.DisableInitContainer
+
+		for _, c := range workspace.Config.Workspace.InitContainers {


For a better readability?

Suggested change

for _, c := range workspace.Config.Workspace.InitContainers {

for _, container := range workspace.Config.Workspace.InitContainers {

tolusha · 2025-11-04T11:42:16Z

controllers/workspace/devworkspace_controller.go

+					continue
+				}
+				// Apply defaults and validation for init-persistent-home
+				validated, err := defaultAndValidateHomeInitContainer(c, workspace)


Not sure, if we can do in this way, but can you check?

Suggested change

validated, err := defaultAndValidateHomeInitContainer(c, workspace)

container, err = defaultAndValidateHomeInitContainer(c, workspace)

akurinnoy added 2 commits October 30, 2025 15:27

feat: configure init containers

9c87c0e

Signed-off-by: Oleksii Kurinnyi <[email protected]>

chore: run make update_devworkspace_api update_devworkspace_crds gen…

c6b803d

…erate_all Signed-off-by: Oleksii Kurinnyi <[email protected]>

openshift-ci bot added the do-not-merge/work-in-progress label Oct 30, 2025

rohanKanojia reviewed Oct 31, 2025

View reviewed changes

apis/controller/v1alpha1/devworkspaceoperatorconfig_types.go Outdated Show resolved Hide resolved

rohanKanojia reviewed Oct 31, 2025

View reviewed changes

controllers/workspace/devworkspace_controller.go Outdated Show resolved Hide resolved

rohanKanojia reviewed Oct 31, 2025

View reviewed changes

test/resources/disabled-init-test-workspace.yaml Show resolved Hide resolved

rohanKanojia reviewed Oct 31, 2025

View reviewed changes

pkg/library/home/util.go Show resolved Hide resolved

rohanKanojia reviewed Oct 31, 2025

View reviewed changes

controllers/workspace/devworkspace_controller.go Outdated Show resolved Hide resolved

akurinnoy and others added 4 commits October 31, 2025 14:42

test: unit and e2e tests for init containers

06a1e28

Signed-off-by: Oleksii Kurinnyi <[email protected]>

Update apis/controller/v1alpha1/devworkspaceoperatorconfig_types.go

01705b1

Co-authored-by: Rohan Kumar <[email protected]> Signed-off-by: Oleksii Kurinnyi <[email protected]>

Apply suggestion from @rohanKanojia

872461b

Co-authored-by: Rohan Kumar <[email protected]> Signed-off-by: Oleksii Kurinnyi <[email protected]>

fixup! feat: configure init containers

1ff60c7

Signed-off-by: Oleksii Kurinnyi <[email protected]>

akurinnoy force-pushed the config-init-containers branch from 2e3e20f to 1ff60c7 Compare October 31, 2025 12:45

akurinnoy added 4 commits October 31, 2025 15:00

fixup! fixup! feat: configure init containers

0cb9cf0

Signed-off-by: Oleksii Kurinnyi <[email protected]>

fixup! fixup! fixup! feat: configure init containers

462783b

Signed-off-by: Oleksii Kurinnyi <[email protected]>

fixup! fixup! fixup! fixup! feat: configure init containers

6f0b072

Signed-off-by: Oleksii Kurinnyi <[email protected]>

fixup! fixup! fixup! fixup! fixup! feat: configure init containers

81727ce

Signed-off-by: Oleksii Kurinnyi <[email protected]>

akurinnoy marked this pull request as ready for review October 31, 2025 15:03

akurinnoy requested review from dkwon17 and ibuziuk as code owners October 31, 2025 15:03

openshift-ci bot removed the do-not-merge/work-in-progress label Oct 31, 2025

tolusha reviewed Nov 3, 2025

View reviewed changes

tolusha reviewed Nov 4, 2025

View reviewed changes

	// InitContainers defines a list of Kubernetes init containers that are automatically injected into all workspace pods.
	// InitContainers defines a list of Kubernetes init containers that are automatically added into all workspace pods.

	// Note: Only trusted administrators should be allowed to edit the DevWorkspaceOperatorConfig.
	// Note: Only administrators should be allowed to edit the DevWorkspaceOperatorConfig.

		disableHomeInit := workspace.Config.Workspace.PersistUserHome.DisableInitContainer != nil &&
		*workspace.Config.Workspace.PersistUserHome.DisableInitContainer

	for _, c := range workspace.Config.Workspace.InitContainers {
	for _, container := range workspace.Config.Workspace.InitContainers {

	validated, err := defaultAndValidateHomeInitContainer(c, workspace)
	container, err = defaultAndValidateHomeInitContainer(c, workspace)

Add ability to configure custom init containers #1532

Are you sure you want to change the base?

Add ability to configure custom init containers #1532

Uh oh!

Conversation

akurinnoy commented Oct 30, 2025

What does this PR do?

What issues does this PR fix or reference?

Is it tested? How?

PR Checklist

Uh oh!

openshift-ci bot commented Oct 30, 2025

Uh oh!

openshift-ci bot commented Oct 30, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

rohanKanojia commented Oct 31, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

akurinnoy commented Nov 3, 2025

Uh oh!

openshift-ci bot commented Nov 3, 2025

Uh oh!

tolusha Nov 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

tolusha Nov 4, 2025 •

edited

Loading