Skip to content
57 changes: 56 additions & 1 deletion tests/mdr-operator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -75,6 +75,61 @@ container or pod level). Only checks the `manager` container.
- **Standalone**: `ginkgo --label-filter="mdr" --focus="runs as non-root" ./tests/mdr-operator/...`
- **Pass criteria**: Pod runAsNonRoot=true; expected manager container exists; manager container runAsUser != 0; allowPrivilegeEscalation=false; readOnlyRootFilesystem=true; capabilities.drop=[ALL]; seccomp profile RuntimeDefault

## Condition Tests

### 5. Verify MDR Conditions With NHC Timed-Out Annotation ([OCP-65763](https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-65763))

Creates an MDR CR with the `remediation.medik8s.io/nhc-timed-out` annotation.
Verifies that the Processing and Succeeded conditions have reason
`RemediationStoppedByNHC`, and that the controller log contains the
"NHC time out annotation found, stopping remediation" message.

- **Operators**: MDR v0.7.0+
- **Cluster**: Any topology (MNO or SNO)
- **Environment**: Connected or disconnected
- **Standalone**: `ginkgo --label-filter="mdr" --focus="nhc-timed-out annotation" ./tests/mdr-operator/...`
- **Pass criteria**: Processing reason=RemediationStoppedByNHC; Succeeded reason=RemediationStoppedByNHC; controller log contains remediation-stopped message; MDR controller pod running after test

### 6. Verify MDR Conditions With Non-Existent Node ([OCP-66137](https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-66137))

Creates an MDR CR named `non-existent-node`. Verifies that the Processing and
Succeeded conditions have status=False and reason
`RemediationCannotStartNodeNotFound`. Guards against accidental real fencing
by asserting the node name does not exist in the cluster before creating the CR.

- **Operators**: MDR v0.7.0+
- **Cluster**: Any topology (MNO or SNO)
- **Environment**: Connected or disconnected
- **Standalone**: `ginkgo --label-filter="mdr" --focus="non-existent node" ./tests/mdr-operator/...`
- **Pass criteria**: Node name confirmed absent from cluster; Processing status=False reason=RemediationCannotStartNodeNotFound; Succeeded status=False reason=RemediationCannotStartNodeNotFound; MDR controller pod running after test

### 7. Verify MDR Conditions With Control-Plane Node ([OCP-66351](https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-66351))

Creates an MDR CR named after a random control-plane node. On baremetal,
control-plane Machines have no controller owner and MDR sets
RemediationCannotStartNoControllerOwner. Skipped on cloud platforms where
CPMS gives control-plane Machines a controller owner, causing MDR to start
Machine deletion (destructive).

- **Operators**: MDR v0.7.0+
- **Cluster**: Any topology (MNO or SNO), baremetal only
- **Environment**: Connected or disconnected
- **Standalone**: `ginkgo --label-filter="mdr && platform:baremetal" --focus="control-plane node name" ./tests/mdr-operator/...`
- **Pass criteria**: Processing status=False reason=RemediationCannotStartNoControllerOwner; Succeeded status=False reason=RemediationCannotStartNoControllerOwner; MDR controller pod running after test

### 8. Verify PermanentNodeDeletionExpected Condition With Control-Plane Node ([OCP-66317](https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-66317))

Creates an MDR CR for a control-plane node and verifies the
`PermanentNodeDeletionExpected` condition on baremetal: status=False,
reason=KeepsNodeName, message confirms node name is preserved.
Skipped on cloud platforms (same reason as test 7).

- **Operators**: MDR v0.7.0+
- **Cluster**: Any topology (MNO or SNO), baremetal only
- **Environment**: Connected or disconnected
- **Standalone**: `ginkgo --label-filter="mdr && platform:baremetal" --focus="PermanentNodeDeletionExpected" ./tests/mdr-operator/...`
- **Pass criteria**: PermanentNodeDeletionExpected status=False reason=KeepsNodeName message="...NOT expected to have a new name"; MDR controller pod running after test

## Destructive Tests -- NHC-Triggered Remediation

Tests that stop kubelet on a worker node, let NHC detect the unhealthy node
Expand All @@ -88,7 +143,7 @@ provider provisions a new VM. The node is re-created (new creation timestamp).
- At least 2 Ready worker nodes (target + spare for cluster schedulability)
- `KUBECONFIG` set with cluster-admin access

### 5. MDR Remediation with Condition Transitions ([OCP-66138](https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-66138))
### 9. MDR Remediation with Condition Transitions ([OCP-66138](https://polarion.engineering.redhat.com/polarion/#/project/OSE/workitem?id=OCP-66138))

Stops kubelet on a worker node. NHC detects the unhealthy node and creates
an MDR CR via the MDR template. Verifies the MDR CR status conditions
Expand Down
56 changes: 55 additions & 1 deletion tests/mdr-operator/internal/mdrparams/const.go
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ const (
// CSVNamePattern is the substring used to match the MDR operator ClusterServiceVersion by name.
CSVNamePattern = "machine-deletion-remediation"

// --- Remediation test constants (RHWA-1248) ---
// Remediation test constants (RHWA-1248)

// NHCCRDName is the CRD name for NodeHealthCheck, used to detect if NHC is installed.
NHCCRDName = "nodehealthchecks.remediation.medik8s.io"
Expand Down Expand Up @@ -67,4 +67,58 @@ const (
// ConditionReasonRemediationStarted is the reason set on Processing and Succeeded
// conditions when remediation begins.
ConditionReasonRemediationStarted = "RemediationStarted"

// Condition test constants (RHWA-1249)

// ConditionReasonStoppedByNHC is the reason set when NHC timed-out annotation is present.
ConditionReasonStoppedByNHC = "RemediationStoppedByNHC"

// ConditionReasonNodeNotFound is the reason set when the target node does not exist.
ConditionReasonNodeNotFound = "RemediationCannotStartNodeNotFound"

// ConditionReasonNoControllerOwner is the reason set when the target node has no controller owner.
ConditionReasonNoControllerOwner = "RemediationCannotStartNoControllerOwner"

// PermanentNodeDeletionExpectedConditionType is the condition indicating node name persistence.
PermanentNodeDeletionExpectedConditionType = "PermanentNodeDeletionExpected"

// ConditionReasonKeepsNodeName is the reason for baremetal clusters where the node name is preserved.
ConditionReasonKeepsNodeName = "MachineDeletionOnBareMetalProviderKeepsNodeName"

// ConditionMessageKeepsNodeName is the expected message for baremetal PermanentNodeDeletionExpected.
ConditionMessageKeepsNodeName = "Machine will be deleted as part of remediation. " +
"This is a BareMetal cluster provider: the new node is NOT expected to have a new name"

// ConditionStatusFalse is the status value "False" for condition checks.
ConditionStatusFalse = "False"

// NHCTimedOutAnnotationKey is the annotation that signals NHC timed out.
NHCTimedOutAnnotationKey = "remediation.medik8s.io/nhc-timed-out"

// NHCTimedOutAnnotationValue is an RFC3339 timestamp for the annotation.
// The MDR controller checks only for key presence, not value format.
NHCTimedOutAnnotationValue = "2024-01-01T00:00:00Z"

// MDRRemStoppedLogMsg is the log message emitted when MDR stops due to NHC timeout.
MDRRemStoppedLogMsg = "NHC time out annotation found, stopping remediation"

// MDRConditionTestName is the MDR CR name for the NHC timed-out condition test.
MDRConditionTestName = "mdr-test-timeout-annotation"

// MDRNonExistentNodeTestName is the MDR CR name for the non-existent node test.
MDRNonExistentNodeTestName = "non-existent-node"

// ControllerLogWindow is the time window for searching controller logs.
ControllerLogWindow = 5 * time.Minute

// Node role labels (mirrors medik8s/common/pkg/labels)

// MasterRoleLabel is the old role label for control-plane nodes.
MasterRoleLabel = "node-role.kubernetes.io/master"

// ControlPlaneRoleLabel is the new role label for control-plane nodes.
ControlPlaneRoleLabel = "node-role.kubernetes.io/control-plane"

// WorkerRoleLabel is the role label for worker nodes.
WorkerRoleLabel = "node-role.kubernetes.io/worker"
)
Loading