Skip to content

OCPBUGS-90560: resolve boot disk image at reconcile time - #181

Open
nader-ziada wants to merge 3 commits into
openshift:mainfrom
nader-ziada:resolve-boot-image-in-reconciler
Open

OCPBUGS-90560: resolve boot disk image at reconcile time#181
nader-ziada wants to merge 3 commits into
openshift:mainfrom
nader-ziada:resolve-boot-image-in-reconciler

Conversation

@nader-ziada

@nader-ziada nader-ziada commented Jul 27, 2026

Copy link
Copy Markdown

When a Machine's boot disk has no image specified, resolve it dynamically by reading the coreos-bootimages ConfigMap from the MCO namespace and parsing the stream metadata for the correct architecture-specific RHCOS image. Falls back to hardcoded defaults when the ConfigMap is unavailable.

Architecture is determined via the GCP MachineTypes API, with prefix-based detection as a secondary fallback.

Summary by CodeRabbit

  • New Features

    • Automatically resolves missing GCP boot disk images based on machine architecture.
    • Supports architecture-specific x86_64 and ARM64 boot images, including newer ARM64 machine types.
    • Selects images from configured OS image streams, including legacy configuration.
    • Uses boot image metadata with reliable default fallbacks.
  • Bug Fixes

    • Machines with unspecified boot disk images can now be created successfully.
    • Added fallback handling when machine architecture information is unavailable.
    • Improved error handling when boot image resolution fails.

When a Machine's boot disk has no image specified, resolve it dynamically
by reading the coreos-bootimages ConfigMap from the MCO namespace and
parsing the stream metadata for the correct architecture-specific RHCOS
image. Falls back to hardcoded defaults when the ConfigMap is unavailable.

Architecture is determined via the GCP MachineTypes API, with prefix-based
detection as a secondary fallback.

Signed-off-by: Nader Ziada <nziada@redhat.com>
@openshift-ci-robot openshift-ci-robot added jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type. jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. labels Jul 27, 2026
@openshift-ci-robot

Copy link
Copy Markdown
Contributor

@nader-ziada: This pull request references Jira Issue OCPBUGS-90560, which is invalid:

  • expected the bug to target the "5.0.0" version, but no target version was set

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

Details

In response to this:

When a Machine's boot disk has no image specified, resolve it dynamically by reading the coreos-bootimages ConfigMap from the MCO namespace and parsing the stream metadata for the correct architecture-specific RHCOS image. Falls back to hardcoded defaults when the ConfigMap is unavailable.

Architecture is determined via the GCP MachineTypes API, with prefix-based detection as a secondary fallback.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

@coderabbitai

coderabbitai Bot commented Jul 27, 2026

Copy link
Copy Markdown

Note

Reviews paused

It looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the reviews.auto_review.auto_pause_after_reviewed_commits setting.

Use the following commands to manage reviews:

  • @coderabbitai resume to resume automatic reviews.
  • @coderabbitai review to trigger a single review.

Use the checkboxes below for quick actions:

  • ▶️ Resume reviews
  • 🔍 Trigger review

Walkthrough

The GCP machine actuator resolves empty boot disk images from machine architecture and CoreOS boot image metadata. Machine creation applies this resolution before UEFI checks. Tests cover stream selection, architecture mapping, fallbacks, and image formatting.

Changes

GCP boot image resolution

Layer / File(s) Summary
Stream-aware image resolution
go.mod, pkg/cloud/gcp/actuators/machine/boot_image.go, pkg/cloud/gcp/actuators/machine/reconciler.go
Adds the stream metadata dependency and resolves the active OS image stream. The resolver reads coreos-bootimages, maps machine architectures, formats GCP image references, and applies architecture-specific fallbacks.
Machine creation integration
pkg/cloud/gcp/actuators/machine/reconciler.go, cmd/manager/main.go
Machine creation resolves empty boot disk images before UEFI checks. The namespace-scoped cache also watches openshift-machine-config-operator.
Architecture mapping and validation
pkg/cloud/gcp/actuators/util/*, pkg/cloud/gcp/actuators/machine/*_test.go
Tests cover stream selection, fallback behavior, architecture lookup, image formatting, and machine creation. The a4x prefix maps to ARM64.

Estimated code review effort: 3 (Moderate) | ~20 minutes

Sequence Diagram(s)

sequenceDiagram
  participant MachineReconciler
  participant GCPMachineTypes
  participant OSImageStream
  participant BootImagesConfigMap
  MachineReconciler->>GCPMachineTypes: Resolve machine architecture
  MachineReconciler->>OSImageStream: Read spec.defaultStream
  OSImageStream-->>MachineReconciler: Return active stream
  MachineReconciler->>BootImagesConfigMap: Read stream metadata
  BootImagesConfigMap-->>MachineReconciler: Return architecture-specific GCP image
Loading

Suggested reviewers: nrb, theobarberbany

🚥 Pre-merge checks | ✅ 15
✅ Passed checks (15 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely describes the primary change: resolving the boot disk image during reconciliation.
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed Added tests use fixed Go test and subtest names; no changed Ginkgo title contains generated IDs, timestamps, node or namespace names, IPs, or other run-varying data.
Test Structure And Quality ✅ Passed Changed tests use standard Go testing, not Ginkgo It blocks; they use in-memory fake clients, perform no cluster waits, and include diagnostic t.Fatalf/t.Errorf messages.
Microshift Test Compatibility ✅ Passed No new Ginkgo e2e tests were added; all changed tests use standard Go Test functions and therefore do not trigger MicroShift API compatibility checks.
Single Node Openshift (Sno) Test Compatibility ✅ Passed The PR adds only standard Go unit tests (func Test...) and no Ginkgo e2e tests, so no SNO multi-node compatibility issue applies.
Topology-Aware Scheduling Compatibility ✅ Passed The patch only changes boot-image resolution, cache namespace access, and GCP architecture mapping; it adds no Kubernetes scheduling constraints, replicas, affinity, selectors, tolerations, topolog...
Ote Binary Stdout Contract ✅ Passed The repository builds controller binaries, not an OpenShift Tests Extension binary, and this PR adds no process-level stdout writes or suite setup output.
Ipv6 And Disconnected Network Test Compatibility ✅ Passed The PR adds only standard Go unit tests with fake Kubernetes clients and mocked GCP services. No new Ginkgo e2e tests, IPv4 assumptions, or external connectivity are present.
No-Weak-Crypto ✅ Passed PR additions contain no MD5, SHA1, DES, 3DES, RC4, Blowfish, or ECB; the only crypto code uses SHA-256 for public artifact checksums, not secrets or tokens.
Container-Privileges ✅ Passed The PR adds no container or Kubernetes security settings; scans of all non-vendored manifests and changed files found no privileged, host namespace, SYS_ADMIN, root, or privilege-escalation configu...
No-Sensitive-Data-In-Logs ✅ Passed New logs contain boot-image references, architecture, machine type/name, stream names, and error context; the resolver does not log passwords, tokens, user data, PII, or secret contents.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Comment @coderabbitai help to get the list of available commands.

@openshift-ci
openshift-ci Bot requested review from nrb and theobarberbany July 27, 2026 19:09
@nader-ziada

Copy link
Copy Markdown
Author

hello @RadekManak, this pr is following your suggestion in https://reports.radekmanak.com/687c2c45bd0b495ab03b1ea295d6d276

another PR will follow for the RBAC in machine-api-operator

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Nitpick comments (1)
pkg/cloud/gcp/actuators/machine/reconciler_test.go (1)

391-424: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Assert the exact create-path image source.

Both cases can pass if the resolver returns an arbitrary RHCOS image, so they do not protect the fallback-versus-ConfigMap behavior introduced here.

  • pkg/cloud/gcp/actuators/machine/reconciler_test.go#L391-L424: assert SourceImage == defaultGCPBootImageX86.
  • pkg/cloud/gcp/actuators/machine/reconciler_test.go#L901-L942: inject testBootImagesConfigMap() into the fake client and assert the exact stream-derived GCP image reference, not merely a string containing rhcos.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/cloud/gcp/actuators/machine/reconciler_test.go` around lines 391 - 424,
Strengthen the create-path assertions in
pkg/cloud/gcp/actuators/machine/reconciler_test.go:391-424 by requiring the boot
disk SourceImage to equal defaultGCPBootImageX86. In
pkg/cloud/gcp/actuators/machine/reconciler_test.go:901-942, inject
testBootImagesConfigMap() into the fake client and assert SourceImage equals the
exact stream-derived GCP image reference instead of only checking for “rhcos”.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/cloud/gcp/actuators/machine/boot_image.go`:
- Around line 52-81: The resolveImageFromConfigMap method must validate
archData.Images.Gcp.Project and Name before calling gcpImageReference; return
the existing missing-image error when either value is blank so fallback behavior
is preserved. Add a regression test covering blank Project and Name metadata.

---

Nitpick comments:
In `@pkg/cloud/gcp/actuators/machine/reconciler_test.go`:
- Around line 391-424: Strengthen the create-path assertions in
pkg/cloud/gcp/actuators/machine/reconciler_test.go:391-424 by requiring the boot
disk SourceImage to equal defaultGCPBootImageX86. In
pkg/cloud/gcp/actuators/machine/reconciler_test.go:901-942, inject
testBootImagesConfigMap() into the fake client and assert SourceImage equals the
exact stream-derived GCP image reference instead of only checking for “rhcos”.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: d5492457-a96d-43a0-85a7-2c48dc9e34b5

📥 Commits

Reviewing files that changed from the base of the PR and between 1d09813 and 1d3edb6.

⛔ Files ignored due to path filters (7)
  • go.sum is excluded by !**/*.sum
  • vendor/github.com/coreos/stream-metadata-go/LICENSE is excluded by !**/vendor/**, !vendor/**
  • vendor/github.com/coreos/stream-metadata-go/stream/artifact_utils.go is excluded by !**/vendor/**, !vendor/**
  • vendor/github.com/coreos/stream-metadata-go/stream/rhcos/rhcos.go is excluded by !**/vendor/**, !vendor/**
  • vendor/github.com/coreos/stream-metadata-go/stream/stream.go is excluded by !**/vendor/**, !vendor/**
  • vendor/github.com/coreos/stream-metadata-go/stream/stream_utils.go is excluded by !**/vendor/**, !vendor/**
  • vendor/modules.txt is excluded by !**/vendor/**, !vendor/**
📒 Files selected for processing (5)
  • go.mod
  • pkg/cloud/gcp/actuators/machine/boot_image.go
  • pkg/cloud/gcp/actuators/machine/boot_image_test.go
  • pkg/cloud/gcp/actuators/machine/reconciler.go
  • pkg/cloud/gcp/actuators/machine/reconciler_test.go

Comment thread pkg/cloud/gcp/actuators/machine/boot_image.go
@nader-ziada

Copy link
Copy Markdown
Author

return "", fmt.Errorf("failed to get coreos-bootimages ConfigMap: %w", err)
}

streamData, ok := cm.Data["stream"]

@djoshy djoshy Jul 31, 2026

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this generally looks fine, but the structure of the configmap has changed in 4.22/5.0 to account for rhel-9 and rhel-10 streams. There is a new key in this CM called streams which has both set of bootimages. The old stream key used here only references rhel-9 images and is planned to be deprecated at some point. You can also take a look at the structure of the configmap from a CI run(grep for the configmap).

I'm assuming this is not getting backported, so we should make sure that we are using correct bootimages from the new streams key and remove any dependance on the stream key. You can examine the OSImageStream object in the MCO namespace to determine what stream is in use by the customer(default is rhel-10 for 5.0 installs)

@nader-ziada nader-ziada Jul 31, 2026

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@djoshy thanks for reviewing, made a change to use streams key from coreos-bootimages ConfigMap

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/cloud/gcp/actuators/machine/boot_image_test.go`:
- Around line 77-84: Handle fixture-construction errors in
pkg/cloud/gcp/actuators/machine/boot_image_test.go:77-84 by updating
testStreamsJSON to return or propagate the json.Marshal error instead of
discarding it, and in pkg/cloud/gcp/actuators/machine/boot_image_test.go:110-116
update testOSImageStream to check unstructured.SetNestedField, failing setup
with the original error when it fails.

In `@pkg/cloud/gcp/actuators/machine/boot_image.go`:
- Line 127: Update the warning in the OSImageStream fallback handling near the
machine boot-image lookup to avoid interpolating the raw Kubernetes client
error. Use a stable, non-sensitive failure reason or omit the error value while
retaining the fallback stream name and behavior.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: 21217808-a5b5-44f6-9d0d-4c2e5a400984

📥 Commits

Reviewing files that changed from the base of the PR and between 1d3edb6 and c0ddd5c.

📒 Files selected for processing (3)
  • pkg/cloud/gcp/actuators/machine/boot_image.go
  • pkg/cloud/gcp/actuators/machine/boot_image_test.go
  • pkg/cloud/gcp/actuators/machine/reconciler.go
🚧 Files skipped from review as they are similar to previous changes (1)
  • pkg/cloud/gcp/actuators/machine/reconciler.go

Comment thread pkg/cloud/gcp/actuators/machine/boot_image_test.go
Comment thread pkg/cloud/gcp/actuators/machine/boot_image.go
The coreos-bootimages ConfigMap now contains a streams key with
per-stream boot image data (rhel-9, rhel-10) for OCP 5.0. Read the
OSImageStream cluster singleton to determine the active stream and
resolve the correct boot image from the streams key. Falls back to
the deprecated stream key for backwards compatibility during upgrades.

Signed-off-by: Nader Ziada <nziada@redhat.com>
@nader-ziada
nader-ziada force-pushed the resolve-boot-image-in-reconciler branch from c0ddd5c to 7d53743 Compare July 31, 2026 19:28
@damdo

damdo commented Aug 3, 2026

Copy link
Copy Markdown
Member

/retest

@RadekManak

Copy link
Copy Markdown
Contributor

/assign @RadekManak

@damdo

damdo commented Aug 5, 2026

Copy link
Copy Markdown
Member

/retest

@theobarberbany

Copy link
Copy Markdown
Contributor

@damdo the regression job history looks awful.

@nader-ziada if you have any bandwith can you dig into it? hopefully won't take too long 😓

@RadekManak

Copy link
Copy Markdown
Contributor

@nader-ziada
Can you take a look through this AI review.
https://reports.radekmanak.com/acc95839d103c60c5ca6b5eba61d2b19

Additionally, (human review)
I think it is worth updating the fallback image.
Can you add "a4x-" prefix to the ARM cpu fallback list.

@nader-ziada

Copy link
Copy Markdown
Author

/test regression-clusterinfra-gcp-ipi-mapi

@openshift-ci

openshift-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please ask for approval from radekmanak. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@nader-ziada

Copy link
Copy Markdown
Author

@RadekManak can you please take a look

P0 — Namespaced cache Fixed. Added openshift-machine-config-operator to DefaultNamespaces in cmd/manager/main.go so the cache-backed client can read the ConfigMap cross-namespace.
P2 — Silent fallback Fixed. Upgraded fallback log lines in boot_image.go from V(3) to Warningf so operators can see when resolution fails and a stale image is used.
Human: a4x- prefix Added a4x to the ARM prefix map in gcp_machine_architecture.go with a test case.
Human: fallback image Updated fallback images from RHCOS 4.14 (414.92.202307070025-0) to RHCOS 4.18 (418.94.202602022246-0) with a comment noting the source (openshift/installer release-4.18 rhcos.json).

@nader-ziada

Copy link
Copy Markdown
Author

@damdo the regression job history looks awful.

@nader-ziada if you have any bandwith can you dig into it? hopefully won't take too long 😓

will look into it

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@cmd/manager/main.go`:
- Around line 140-142: The default namespace scope must remain limited to
watchNamespace; remove openshift-machine-config-operator from DefaultNamespaces
and configure the coreos-bootimages lookup in the boot image resolver with a
per-object cache scope or uncached API reader, without broadening Machine or
MachineSet informer watches.
🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: openshift/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Enterprise

Run ID: f921c037-4b0a-470c-add6-4c9fe604c132

📥 Commits

Reviewing files that changed from the base of the PR and between 7d53743 and 96816bf.

📒 Files selected for processing (5)
  • cmd/manager/main.go
  • pkg/cloud/gcp/actuators/machine/boot_image.go
  • pkg/cloud/gcp/actuators/machine/reconciler.go
  • pkg/cloud/gcp/actuators/util/gcp_machine_architecture.go
  • pkg/cloud/gcp/actuators/util/gcp_machine_architecture_test.go
🚧 Files skipped from review as they are similar to previous changes (2)
  • pkg/cloud/gcp/actuators/machine/reconciler.go
  • pkg/cloud/gcp/actuators/machine/boot_image.go

Comment thread cmd/manager/main.go Outdated
@nader-ziada
nader-ziada force-pushed the resolve-boot-image-in-reconciler branch from 96816bf to 5183295 Compare August 7, 2026 13:21
- Add openshift-machine-config-operator to cache DefaultNamespaces so
  the cache-backed client can read the coreos-bootimages ConfigMap
  cross-namespace (P0 fix)
- Upgrade fallback log lines from V(3) to Warningf for operator
  visibility when image resolution fails
- Update fallback images from RHCOS 4.14 to 4.18
  (418.94.202602022246-0)
- Add a4x prefix to ARM64 machine type detection

Signed-off-by: Nader Ziada <nziada@redhat.com>
@nader-ziada
nader-ziada force-pushed the resolve-boot-image-in-reconciler branch from 5183295 to efc3d2f Compare August 7, 2026 13:37
@openshift-ci

openshift-ci Bot commented Aug 7, 2026

Copy link
Copy Markdown
Contributor

@nader-ziada: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/regression-clusterinfra-gcp-ipi-mapi efc3d2f link false /test regression-clusterinfra-gcp-ipi-mapi

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

jira/invalid-bug Indicates that a referenced Jira bug is invalid for the branch this PR is targeting. jira/severity-important Referenced Jira bug's severity is important for the branch this PR is targeting. jira/valid-reference Indicates that this PR references a valid Jira ticket of any type.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants