OCPBUGS-58259: Reduce Frequency of Update Requests for Copied CSVs (#3597) [release-4.19]

[tmshort]

Use a TransformFunc to generate hashes of the copied CSVs to reduced storage needs, and to minimize API access.

The following is the complete unsquashed commit listing

• (bugfix): reduce frequency of update requests for CSVs

by adding annotations to copied CSVs that are populated with hashes of the non-status fields and the status fields.

This seems to be how this was intended to work, but was not actually working this way because the annotations never actually existed on the copied CSV. This resulted in a hot loop of update requests being made on all copied CSVs.

• update unit tests

• updates to test so far

• Small changes

• Add metadata drift guard to copyToNamespace

Since we switched to a PartialObjectMetadata cache to save memory, we lost visibility into copied CSV spec and status fields, and the reintroduced nonStatusCopyHash/statusCopyHash annotations only partially solved the problem. Manual edits to a copied CSV could still go undetected, causing drift without reconciliation.

This commit adds two new annotations: olm.operatorframework.io/observedGeneration and olm.operatorframework.io/observedResourceVersion. It implements a mechanism to guard agains metadata drift at the top of the existing-copy path in copyToNamespace. If a stored observedGeneration or observedResourceVersion no longer matches the live object, the operator now:

  • Updates the spec and hash annotations
  • Updates the status subresource
  • Records the new generation and resourceVersion in the guard annotations

Because the guard only fires when its annotations are already present, all existing unit tests pass unchanged. We preserve the memory benefits of the metadata‐only informer, avoid extra GETs, and eliminate unnecessary API churn.

Future work may explore a WithTransform informer to regain full object visibility with minimal memory impact.

• Tests for metadata guard

Verifies that exactly three updates (spec, status, guard) are issued when the observedGeneration doesn’t match.

• Persist observed annotations on all status updates

• GCI the file

• Use TransformFunc

Unit tests not updated

• Update operatorgroup tests to compile

• Restore operatorgroup_test from master

Remove metadatalister

• Remove more PartialObjectMetadata

• Remove hashes from operator_test

• Fix error messages for static-analysis

• Update test annotations and test client

• Rename pruning to listerwatcher

• Set resync to 6h

• Add CSV copy revert syncer

• Log tweaks

• Consolidate revert and gc syncers

• Add logging and reduce the amount of metadata in the TransformFunc

• Handle the copy CSV revert via a requeue of the primary CSV

• Revert "Set resync to 6h"

This reverts commit 855f940a2199bd4071c51f14ef44728550bf13cf.

• Add delete handler for copied csv

• Revert whitespace change

• Rename function, fix comment

---

Upstream-repository: operator-lifecycle-manager
Upstream-commit: d055f28750cf62f966f566d36990fff5285c7a71 (cherry picked from commit bc111a9)

[openshift-ci-robot]

@tmshort: This pull request references Jira Issue OCPBUGS-58259, which is invalid:

• expected the bug to target either version "4.19." or "openshift-4.19.", but it targets "4.20" instead
• release note text must be set and not match the template OR release note type must be set to "Release Note Not Required". For more information you can reference the OpenShift Bug Process.
• expected dependent Jira Issue OCPBUGS-37982 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead
• expected dependent Jira Issue OCPBUGS-37982 to target a version in 4.20.0, but it targets "4.20" instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Use a TransformFunc to generate hashes of the copied CSVs to reduced storage needs, and to minimize API access.

The following is the complete unsquashed commit listing

• (bugfix): reduce frequency of update requests for CSVs

by adding annotations to copied CSVs that are populated with hashes of the non-status fields and the status fields.

This seems to be how this was intended to work, but was not actually working this way because the annotations never actually existed on the copied CSV. This resulted in a hot loop of update requests being made on all copied CSVs.

• update unit tests

• updates to test so far

• Small changes

• Add metadata drift guard to copyToNamespace

Since we switched to a PartialObjectMetadata cache to save memory, we lost visibility into copied CSV spec and status fields, and the reintroduced nonStatusCopyHash/statusCopyHash annotations only partially solved the problem. Manual edits to a copied CSV could still go undetected, causing drift without reconciliation.

This commit adds two new annotations: olm.operatorframework.io/observedGeneration and olm.operatorframework.io/observedResourceVersion. It implements a mechanism to guard agains metadata drift at the top of the existing-copy path in copyToNamespace. If a stored observedGeneration or observedResourceVersion no longer matches the live object, the operator now:

 • Updates the spec and hash annotations
 • Updates the status subresource
 • Records the new generation and resourceVersion in the guard annotations

Because the guard only fires when its annotations are already present, all existing unit tests pass unchanged. We preserve the memory benefits of the metadata‐only informer, avoid extra GETs, and eliminate unnecessary API churn.

Future work may explore a WithTransform informer to regain full object visibility with minimal memory impact.

• Tests for metadata guard

Verifies that exactly three updates (spec, status, guard) are issued when the observedGeneration doesn’t match.

• Persist observed annotations on all status updates

• GCI the file

• Use TransformFunc

Unit tests not updated

• Update operatorgroup tests to compile

• Restore operatorgroup_test from master

Remove metadatalister

• Remove more PartialObjectMetadata

• Remove hashes from operator_test

• Fix error messages for static-analysis

• Update test annotations and test client

• Rename pruning to listerwatcher

• Set resync to 6h

• Add CSV copy revert syncer

• Log tweaks

• Consolidate revert and gc syncers

• Add logging and reduce the amount of metadata in the TransformFunc

• Handle the copy CSV revert via a requeue of the primary CSV

• Revert "Set resync to 6h"

This reverts commit 855f940a2199bd4071c51f14ef44728550bf13cf.

• Add delete handler for copied csv

• Revert whitespace change

• Rename function, fix comment

---

Upstream-repository: operator-lifecycle-manager
Upstream-commit: d055f28750cf62f966f566d36990fff5285c7a71 (cherry picked from commit bc111a9)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[tmshort]

/jira refresh

[openshift-ci-robot]

@tmshort: This pull request references Jira Issue OCPBUGS-58259, which is invalid:

• release note text must be set and not match the template OR release note type must be set to "Release Note Not Required". For more information you can reference the OpenShift Bug Process.
• expected dependent Jira Issue OCPBUGS-37982 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[tmshort]

/jira refresh

[openshift-ci-robot]

@tmshort: This pull request references Jira Issue OCPBUGS-58259, which is invalid:

• expected dependent Jira Issue OCPBUGS-37982 to be in one of the following states: VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA), but it is ON_QA instead

Comment /jira refresh to re-evaluate validity if changes to the Jira bug are made, or edit the title of this pull request to link to a different bug.

In response to this:

/jira refresh

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[openshift-ci]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: oceanc80, tmshort

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• DOWNSTREAM_OWNERS [oceanc80,tmshort]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[oceanc80]

/label backport-risk-assessed

[kuiwang02]

/hold
I will pre-verify it soon. and will unhold it after it is done.

thanks

[kuiwang02]

pre-verification pass
/unhold

[kuiwang02]

/label qe-approved

[openshift-ci-robot]

@tmshort: This pull request references Jira Issue OCPBUGS-58259, which is valid.

7 validation(s) were run on this bug

• bug is open, matching expected state (open)
• bug target version (4.19.z) matches configured target version for branch (4.19.z)
• bug is in the state POST, which is one of the valid states (NEW, ASSIGNED, POST)
• release note text is set and does not match the template
• dependent bug Jira Issue OCPBUGS-37982 is in the state Verified, which is one of the valid states (VERIFIED, RELEASE PENDING, CLOSED (ERRATA), CLOSED (CURRENT RELEASE), CLOSED (DONE), CLOSED (DONE-ERRATA))
• dependent Jira Issue OCPBUGS-37982 targets the "4.20.0" version, which is one of the valid target versions: 4.20.0
• bug has dependents

Requesting review from QA contact:
/cc @kuiwang02

The bug has been updated to refer to the pull request using the external bug tracker.

In response to this:

Use a TransformFunc to generate hashes of the copied CSVs to reduced storage needs, and to minimize API access.

The following is the complete unsquashed commit listing

• (bugfix): reduce frequency of update requests for CSVs

by adding annotations to copied CSVs that are populated with hashes of the non-status fields and the status fields.

This seems to be how this was intended to work, but was not actually working this way because the annotations never actually existed on the copied CSV. This resulted in a hot loop of update requests being made on all copied CSVs.

• update unit tests

• updates to test so far

• Small changes

• Add metadata drift guard to copyToNamespace

Since we switched to a PartialObjectMetadata cache to save memory, we lost visibility into copied CSV spec and status fields, and the reintroduced nonStatusCopyHash/statusCopyHash annotations only partially solved the problem. Manual edits to a copied CSV could still go undetected, causing drift without reconciliation.

This commit adds two new annotations: olm.operatorframework.io/observedGeneration and olm.operatorframework.io/observedResourceVersion. It implements a mechanism to guard agains metadata drift at the top of the existing-copy path in copyToNamespace. If a stored observedGeneration or observedResourceVersion no longer matches the live object, the operator now:

 • Updates the spec and hash annotations
 • Updates the status subresource
 • Records the new generation and resourceVersion in the guard annotations

Because the guard only fires when its annotations are already present, all existing unit tests pass unchanged. We preserve the memory benefits of the metadata‐only informer, avoid extra GETs, and eliminate unnecessary API churn.

Future work may explore a WithTransform informer to regain full object visibility with minimal memory impact.

• Tests for metadata guard

Verifies that exactly three updates (spec, status, guard) are issued when the observedGeneration doesn’t match.

• Persist observed annotations on all status updates

• GCI the file

• Use TransformFunc

Unit tests not updated

• Update operatorgroup tests to compile

• Restore operatorgroup_test from master

Remove metadatalister

• Remove more PartialObjectMetadata

• Remove hashes from operator_test

• Fix error messages for static-analysis

• Update test annotations and test client

• Rename pruning to listerwatcher

• Set resync to 6h

• Add CSV copy revert syncer

• Log tweaks

• Consolidate revert and gc syncers

• Add logging and reduce the amount of metadata in the TransformFunc

• Handle the copy CSV revert via a requeue of the primary CSV

• Revert "Set resync to 6h"

This reverts commit 855f940a2199bd4071c51f14ef44728550bf13cf.

• Add delete handler for copied csv

• Revert whitespace change

• Rename function, fix comment

---

Upstream-repository: operator-lifecycle-manager
Upstream-commit: d055f28750cf62f966f566d36990fff5285c7a71 (cherry picked from commit bc111a9)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[camilamacedo86]

/test e2e-gcp-olm

[grokspawn]

/retest-required

[tmshort]

/retest-required

[tmshort]

/retest-required
We seem to be consistently failing:

End-to-end: [It] Starting CatalogSource e2e tests when The namespace is labled as Pod Security Admission policy enforce:baseline when A CatalogSource built with opm v1.21.0 (<v1.23.2)is created with spec.GrpcPodConfig.SecurityContextConfig set to legacy The registry pod comes up successfully [CatalogSource] 

However, this PR doesn't do anything with CatSrc's...
Going to try one more time...

[camilamacedo86]

/test e2e-gcp-olm

[tmshort]

/retest-required

[camilamacedo86]

/test e2e-gcp-olm

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 4a6530a and 2 for PR HEAD 00d6504 in total

[tmshort]

This keeps failing, and seems unrelated....

End-to-end: [It] Starting CatalogSource e2e tests when The namespace is labled as Pod Security Admission policy enforce:baseline when A CatalogSource built with opm v1.21.0 (<v1.23.2)is created with spec.GrpcPodConfig.SecurityContextConfig set to legacy The registry pod comes up successfully [CatalogSource] 

[openshift-ci-robot]

/retest-required

Remaining retests: 0 against base HEAD 4a6530a and 2 for PR HEAD 00d6504 in total

[openshift-ci]

@tmshort: all tests passed!

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

[openshift-ci-robot]

@tmshort: Jira Issue OCPBUGS-58259: All pull requests linked via external trackers have merged:

• openshift/operator-framework-olm#1030

Jira Issue OCPBUGS-58259 has been moved to the MODIFIED state.

In response to this:

Use a TransformFunc to generate hashes of the copied CSVs to reduced storage needs, and to minimize API access.

The following is the complete unsquashed commit listing

• (bugfix): reduce frequency of update requests for CSVs

by adding annotations to copied CSVs that are populated with hashes of the non-status fields and the status fields.

This seems to be how this was intended to work, but was not actually working this way because the annotations never actually existed on the copied CSV. This resulted in a hot loop of update requests being made on all copied CSVs.

• update unit tests

• updates to test so far

• Small changes

• Add metadata drift guard to copyToNamespace

Since we switched to a PartialObjectMetadata cache to save memory, we lost visibility into copied CSV spec and status fields, and the reintroduced nonStatusCopyHash/statusCopyHash annotations only partially solved the problem. Manual edits to a copied CSV could still go undetected, causing drift without reconciliation.

This commit adds two new annotations: olm.operatorframework.io/observedGeneration and olm.operatorframework.io/observedResourceVersion. It implements a mechanism to guard agains metadata drift at the top of the existing-copy path in copyToNamespace. If a stored observedGeneration or observedResourceVersion no longer matches the live object, the operator now:

 • Updates the spec and hash annotations
 • Updates the status subresource
 • Records the new generation and resourceVersion in the guard annotations

Because the guard only fires when its annotations are already present, all existing unit tests pass unchanged. We preserve the memory benefits of the metadata‐only informer, avoid extra GETs, and eliminate unnecessary API churn.

Future work may explore a WithTransform informer to regain full object visibility with minimal memory impact.

• Tests for metadata guard

Verifies that exactly three updates (spec, status, guard) are issued when the observedGeneration doesn’t match.

• Persist observed annotations on all status updates

• GCI the file

• Use TransformFunc

Unit tests not updated

• Update operatorgroup tests to compile

• Restore operatorgroup_test from master

Remove metadatalister

• Remove more PartialObjectMetadata

• Remove hashes from operator_test

• Fix error messages for static-analysis

• Update test annotations and test client

• Rename pruning to listerwatcher

• Set resync to 6h

• Add CSV copy revert syncer

• Log tweaks

• Consolidate revert and gc syncers

• Add logging and reduce the amount of metadata in the TransformFunc

• Handle the copy CSV revert via a requeue of the primary CSV

• Revert "Set resync to 6h"

This reverts commit 855f940a2199bd4071c51f14ef44728550bf13cf.

• Add delete handler for copied csv

• Revert whitespace change

• Rename function, fix comment

---

Upstream-repository: operator-lifecycle-manager
Upstream-commit: d055f28750cf62f966f566d36990fff5285c7a71 (cherry picked from commit bc111a9)

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the openshift-eng/jira-lifecycle-plugin repository.

[tmshort]

/cherry-pick release-4.18

[openshift-cherrypick-robot]

@tmshort: new pull request created: #1034

In response to this:

/cherry-pick release-4.18

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: operator-registry
This PR has been included in build operator-registry-container-v4.19.0-202507081507.p0.g0e4e456.assembly.stream.el9.
All builds following this will include this PR.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: ose-operator-framework-tools
This PR has been included in build ose-operator-framework-tools-container-v4.19.0-202507081507.p0.g0e4e456.assembly.stream.el9.
All builds following this will include this PR.

[openshift-bot]

[ART PR BUILD NOTIFIER]

Distgit: operator-lifecycle-manager
This PR has been included in build operator-lifecycle-manager-container-v4.19.0-202507081507.p0.g0e4e456.assembly.stream.el9.
All builds following this will include this PR.

[openshift-merge-robot]

Fix included in accepted release 4.19.0-0.nightly-2025-07-09-044836