Skip to content

ACM-39570: fix namespace deletion blocked by HCP proxy rejecting cleanup requests - #773

Merged
openshift-merge-bot[bot] merged 3 commits into
stolostron:mainfrom
kurwang:ACM-39570
Aug 4, 2026
Merged

ACM-39570: fix namespace deletion blocked by HCP proxy rejecting cleanup requests#773
openshift-merge-bot[bot] merged 3 commits into
stolostron:mainfrom
kurwang:ACM-39570

Conversation

@kurwang

@kurwang kurwang commented Aug 3, 2026

Copy link
Copy Markdown
Contributor

Description of the change(s):

  • Add handleEmptyCollection to return an empty HostedClusterList for collection-level GET/DELETE requests that arrive without a hostingCluster query parameter
  • Update handleRoute to detect collection endpoints and route to handleEmptyCollection when hostingCluster is absent, instead of returning 400 BadRequest
  • Add list and deletecollection to the advertised verbs in the API discovery document for the hostedclusters resource

Why do we need this PR:

  • The Kubernetes namespace controller sends standard collection-level DELETE requests (DELETE /apis/hcp.ocm.io/v1alpha1/namespaces/{ns}/hostedclusters) during namespace cleanup without the custom ?hostingCluster query parameter
  • The HCP proxy unconditionally requires hostingCluster on all requests and rejects these with 400 BadRequest, causing every namespace deletion on the cluster to stall indefinitely with NamespaceDeletionContentFailure
  • Returning an empty list is semantically correct since the proxy stores no resources locally — it only proxies to spoke clusters

Issue reference:

Test API/Unit - Success

=== RUN   Test_handleRoute_WhenMissingHostingCluster_OnNamedEndpoint_ItShouldReturn400
--- PASS: Test_handleRoute_WhenMissingHostingCluster_OnNamedEndpoint_ItShouldReturn400 (0.00s)
=== RUN   Test_handleRoute_WhenMissingHostingCluster_OnCollectionGET_ItShouldReturnEmptyList
--- PASS: Test_handleRoute_WhenMissingHostingCluster_OnCollectionGET_ItShouldReturnEmptyList (0.00s)
=== RUN   Test_handleRoute_WhenMissingHostingCluster_OnCollectionDELETE_ItShouldReturnEmptyList
--- PASS: Test_handleRoute_WhenMissingHostingCluster_OnCollectionDELETE_ItShouldReturnEmptyList (0.00s)
=== RUN   Test_handleDiscovery_WhenVersionPath_ItShouldReturnAPIResourceList
--- PASS: Test_handleDiscovery_WhenVersionPath_ItShouldReturnAPIResourceList (0.01s)
=== RUN   Test_handleRoute_WhenListPath_ItShouldReturn405
--- PASS: Test_handleRoute_WhenListPath_ItShouldReturn405 (0.02s)
PASS
ok  	github.com/stolostron/hypershift-addon-operator/pkg/manager	2.120s

<!-- This is an auto-generated comment: release notes by coderabbit.ai -->

## Summary by CodeRabbit

* **New Features**
  * HostedCluster API discovery now advertises collection listing and deletion capabilities.
  * Collection-level list and delete requests without a hosting cluster parameter now succeed with an empty result.

* **Bug Fixes**
  * Namespace cleanup requests are no longer blocked by validation errors when no hosted clusters are present.
  * Requests targeting a specific hosted cluster still require the hosting cluster parameter.

<!-- end of auto-generated comment: release notes by coderabbit.ai -->

…espace deletion

The HCP proxy's handleRoute unconditionally required the hostingCluster
query parameter, causing the Kubernetes namespace controller's cleanup
DELETE/LIST requests to be rejected with 400 BadRequest. This blocked
namespace deletion cluster-wide since the namespace controller cannot
confirm resource cleanup for the hcp.ocm.io/v1alpha1 API group.

Return an empty HostedClusterList for collection-level requests without
hostingCluster, which is semantically correct since the proxy stores no
resources locally. Also add list and deletecollection to the discovery
verbs so the namespace controller knows these operations are supported.

Ref: https://issues.redhat.com/browse/ACM-39570
Signed-off-by: kurwang <kurwang@redhat.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
@coderabbitai

coderabbitai Bot commented Aug 3, 2026

Copy link
Copy Markdown

Warning

Review limit reached

@kurwang, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 21 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details
⚙️ Run configuration

Configuration used: Repository: stolostron/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: e3fe6f84-9972-4a7a-8c4d-e6fdcc084416

📥 Commits

Reviewing files that changed from the base of the PR and between aa18cb0 and 0bb1676.

📒 Files selected for processing (3)
  • pkg/manager/hcp_proxy.go
  • pkg/manager/hcp_proxy_test.go
  • test/e2e/hcp_proxy_test.go

Walkthrough

The HCP proxy now advertises list and deletecollection for HostedClusters. Collection GET and DELETE requests without hostingCluster return an empty HostedClusterList with HTTP 200. Named requests still require the parameter.

Changes

HostedCluster collection handling

Layer / File(s) Summary
Collection discovery and routing
pkg/manager/hcp_proxy.go
The proxy advertises collection verbs, accepts collection requests without hostingCluster, and returns an empty HostedClusterList.
Collection behavior tests
pkg/manager/hcp_proxy_test.go
Tests verify discovery verbs, named-request validation, and successful empty responses for collection GET and DELETE requests.

Estimated code review effort: 2 (Simple) | ~10 minutes

Suggested reviewers: yiraechristinekim

🚥 Pre-merge checks | ✅ 9 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Test Structure And Quality ⚠️ Warning Added assertions in pkg/manager/hcp_proxy_test.go:321-365 omit meaningful failure messages, including require.NoError calls, violating requirement 4. Add descriptive messages to every added assert and require call, such as the expected status, response kind, and empty items.
✅ Passed checks (9 passed)
Check name Status Explanation
Docstring Coverage ✅ Passed No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
Stable And Deterministic Test Names ✅ Passed The added Go test names are fixed identifiers with static text; they contain no generated names, timestamps, UUIDs, IPs, or interpolated runtime values.
No-Weak-Crypto ✅ Passed The exact PR diff only changes API verbs, routing, and empty-list tests; it adds no weak algorithms, custom crypto, or secret/token comparisons.
Container-Privileges ✅ Passed The PR changes only two Go files. No manifest privilege settings were added, and existing deployment settings use non-root, privileged:false, hostPID/hostNetwork/hostIPC:false.
No-Sensitive-Data-In-Logs ✅ Passed The commit adds no logging statements or log fields. The existing request middleware logs only HTTP method and URL path, not query strings, credentials, tokens, or payloads.
Title check ✅ Passed The title clearly identifies the namespace deletion fix caused by HCP proxy cleanup request rejection.
Description check ✅ Passed The description covers the changes, motivation, issue reference, and successful unit test results.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@kurwang

kurwang commented Aug 3, 2026

Copy link
Copy Markdown
Contributor Author

Root cause

When any namespace is deleted, the Kubernetes namespace controller enumerates all registered API groups and sends DELETE /apis/hcp.ocm.io/v1alpha1/namespaces/{ns}/hostedclusters to clean up resources. This is a standard Kubernetes API call with no custom query parameters. The HCP proxy rejected it with 400 BadRequest because hostingCluster was missing — a required parameter the namespace controller has no knowledge of. The proxy stores no resources locally (it only forwards to spoke clusters), so returning an empty list is the correct response when no spoke is targeted.

@kurwang kurwang changed the title fix: handle collection requests without hostingCluster to unblock nam… ACM-39570: fix namespace deletion blocked by HCP proxy rejecting cleanup requests Aug 3, 2026

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (1)
pkg/manager/hcp_proxy_test.go (1)

338-351: 🗄️ Data Integrity & Integration | 🔵 Trivial | ⚡ Quick win

Assert the complete empty-list payload.

The GET and DELETE tests verify only kind and empty items. They do not verify apiVersion or metadata.resourceVersion, which handleEmptyCollection now sets. Add assertions for these fields so an incomplete Kubernetes list response cannot pass.

Also applies to: 353-366

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@pkg/manager/hcp_proxy_test.go` around lines 338 - 351, The collection GET and
DELETE tests around handleRoute and handleEmptyCollection only validate kind and
empty items. Extend both test assertions to verify the response apiVersion and
metadata.resourceVersion fields match the values produced by
handleEmptyCollection, while preserving the existing empty-list checks.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@pkg/manager/hcp_proxy_test.go`:
- Around line 320-325: Update the handler tests around the status checks at
lines 286, 298, 311, 345, and 360 to use require.Equal with descriptive failure
messages, change collection items assertions to require.Empty, and add a
descriptive message to the JSON decode require.NoError call. Keep setup-failure
assertions fatal so subsequent test logic does not run on invalid responses.

In `@pkg/manager/hcp_proxy.go`:
- Line 499: Update the JSON response encoding call in the surrounding proxy
handler to handle the error returned by json.Encoder.Encode instead of assigning
it to _. Wrap encoding failures with fmt.Errorf("context: %w", err), then log or
return the wrapped error through the existing proxy error-handling path with
useful context.
- Around line 411-419: Update the collection shortcut around sanitizeProxyName
and handleEmptyCollection so it applies only when the hostingCluster query key
is absent and the request method is GET or DELETE. When the key is present,
validate every value using the existing allow-list validation and return 400 for
invalid or empty values; unsupported collection methods must continue through
normal handling rather than returning HostedClusterList. Add regression tests
covering invalid values and unsupported methods.
- Line 388: Update the resource discovery entry near the APIResourceList verbs
and dispatchCollection consistently: either implement delegation for collection
GET/DELETE to their intended handlers, or remove list and deletecollection from
the advertised verbs so discovery exposes only supported operations. Preserve
create and delete behavior already supported by dispatchCollection.

---

Nitpick comments:
In `@pkg/manager/hcp_proxy_test.go`:
- Around line 338-351: The collection GET and DELETE tests around handleRoute
and handleEmptyCollection only validate kind and empty items. Extend both test
assertions to verify the response apiVersion and metadata.resourceVersion fields
match the values produced by handleEmptyCollection, while preserving the
existing empty-list checks.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Repository: stolostron/coderabbit/.coderabbit.yaml

Review profile: CHILL

Plan: Pro Plus

Run ID: 29a26000-e21c-4b35-a285-9a144b06bfcc

📥 Commits

Reviewing files that changed from the base of the PR and between 91fb96a and aa18cb0.

📒 Files selected for processing (2)
  • pkg/manager/hcp_proxy.go
  • pkg/manager/hcp_proxy_test.go

Comment thread pkg/manager/hcp_proxy_test.go
Comment thread pkg/manager/hcp_proxy.go
Comment thread pkg/manager/hcp_proxy.go Outdated
Comment thread pkg/manager/hcp_proxy.go
kurwang added 2 commits August 3, 2026 12:44
Signed-off-by: kurwang <kurwang@redhat.com>
Signed-off-by: kurwang <kurwang@redhat.com>
@sonarqubecloud

sonarqubecloud Bot commented Aug 3, 2026

Copy link
Copy Markdown

@openshift-ci

openshift-ci Bot commented Aug 4, 2026

Copy link
Copy Markdown

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: kurwang, rokej

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@openshift-ci openshift-ci Bot added the lgtm label Aug 4, 2026
@openshift-merge-bot
openshift-merge-bot Bot merged commit 3868d04 into stolostron:main Aug 4, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants