🌱 test: replace ArgoCD manifest regression test with mock-based resource validation #2103
Conversation
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: The full list of commands accepted by this bot can be found here.
Needs approval from an approver in each of these files:
Approvers can indicate their approval by writing |
✅ Deploy Preview for olmv1 ready!
To edit notification comments on pull requests, go to your Netlify project configuration. |
camilamacedo86
force-pushed
the
remove-bundles-argo
branch
from
9be93c2 to
f32fc0e
Compare
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #2103 +/- ##
==========================================
- Coverage 73.48% 73.43% -0.05%
==========================================
Files 78 78
Lines 7240 7240
==========================================
- Hits 5320 5317 -3
- Misses 1568 1570 +2
- Partials 352 353 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
camilamacedo86
force-pushed
the
remove-bundles-argo
branch
7 times, most recently
from
311dab9 to
36ae501
Compare
camilamacedo86
force-pushed
the
remove-bundles-argo
branch
2 times, most recently
from
49b8051 to
f7a2e82
Compare
camilamacedo86
force-pushed
the
remove-bundles-argo
branch
from
f7a2e82 to
0abdf8f
Compare
camilamacedo86
changed the title
| // - All expected resources are returned. | ||
| // - The full content of each resource matches the expected values | ||
| // It validates that the rendered objects are correctly rendered. | ||
| func Test_Render_ValidatesOutputForAllInstallModes(t *testing.T) { |
There was a problem hiding this comment.
@perdasilva I could improve this one.
I hope that you liked it
@anik120 and @thetechnick were involved in the introduction of this one, so I think it's good to ping them to help us PTAL this one and get it merged.
There was a problem hiding this comment.
I think you've raised some valid concerns in the description, but missed the main point of this test. What we want to verify is that the registry+v1 bundle renderer consistently behaves in the expected way. That is, given the same bundle (+ config) the renderer always outputs the same manifests.
The way we decided to do this is by taking an existing bundle that we are confident is correct (since we use the ArgoCD operator in our config sample - we took that one) and using the registry+v1 bundle renderer to generate manifests with different configurations (right now watchNamespace, but will also be subscriptionConfig in the near future) and make sure that the output doesn't change (i.e. the renderer's behavior has not changed).
The way I'm reading the test here is we create our own renderer which returns the mocked resources and we verify that implementation. Does that make sense?
I'm also not a huge fan of carrying so many manifests and I hear your concerns about the whats and whys. Maybe we could find a different way to address those concerns? Maybe adding a README, etc.?
What we want to be certain of is that we consistently render registry+v1 bundles in the same way. The most important part of that is whether the ClusterServiceVersion gets transpiled in to the right Deployment, RBAC, Webhook resources. Other resource kinds will just get tacked on. So, things like NetworkPolicy are not so important - or are as important as any other (service, configmap, etc.).
Description
This PR replaces the old render regression test that used ArgoCD YAML files and
make verifyto check for changes. Instead, we now use a much simpler and clearer mock-based test. Context: #1897Why we changed it:
make verifyfailed, people could just run it again and accidentally accept changes without noticing regressions.What’s better now:
Deployment,Service,Role,NetworkPolicy, etc.It’s clearer, easier to maintain, and checks what we care about. 🌱