Enable codecov coverage checks - #1782
Conversation
Welcome to Codecov 🎉Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests. Thanks for integrating Codecov - We've got you covered ☂️ |
osmman
left a comment
There was a problem hiding this comment.
There are a couple of issues with the current approach that should be addressed before merging.
1. Duplicated workflow — should extend existing CI instead
The build-operator job in main.yml already runs make docker-build, which depends on the test target (Makefile:167). That target executes the exact same command:
go test $(go list ./... | grep -v /e2e)
with the same envtest setup. This new workflow duplicates the entire checkout → Go install → envtest setup → test run pipeline, effectively doubling CI resource usage and execution time for the same tests.
Suggestion: Instead of a separate workflow, add -coverprofile=coverage.out to the existing test execution (e.g., via a Makefile variable or by modifying the test target) and add the Codecov upload step to the build-operator job in main.yml. This way coverage is collected as a side effect of the tests that already run, with zero additional cost.
2. E2e tests excluded from coverage
The grep -v /e2e filter excludes e2e tests entirely from coverage collection. E2e tests exercise a significant portion of the operator's code paths — especially the controller reconciliation logic, action chains, and integration with Kubernetes APIs.
While it's true that e2e tests run against a deployed operator in a Kind cluster (so they can't directly contribute to in-process coverage), the reported coverage numbers will be misleadingly low. Combined with the 70% patch coverage requirement on new code, this creates a problematic situation: a developer could write code that is thoroughly validated by e2e tests but has no unit tests, and Codecov would block the PR.
Consider either:
- Lowering the patch target or making it
informational(like the project target) until e2e coverage can be integrated - Exploring Go's built-in integration test coverage (
go build -cover) to collect coverage from the operator binary running in the Kind cluster
3. Minor: redundant .gitignore entry
coverage.out is already covered by the existing *.out glob on line 22 of .gitignore. The explicit entry is unnecessary.
bd57147 to
fc86522
Compare
I want to firstly add only unit tests, as the doc says: "Unit test coverage is the priority. E2e coverage is a follow-up for Go/Python/Node.js services that have integration test pipelines." |
|
Try to rebase pr agains main. It should fix linter error |
…[SECURESIGN-4378] Add Codecov integration for automated code coverage reporting: - codecov.yml with patch target 70% (5% threshold) and project target auto (informational) - GitHub Actions workflow to run tests with coverage and upload to Codecov - Explicit coverage.out entry in .gitignore Implements SECURESIGN-4378 Assisted-by: Claude Code
Bump actions/checkout, actions/setup-go, and codecov/codecov-action to v6. Implements SECURESIGN-4378 Assisted-by: Claude Code
The test suite requires generated code (embed/images.env via go generate) and envtest binaries (etcd, kube-apiserver) to run. Also exclude e2e tests which require a full cluster. Implements SECURESIGN-4378 Assisted-by: Claude Code
…akefile The previous workflow hardcoded the envtest binary name and K8s version, and used relative paths which broke when go test changed directories per package. Now all values are derived from the Makefile dynamically. Implements SECURESIGN-4378 Assisted-by: Claude Code
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove the separate code-coverage.yml workflow and collect coverage as a side effect of the tests that already run in build-operator. Update codecov.yml to match org standard configuration. - Add -coverprofile to Makefile test target - Add Codecov upload step with 'unit' flag to build-operator job - Add flag_management with carryforward to codecov.yml - Make patch target informational until e2e coverage is integrated - Remove redundant coverage.out .gitignore entry (*.out covers it) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
fc86522 to
346afd4
Compare
Summary
Enable automated code coverage reporting via Codecov for the secure-sign-operator repository.
What's included
codecov.yml— Codecov configuration with two status check thresholds:auto(tracks against the base commit) and is set toinformational— it reports but does not block merges. This provides visibility into overall coverage trends without being disruptive..github/workflows/code-coverage.yml— GitHub Actions workflow that:main/release*and pull requests targeting those branchesgo test -v -coverprofile=coverage.out ./...codecov/codecov-action@v5.gitignore— Added explicitcoverage.outentry to prevent local coverage artifacts from being committedRequired setup
Implements SECURESIGN-4378
Test plan
Code Coverageworkflow triggers on this PRcoverage.outis generated during the workflow but ignored by gitCODECOV_TOKENin repo secrets and re-runBased on the doc