Skip to content

Enable codecov coverage checks - #1782

Merged
osmman merged 6 commits into
mainfrom
add-codecov-coverage
May 18, 2026
Merged

Enable codecov coverage checks#1782
osmman merged 6 commits into
mainfrom
add-codecov-coverage

Conversation

@kdacosta0

@kdacosta0 kdacosta0 commented May 7, 2026

Copy link
Copy Markdown
Member

Summary

Enable automated code coverage reporting via Codecov for the secure-sign-operator repository.

What's included

  • codecov.yml — Codecov configuration with two status check thresholds:
    • Patch coverage: requires 70% coverage on new/changed lines (with 5% threshold tolerance). This ensures new code meets a minimum quality bar.
    • Project coverage: targets auto (tracks against the base commit) and is set to informational — it reports but does not block merges. This provides visibility into overall coverage trends without being disruptive.
  • .github/workflows/code-coverage.yml — GitHub Actions workflow that:
    • Triggers on pushes to main/release* and pull requests targeting those branches
    • Runs go test -v -coverprofile=coverage.out ./...
    • Uploads the coverage report to Codecov using codecov/codecov-action@v5
  • .gitignore — Added explicit coverage.out entry to prevent local coverage artifacts from being committed

Required setup

CODECOV_TOKEN must be configured in the repository secrets.

  1. Sign up / log in at codecov.io
  2. Add the securesign/secure-sign-operator repository
  3. Copy the upload token
  4. Add it as a repository secret named CODECOV_TOKEN in GitHub Settings > Secrets and variables > Actions

Implements SECURESIGN-4378

Test plan

  • Verify the Code Coverage workflow triggers on this PR
  • Verify coverage.out is generated during the workflow but ignored by git
  • Verify Codecov receives the coverage report (check codecov.io dashboard)
  • Verify Codecov status checks appear on this PR (patch and project)
  • If the workflow fails with a token error, configure CODECOV_TOKEN in repo secrets and re-run

Based on the doc

@codecov-commenter

Copy link
Copy Markdown

Welcome to Codecov 🎉

Once you merge this PR into your default branch, you're all set! Codecov will compare coverage reports and display results in all future pull requests.

Thanks for integrating Codecov - We've got you covered ☂️

@kdacosta0 kdacosta0 closed this May 11, 2026
@kdacosta0 kdacosta0 reopened this May 11, 2026

@osmman osmman left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are a couple of issues with the current approach that should be addressed before merging.

1. Duplicated workflow — should extend existing CI instead

The build-operator job in main.yml already runs make docker-build, which depends on the test target (Makefile:167). That target executes the exact same command:

go test $(go list ./... | grep -v /e2e)

with the same envtest setup. This new workflow duplicates the entire checkout → Go install → envtest setup → test run pipeline, effectively doubling CI resource usage and execution time for the same tests.

Suggestion: Instead of a separate workflow, add -coverprofile=coverage.out to the existing test execution (e.g., via a Makefile variable or by modifying the test target) and add the Codecov upload step to the build-operator job in main.yml. This way coverage is collected as a side effect of the tests that already run, with zero additional cost.

2. E2e tests excluded from coverage

The grep -v /e2e filter excludes e2e tests entirely from coverage collection. E2e tests exercise a significant portion of the operator's code paths — especially the controller reconciliation logic, action chains, and integration with Kubernetes APIs.

While it's true that e2e tests run against a deployed operator in a Kind cluster (so they can't directly contribute to in-process coverage), the reported coverage numbers will be misleadingly low. Combined with the 70% patch coverage requirement on new code, this creates a problematic situation: a developer could write code that is thoroughly validated by e2e tests but has no unit tests, and Codecov would block the PR.

Consider either:

  • Lowering the patch target or making it informational (like the project target) until e2e coverage can be integrated
  • Exploring Go's built-in integration test coverage (go build -cover) to collect coverage from the operator binary running in the Kind cluster

3. Minor: redundant .gitignore entry

coverage.out is already covered by the existing *.out glob on line 22 of .gitignore. The explicit entry is unnecessary.

@kdacosta0
kdacosta0 marked this pull request as draft May 12, 2026 08:38
@kdacosta0
kdacosta0 force-pushed the add-codecov-coverage branch from bd57147 to fc86522 Compare May 18, 2026 09:49
@kdacosta0

Copy link
Copy Markdown
Member Author

2. E2e tests excluded from coverage

The grep -v /e2e filter excludes e2e tests entirely from coverage collection. E2e tests exercise a significant portion of the operator's code paths — especially the controller reconciliation logic, action chains, and integration with Kubernetes APIs.

While it's true that e2e tests run against a deployed operator in a Kind cluster (so they can't directly contribute to in-process coverage), the reported coverage numbers will be misleadingly low. Combined with the 70% patch coverage requirement on new code, this creates a problematic situation: a developer could write code that is thoroughly validated by e2e tests but has no unit tests, and Codecov would block the PR.

Consider either:

  • Lowering the patch target or making it informational (like the project target) until e2e coverage can be integrated
  • Exploring Go's built-in integration test coverage (go build -cover) to collect coverage from the operator binary running in the Kind cluster

I want to firstly add only unit tests, as the doc says:

"Unit test coverage is the priority. E2e coverage is a follow-up for Go/Python/Node.js services that have integration test pipelines."

@kdacosta0
kdacosta0 marked this pull request as ready for review May 18, 2026 10:31
@osmman

osmman commented May 18, 2026

Copy link
Copy Markdown
Collaborator

Try to rebase pr agains main. It should fix linter error

kdacosta0 and others added 6 commits May 18, 2026 14:00
…[SECURESIGN-4378]

Add Codecov integration for automated code coverage reporting:
- codecov.yml with patch target 70% (5% threshold) and project target auto (informational)
- GitHub Actions workflow to run tests with coverage and upload to Codecov
- Explicit coverage.out entry in .gitignore

Implements SECURESIGN-4378

Assisted-by: Claude Code
Bump actions/checkout, actions/setup-go, and codecov/codecov-action to v6.

Implements SECURESIGN-4378

Assisted-by: Claude Code
The test suite requires generated code (embed/images.env via go generate)
and envtest binaries (etcd, kube-apiserver) to run. Also exclude e2e tests
which require a full cluster.

Implements SECURESIGN-4378

Assisted-by: Claude Code
…akefile

The previous workflow hardcoded the envtest binary name and K8s version,
and used relative paths which broke when go test changed directories
per package. Now all values are derived from the Makefile dynamically.

Implements SECURESIGN-4378

Assisted-by: Claude Code
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Remove the separate code-coverage.yml workflow and collect coverage
as a side effect of the tests that already run in build-operator.
Update codecov.yml to match org standard configuration.

- Add -coverprofile to Makefile test target
- Add Codecov upload step with 'unit' flag to build-operator job
- Add flag_management with carryforward to codecov.yml
- Make patch target informational until e2e coverage is integrated
- Remove redundant coverage.out .gitignore entry (*.out covers it)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@kdacosta0
kdacosta0 force-pushed the add-codecov-coverage branch from fc86522 to 346afd4 Compare May 18, 2026 12:00
@osmman
osmman merged commit 3c41df0 into main May 18, 2026
15 checks passed
@osmman
osmman deleted the add-codecov-coverage branch May 18, 2026 13:25
@osmman osmman added the test label Jul 20, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants