ci: onboard codecov#171
Conversation
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
WalkthroughAdds CI coverage generation and artifact export, a downstream workflow to upload coverage to Codecov, Makefile targets for producing and uploading coverage (with metadata extraction), a ChangesCodecov Integration for Test Coverage Reporting
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~20 minutes 🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches🧪 Generate unit tests (beta)
Comment |
coderabbitai
Bot
added
chore
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 6
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/test-with-coverage.yml:
- Around line 16-19: Replace mutable action tags with pinned commit SHAs for
actions/checkout, actions/setup-go, and actions/upload-artifact and add safer
checkout defaults: set persist-credentials: false and fetch-depth: 0 remains
fine; for actions/setup-go (the step using setup-go) explicitly control caching
instead of relying on implicit behavior—either set cache: 'false' and add a
separate actions/cache step for Go modules or pin the setup-go SHA that supports
the desired cache mode; ensure upload-artifact uses a pinned SHA as well. Locate
the steps referencing actions/checkout, actions/setup-go, and
actions/upload-artifact and update their versions to commit SHAs and add
persist-credentials: false to the checkout step and an explicit cache
configuration for Go.
In @.github/workflows/upload-coverage.yml:
- Line 15: Replace the floating GitHub Action tags with immutable commit SHAs:
change the uses entries for "actions/download-artifact@v8" and
"codecov/codecov-action@v6" to their corresponding commit SHAs (e.g.,
actions/download-artifact@<commit-sha> and codecov/codecov-action@<commit-sha>),
locating the usages by the exact strings "actions/download-artifact@v8" and
"codecov/codecov-action@v6" and updating the right-hand side to the commit SHA
values to prevent upstream tag drift.
- Around line 2-5: Tighten the workflow_run trust boundary: update the upload
job to only run when the triggering workflow run is from a trusted branch/repo
by adding an if condition that checks github.event.workflow_run.head_branch ==
'master' and github.event.workflow_run.head_repository.full_name ==
github.repository (or add on.workflow_run.branches to the trigger); also replace
tag-based third-party actions actions/download-artifact@v8 and
codecov/codecov-action@v6 with their pinned commit SHAs to avoid supply-chain
risk. Ensure the checks are applied to the job that reads secrets.CODECOV_TOKEN
and that the action refs are updated wherever those action names appear.
In `@Makefile`:
- Line 98: The Makefile currently runs the `env` command (exposed on line shown)
which can leak secrets into CI logs; remove or replace that invocation so the
full environment is not printed in CI. Update the Makefile rule that calls `env`
(the `env` invocation or the target that executes it) to instead either print
only explicit safe variables, conditionally run full `env` behind a
developer-only flag like `CI_DEBUG`, or redact sensitive vars before logging;
ensure the changed target name or invocation (the `env` call) is updated
consistently and documented in the Makefile comments.
- Line 118: Remove the hardcoded secret assigned to CODECOV_TOKEN in the
Makefile and replace it with a secure retrieval from the environment (e.g., read
CODECOV_TOKEN from the process or CI environment) so no plaintext token is
committed; update references that rely on the Makefile variable (CODECOV_TOKEN)
to fail gracefully if the env var is missing and rotate/revoke the leaked token
immediately outside the repo.
- Around line 101-115: The Makefile's upload-codecov-report step currently runs
unpinned remote code via bash <(curl -s https://codecov.io/bash) and uses
CODECOV_TOKEN directly; replace these invocations in the upload section (both PR
and post-merge branches) with a pinned/verified uploader approach (for example,
invoke the official Codecov GitHub Action in CI or vendor a specific commit of
the uploader script) and stop executing curl|bash at runtime; also ensure the
token is read from a CI secret/env variable (do not hard-code CODECOV_TOKEN into
the repo) and keep existing variables (COV_DIR, PR_COMMIT, BASE_COMMIT,
REPO_OWNER, REPO_NAME, PULL_NUMBER) for metadata when calling the safe uploader.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Enterprise
Run ID: 8c99b263-9a74-4bb2-8313-138f6d49d85a
📒 Files selected for processing (5)
.codecov.yaml.github/workflows/test-with-coverage.yml.github/workflows/upload-coverage.yml.gitignoreMakefile
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (4)
- GitHub Check: test
- GitHub Check: E2E Tests
- GitHub Check: Bundle Validation
- GitHub Check: Unit Tests
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
-Focus on major issues impacting performance, readability, maintainability and security. Avoid nitpicks and avoid verbosity.
Files:
Makefile
🪛 Betterleaks (1.3.1)
Makefile
[high] 118-118: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🪛 checkmake (0.3.2)
Makefile
[warning] 92-92: Target body for "upload-codecov-report" exceeds allowed length of 5 lines (6).
(maxbodylength)
🪛 zizmor (1.25.2)
.github/workflows/upload-coverage.yml
[error] 2-5: use of fundamentally insecure workflow trigger (dangerous-triggers): workflow_run is almost always used insecurely
(dangerous-triggers)
[error] 15-15: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 22-22: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
.github/workflows/test-with-coverage.yml
[warning] 16-19: credential persistence through GitHub Actions artifacts (artipacked): does not set persist-credentials: false
(artipacked)
[error] 17-17: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 22-22: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 35-35: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 22-22: runtime artifacts potentially vulnerable to a cache poisoning attack (cache-poisoning): enables caching by default
(cache-poisoning)
|
|
||
| steps: | ||
| - name: Download coverage artifact | ||
| uses: actions/download-artifact@v8 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/upload-coverage.yml"
echo "== File: $FILE =="
nl -ba "$FILE" | sed -n '1,120p'
echo
echo "== Actions used (uses:) =="
rg -n "uses:\s" "$FILE" || trueRepository: codeready-toolchain/claw-operator
Length of output: 188
🏁 Script executed:
#!/bin/bash
set -euo pipefail
FILE=".github/workflows/upload-coverage.yml"
echo "== File: $FILE (numbered) =="
cat -n "$FILE" | sed -n '1,200p'
echo
echo "== Actions used (uses:) =="
rg -n "uses:\s" "$FILE" || true
echo
echo "== Look for codecov/codecov-action =="
rg -n "codecov/codecov-action" "$FILE" || true
echo
echo "== Look for actions/download-artifact =="
rg -n "actions/download-artifact" "$FILE" || trueRepository: codeready-toolchain/claw-operator
Length of output: 1607
Pin GitHub Actions to commit SHAs to prevent upstream tag drift.
- Update
actions/download-artifact@v8(line 15) to a commit SHA - Update
codecov/codecov-action@v6(line 22) to a commit SHA
🧰 Tools
🪛 zizmor (1.25.2)
[error] 15-15: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/upload-coverage.yml at line 15, Replace the floating
GitHub Action tags with immutable commit SHAs: change the uses entries for
"actions/download-artifact@v8" and "codecov/codecov-action@v6" to their
corresponding commit SHAs (e.g., actions/download-artifact@<commit-sha> and
codecov/codecov-action@<commit-sha>), locating the usages by the exact strings
"actions/download-artifact@v8" and "codecov/codecov-action@v6" and updating the
right-hand side to the commit SHA values to prevent upstream tag drift.
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In @.github/workflows/test.yml:
- Around line 29-30: The workflow step named "Upload coverage artifact"
currently uses the floating ref actions/upload-artifact@v7; replace that with
the exact immutable commit SHA for the actions/upload-artifact repository (e.g.,
actions/upload-artifact@<full-commit-sha>) to pin the action. Locate the step by
the step name "Upload coverage artifact" or the reference
"actions/upload-artifact@v7", fetch the corresponding canonical commit SHA from
the actions/upload-artifact GitHub repo, and update the workflow entry to use
that full SHA instead of the tag.
In @.github/workflows/upload-coverage.yml:
- Around line 2-5: The workflow currently listens for workflow_run.workflows = [
test-with-coverage ] which doesn't match the actual test workflow name "Tests",
so the upload job never triggers; update the workflow_run.workflows array to use
the exact name of the test workflow (e.g., "Tests") or rename the test workflow
to "test-with-coverage" so the names match, ensuring the upload-coverage
workflow triggers when the test workflow completes.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Enterprise
Run ID: d7ca44ff-64d7-4575-94e7-880329fecc6a
📒 Files selected for processing (5)
.codecov.yaml.github/workflows/test.yml.github/workflows/upload-coverage.yml.gitignoreMakefile
✅ Files skipped from review due to trivial changes (1)
- .gitignore
🚧 Files skipped from review as they are similar to previous changes (1)
- .codecov.yaml
📜 Review details
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (2)
- GitHub Check: E2E Tests
- GitHub Check: Unit Tests
🧰 Additional context used
📓 Path-based instructions (1)
**
⚙️ CodeRabbit configuration file
-Focus on major issues impacting performance, readability, maintainability and security. Avoid nitpicks and avoid verbosity.
Files:
Makefile
🪛 Betterleaks (1.3.1)
Makefile
[high] 113-113: Detected a Generic API Key, potentially exposing access to various services and sensitive operations.
(generic-api-key)
🪛 checkmake (0.3.2)
Makefile
[warning] 87-87: Target body for "upload-codecov-report" exceeds allowed length of 5 lines (6).
(maxbodylength)
🪛 zizmor (1.25.2)
.github/workflows/test.yml
[error] 30-30: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
.github/workflows/upload-coverage.yml
[error] 2-5: use of fundamentally insecure workflow trigger (dangerous-triggers): workflow_run is almost always used insecurely
(dangerous-triggers)
[error] 15-15: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
[error] 22-22: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🔀 Multi-repo context codeready-toolchain/api, codeready-toolchain/toolchain-common, codeready-toolchain/host-operator, codeready-toolchain/toolchain-e2e
[::codeready-toolchain/api::]
- make/test.mk: lines show coverage generation to $(COV_DIR)/coverage.txt and an upload-codecov-report target using bash <(curl -s https://codecov.io/bash) with -f $(COV_DIR)/coverage.txt and PR vs base commit logic. (matches output lines ~28-46, 52-54)
- make/test.mk: contains a hard-coded CODECOV_TOKEN := "826c8011-d0bd-4223-b7e9-bc56dae66f3e" (line shown in results)
[::codeready-toolchain/toolchain-common::]
- codecov.yaml exists at repo root (referenced in search output).
- make/test.mk: defines test-with-coverage/test-ci/upload-codecov-report targets; writes coverage to $(COV_DIR)/coverage.txt and uploads via codecov bash uploader with PR vs base commit branching. (matches lines ~8,20-37,43-45)
- make/test.mk: contains a hard-coded CODECOV_TOKEN := "543cc327-510b-4e3e-9574-2c9cba1f2bc7"
- sonar-project.properties: expects coverage report at out/coverage/coverage.txt (sonar.go.coverage.reportPaths=out/coverage/coverage.txt)
- README.adoc: Codecov badge referenced for this repo
[::codeready-toolchain/host-operator::]
- make/test.mk: generates coverage to $(COV_DIR)/coverage.txt (excluding /cmd/manager) and upload-codecov-report target using codecov bash uploader with PR/base branching. (lines ~28-46, 52-54)
- make/test.mk: contains a hard-coded CODECOV_TOKEN := "e0747034-8ed2-4165-8d0b-3015d94307f9"
- README.adoc: Codecov badge referenced for this repo
[::codeready-toolchain/toolchain-e2e::]
- Multiple CI scripts and makefiles parse CLONEREFS_OPTIONS for PR metadata (jq '.refs[0].pulls[0].sha', author, head_ref, etc.), which matches the PR vs base commit logic used by the upload targets in other repos (scripts: scripts/ci/manage-operator.sh lines ~31-73; make/test.mk: has CLONEREFS_OPTIONS checks).
- make/ksctl.mk and other makefiles reference CLONEREFS_OPTIONS and extract author/branch info.
Summary / implications:
- The PR's Codecov onboarding aligns with existing patterns across repos: tests produce coverage at $(COV_DIR)/coverage.txt and upload using the codecov bash uploader, relying on CLONEREFS_OPTIONS to set PR vs base commit metadata.
- Important security/ops note: several repositories contain hard-coded CODECOV_TOKEN values in make/test.mk (api, toolchain-common, host-operator). These are visible in the repo files discovered above and should be reviewed/rotated or moved to CI secrets.
- sonar config in toolchain-common points to a different coverage path (out/coverage/coverage.txt) — confirm it matches the new Makefile OUT_DIR/COV_DIR layout or adjust accordingly.
| - name: Upload coverage artifact | ||
| uses: actions/upload-artifact@v7 |
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Show the workflow section around the referenced lines and all upload-artifact usages
echo "== .github/workflows/test.yml =="
if [ -f .github/workflows/test.yml ]; then
nl -ba .github/workflows/test.yml | sed -n '1,120p' | sed -n '20,70p'
echo "---- snippet around upload-artifact ----"
rg -n "actions/upload-artifact@" .github/workflows/test.yml || true
else
echo "File .github/workflows/test.yml not found"
fi
echo
echo "== All workflows using actions/upload-artifact@ in this repo =="
rg -n "actions/upload-artifact@" .github/workflows || trueRepository: codeready-toolchain/claw-operator
Length of output: 171
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== .github/workflows/test.yml =="
if [ -f .github/workflows/test.yml ]; then
sed -n '20,80p' .github/workflows/test.yml | awk '{printf "%d\t%s\n", NR+19, $0}'
echo "---- upload-artifact usages ----"
rg -n "actions/upload-artifact@" .github/workflows/test.yml || true
else
echo "File .github/workflows/test.yml not found"
fi
echo
echo "== All workflows using actions/upload-artifact@ in this repo =="
rg -n "actions/upload-artifact@" .github/workflows || trueRepository: codeready-toolchain/claw-operator
Length of output: 713
🏁 Script executed:
#!/bin/bash
set -euo pipefail
rg -n --hidden --no-messages "build/_output/coverage/coverage\.txt" . || true
rg -n --hidden --no-messages "coverage/coverage\.txt" Makefile .github/workflows . || true
ls -la Makefile Makefile.* 2>/dev/null || trueRepository: codeready-toolchain/claw-operator
Length of output: 780
🏁 Script executed:
#!/bin/bash
set -euo pipefail
echo "== Search exact coverage output path in repo =="
rg -n --hidden --no-messages "build/_output/coverage/coverage\.txt" . || true
echo
echo "== Search for coverage/coverage.txt and _output/coverage =="
rg -n --hidden --no-messages "coverage/coverage\.txt" . || true
rg -n --hidden --no-messages "_output/coverage" . || true
echo
echo "== Inspect Makefile coverage-related targets/vars =="
rg -n --hidden --no-messages "coverage|COV_DIR|_output" Makefile || trueRepository: codeready-toolchain/claw-operator
Length of output: 1496
Pin actions/upload-artifact to an immutable commit SHA.
.github/workflows/test.yml uses actions/upload-artifact@v7 at the coverage upload step (floating ref); pin to the exact commit SHA to avoid CI supply-chain drift.
🧰 Tools
🪛 zizmor (1.25.2)
[error] 30-30: unpinned action reference (unpinned-uses): action is not pinned to a hash (required by blanket policy)
(unpinned-uses)
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In @.github/workflows/test.yml around lines 29 - 30, The workflow step named
"Upload coverage artifact" currently uses the floating ref
actions/upload-artifact@v7; replace that with the exact immutable commit SHA for
the actions/upload-artifact repository (e.g.,
actions/upload-artifact@<full-commit-sha>) to pin the action. Locate the step by
the step name "Upload coverage artifact" or the reference
"actions/upload-artifact@v7", fetch the corresponding canonical commit SHA from
the actions/upload-artifact GitHub repo, and update the workflow entry to use
that full SHA instead of the tag.
alexeykazakov
left a comment
alexeykazakov
left a comment
There was a problem hiding this comment.
Thanks! We can also add a badge to README.md
But we can add it in the next PR after codecov is enabled.
2 participants
Description
Onboard CodeCov (action item from the Agentic Readiness discussion)
Summary by CodeRabbit