Repository navigation
Instrument ci.yml with shared-actions telemetry (DF12-98) - #498
Conversation
Start ci-telemetry-start after the checkout and report ci-telemetry-finish under if: always() in coverage-check, linux-full and windows-compat, pinned to shared-actions 81374b6f. Finish reads the sccache-stats.json the existing recorder writes and takes setup-rust's sccache-status. A contract holds placement, always(), the status wiring and the pin.
|
Note Reviews pausedIt looks like this branch is under active development. To avoid overwhelming you with review comments due to an influx of new commits, CodeRabbit has automatically paused this review. You can configure this behavior by changing the Use the following commands to manage reviews:
Use the checkboxes below for quick actions:
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
Summary
ValidationThe supplied evidence confirms the contract tests were added. It does not confirm test or lint results. WalkthroughAdd CI telemetry start and finish steps to ChangesCI telemetry reporting
Suggested labels: Priority: ⬇️ Low Change: Feature Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation | Resolution |
|---|---|---|---|
| Testing (Unit And Behavioural) |  |
FAIL: Add behavioural or end-to-end coverage for the new CI telemetry integration. The workflow now invokes ci-telemetry-start and ci-telemetry-finish in all three jobs (`.github/workflows/ci.yml:… |
Add an opt-in workflow integration test, using the existing act smoke-test harness or an equivalent runner, that executes the pinned telemetry actions through a representative ci.yml path. Assert one DF12_CI_TELEMETRY record, successf… |
| Docstring Coverage |  |
Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 1 files. (2 skipped: 2… | Write docstrings for the functions missing them to satisfy the coverage threshold. |
-
Check name Status Explanation Description check The description directly explains the CI telemetry changes, affected jobs, tests, documentation, and related work. Title check The title clearly identifies the main change: adding shared-actions telemetry to ci.yml. It also references DF12-98, which the description identifies as the related work item. Linked Issues check Check skipped because no linked issues were found for this pull request. Out of Scope Changes check Check skipped because no linked issues were found for this pull request. Testing (Overall) Pass the testing check. The added contract suite is substantive and targets the changed workflow behaviour. It selects exactly coverage-check,linux-fullandwindows-compat(tests/workflow_contr…User-Facing Documentation Pass this check. The pull request adds internal CI telemetry only. It does not change Whitaker's user-facing lint, installer, or library interfaces. The user's guide identifies the developer's guide a… Developer Documentation Pass. Accept the documentation update. docs/developers-guide.mdlines 836-848 documents the newci-telemetry-start/ci-telemetry-finishtooling, placement,if: always(), pinned shared action, `…Module-Level Documentation Pass this check. The new Python module has a module-level docstring at tests/workflow_contracts/ci_telemetry_test.py:1-16. It explains the module's purpose, its telemetry contract, its relationship …Testing (Property / Proof) Keep the parameterized contract tests; do not add a property test. The invariant covers a complete, fixed table of three ci.ymljobs, asserted explicitly at `tests/workflow_contracts/ci_telemetry_te…Testing (Compile-Time / Ui) Pass this check. The pull request adds no Rust or TypeScript compile-time behaviour. It wires an existing pinned shared action into YAML and adds focused workflow-contract assertions in `tests/workflo… Unit Architecture PASS — The PR adds explicit CI command steps, not query APIs. .github/workflows/ci.ymlstarts and finishes telemetry through pinned shared-actions at lines 87–88, 260–264, 278–279, 470–474, 542–543 …Domain Architecture Pass this check. The pull request changes only CI workflow instrumentation, workflow contract tests, and developer documentation. It does not change domain code, commands, repositories, adapters, or b… Observability PASS. The PR adds bounded CI resource telemetry at meaningful job boundaries. Each of the three jobs starts telemetry immediately after checkout and finishes under if: always()with the pinned share…Full details: Testing (Overall)
-
Explanation
Pass the testing check. The added contract suite is substantive and targets the changed workflow behaviour. It selects exactly
coverage-check,linux-fullandwindows-compat(tests/workflow_contracts/ci_telemetry_test.py:55-61), then parameterizes assertions for every job. It checks the start action immediately after checkout and exactly once (lines 64-70), the pinned finish action exactly once underif: always()with only housekeeping after it (lines 73-85), thesetup-ruststatus wiring (lines 88-94), and recorder-before-report ordering (lines 97-103). The workflow diff adds the corresponding start and finish steps for all three jobs (.github/workflows/ci.yml:87-88, 260-264, 278-279, 470-474, 542-543, 617-621). The suite is included bymake test-workflow-contracts(Makefile:255-258). These assertions fail for omission, wrong placement, wrong pin, non-guaranteed reporting, incorrect status wiring, and premature reporting. They do not duplicate tests for the shared action's internal implementation, which is outside this check's requirement.
Full details: User-Facing Documentation
-
Explanation
Pass this check. The pull request adds internal CI telemetry only. It does not change Whitaker's user-facing lint, installer, or library interfaces. The user's guide identifies the developer's guide as the documentation for Whitaker development, and the pull request documents the telemetry behaviour there at
docs/developers-guide.md:836-848. No user-facing documentation or migration note is required.
Full details: Developer Documentation
-
Explanation
Pass. Accept the documentation update.
docs/developers-guide.mdlines 836-848 documents the newci-telemetry-start/ci-telemetry-finishtooling, placement,if: always(), pinned shared action,DF12_CI_TELEMETRYrecord, sccache status and statistics wiring, fallback behaviour, Windows behaviour, and missing-record condition.tests/workflow_contracts/ci_telemetry_test.pyverifies the documented integration contract. The repository roadmap has no CI observability item, and the pull request adds no execplan or architecture decision that requires a Whitaker document update. The pinned shared-actions commit81374b6f183e59f0570ad36217ded90d3240fda6also contains the corresponding telemetry ADR and migration guidance.
Full details: Module-Level Documentation
-
Explanation
Pass this check. The new Python module has a module-level docstring at
tests/workflow_contracts/ci_telemetry_test.py:1-16. It explains the module's purpose, its telemetry contract, its relationship toci.yml, the collector,setup-rust, and the statistics recorder, and how to run it. The other changed files are workflow and documentation files, not code modules.
Full details: Testing (Property / Proof)
-
Explanation
Keep the parameterized contract tests; do not add a property test. The invariant covers a complete, fixed table of three
ci.ymljobs, asserted explicitly attests/workflow_contracts/ci_telemetry_test.py:55-61. The tests then apply the start, finish, status, and ordering checks to each job at lines 64-103. The workflow introduces no broad input space or generated state transitions; it adds the same telemetry pair to the three declared jobs at.github/workflows/ci.yml:87-88,260-264,278-279,470-474,542-543, and617-621.
Full details: Testing (Compile-Time / Ui)
-
Explanation
Pass this check. The pull request adds no Rust or TypeScript compile-time behaviour. It wires an existing pinned shared action into YAML and adds focused workflow-contract assertions in
tests/workflow_contracts/ci_telemetry_test.py:47-103. The structured telemetry output is implemented and tested in shared-actions commit81374b6f183e59f0570ad36217ded90d3240fda6: the marker test checks the marker, JSON schema and exact key set (.github/actions/ci-telemetry-finish/tests/test_ci_telemetry_summarize.py:245-289), and the end-to-end test checks one marker plus semantic output (.github/actions/ci-telemetry-finish/tests/test_ci_telemetry_finish.py:116-149). A duplicate snapshot in this caller would add little value and would risk brittleness from runner-specific fields.
Full details: Unit Architecture
-
Explanation
PASS — The PR adds explicit CI command steps, not query APIs.
.github/workflows/ci.ymlstarts and finishes telemetry through pinned shared-actions at lines 87–88, 260–264, 278–279, 470–474, 542–543 and 617–621. The finish action receivessetup-rust'ssccache-statusthrough an explicit input and runs underif: always(). At shared-actions commit81374b6f183e59f0570ad36217ded90d3240fda6, the composite actions isolate sampling, file access and log output behind a named action boundary (.github/actions/ci-telemetry-start/action.ymland.github/actions/ci-telemetry-finish/action.yml). No changed unit hides fallibility, dependencies or side-effects behind a pure read API.
Full details: Domain Architecture
-
Explanation
Pass this check. The pull request changes only CI workflow instrumentation, workflow contract tests, and developer documentation. It does not change domain code, commands, repositories, adapters, or business rules. The shared-actions telemetry calls remain in the CI infrastructure boundary and do not introduce transport, persistence, or vendor-specific concerns into the domain model.
Full details: Observability
-
Explanation
PASS. The PR adds bounded CI resource telemetry at meaningful job boundaries. Each of the three jobs starts telemetry immediately after checkout and finishes under
if: always()with the pinned shared action (.github/workflows/ci.yml:81-88, 260-264, 279, 470-474, 542-543, 617-621). The finish step receivessetup-rust'ssccache-statusafter the existing statistics recorder, so cache fallback and effectiveness remain diagnosable (.github/workflows/ci.yml:237-264, 447-474, 602-621;scripts/record-sccache-effectiveness.sh:1-24). The referenced shared action at commit81374b6f183e59f0570ad36217ded90d3240fda6emits oneDF12_CI_TELEMETRYrecord with CPU, memory, disk, job identity and sccache fields, uses null values for degraded sccache states, reports unsupported operating systems, and does not fail the job (leynos/shared-actions@81374b6f183e59f0570ad36217ded90d3240fda6:.github/actions/ci-telemetry-finish/action.yml:1-45;.../scripts/finish.py:1-20, 175-214, 260-309;.../scripts/sccache_stats.py:1-24, 337-385;.../scripts/summarize.py:1-22, 180-220). The record uses bounded metric fields and named CI identity fields, without customer payloads or secrets. No new alert or operational failure mode is introduced.
-
Full details: Testing (Unit And Behavioural)
-
Explanation
FAIL: Add behavioural or end-to-end coverage for the new CI telemetry integration. The workflow now invokes
ci-telemetry-startandci-telemetry-finishin all three jobs (.github/workflows/ci.yml:87-264,278-474, and542-621). The added tests only parse the workflow throughall_jobs()and inspect step dictionaries (tests/workflow_contracts/ci_telemetry_test.py:39-103). They verify useful placement and wiring invariants, but they never execute either composite action, emit or parseDF12_CI_TELEMETRY, exercise a failed job, or verify runtime sccache fallback behaviour. The existing shared-action tests cover the action repository, not this repository's caller integration.Resolution
Add an opt-in workflow integration test, using the existing
actsmoke-test harness or an equivalent runner, that executes the pinned telemetry actions through a representativeci.ymlpath. Assert oneDF12_CI_TELEMETRYrecord, successful reporting after a failing work step, and null sccache counts for fallback. Keep the static contract tests for coverage of all three jobs and retain the existing workflow test target.
Full details: Docstring Coverage
-
Explanation
Docstring coverage is 55.56% which is insufficient. The required threshold is 80.00%. Docstring coverage is scoped to functions touched by this diff. Analyzed 9 functions across 1 files. (2 skipped: 2 unsupported.)
Resolution
Write docstrings for the functions missing them to satisfy the coverage threshold.
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Start the record after checkout.
Finish it when each job ends.
Pass the sccache status through.
Check the steps and their order.
Document the metrics they send.
Comment @coderabbitai help to get the list of available commands.
Reviewer's GuideThe PR instruments coverage-check, linux-full, and windows-compat with commit-pinned shared-actions telemetry, wiring existing setup-rust and sccache outputs into a single always-run collector record; documentation and contract tests enforce the required placement, ordering, and configuration. Sequence diagram for CI telemetry collectionsequenceDiagram
participant Job as CI job
participant Start as ci-telemetry-start
participant Rust as setup-rust
participant Recorder as record-sccache-effectiveness.sh
participant Finish as ci-telemetry-finish
participant Collector as CI observability collector
Job->>Start: Start telemetry after checkout
Job->>Rust: Run setup-rust
Rust-->>Finish: sccache-status
Job->>Recorder: Write sccache-stats.json
alt Job succeeds or fails
Job->>Finish: Run with if: always()
Finish->>Finish: Read sccache-stats.json
Finish->>Collector: Print DF12_CI_TELEMETRY record
end
File-Level Changes
Tips and commandsInteracting with Sourcery
Customizing Your ExperienceAccess your dashboard to:
Getting Help
|
![codescene-access[bot]](https://avatars.githubusercontent.com/in/3755875?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard. |
|
The account paying for this security review has reached its Codex usage limits. The payer can check the Codex usage dashboard. For personal accounts, using credits requires enabling “Use credits for security reviews” in Code review settings. If you do not manage the paying account, contact this repository's admins. |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
|
@coderabbitai At head 3bdcad4, your walkthrough for 26edf68 showed one failed row, Testing (Unit And Behavioural), and the Docstring Coverage warning. The docstrings are added (every helper in ci_telemetry_test.py now has one). For the Testing row I argue it cannot be met the way the resolution asks, and that the behaviour it names is already proven, with evidence:
Does that clear the Testing (Unit And Behavioural) row, and is anything else blocking a merge at this head? |
|
Tip For best results, initiate chat on the files or code changes. I couldn't verify the requested revision for You are interacting with an AI system. |
|
@coderabbitai At head 3bdcad4, please assess the one failed row of your walkthrough for 26edf68, Testing (Unit And Behavioural), on the evidence in this pull request alone; no other repository needs to be read. The Docstring Coverage warning is fixed (every helper in ci_telemetry_test.py has a docstring). The row asks for an act run of the pinned telemetry actions. I argue that is not the right test here:
Does that clear the Testing (Unit And Behavioural) row, and is anything else blocking a merge at this head? |
|
Tip For best results, initiate chat on the files or code changes. 🧩 Analysis chain🏁 Script executed: #!/bin/bash
set -euo pipefail
git rev-parse HEAD
git status --short
fd '^ci_telemetry_test\.py$'
ast-grep outline tests/workflow_contracts/ci_telemetry_test.py --items all
git diff --stat 26edf681 3bdcad47
gh pr view 498 --repo leynos/whitaker --json headRefOid,state,mergeable,mergeStateStatus,reviewDecision,statusCheckRollup
gh api repos/leynos/whitaker/issues/498/comments --paginate --jq '.[] | select(.user.login == "coderabbitai[bot]") | {id,body}'
gh api repos/leynos/whitaker/pulls/498/reviews --paginate --jq '.[] | {user:.user.login,state,commit_id,body}'Length of output: 2530 🏁 Script executed: #!/bin/bash
set -u
sed -n '1,150p' tests/workflow_contracts/ci_telemetry_test.py
rg -n -C 5 'ci-telemetry|sccache-status|Record sccache|Check sccache|Record disk headroom|id: setup-rust|actions/checkout@' .github/workflows/ci.yml
gh pr view 498 --repo leynos/whitaker --json headRefOid,state,mergeable,mergeStateStatus,reviewDecision,statusCheckRollup
gh api repos/leynos/whitaker/issues/498/comments --paginate --jq '.[] | select(.user.login == "coderabbitai[bot]") | {id,body}'
gh api repos/leynos/whitaker/pulls/498/reviews --paginate --jq '.[] | {user:.user.login,state,commit_id,body}'
gh api graphql -f query='query { repository(owner:"leynos",name:"whitaker") { pullRequest(number:498) { reviewThreads(first:100) { nodes { isResolved comments(first:10) { nodes { author { login } body path } } } } } } }'Length of output: 44632 🏁 Script executed: #!/bin/bash
set -euo pipefail
gh run view 39064746392 --repo leynos/whitaker --json headSha,conclusion,event
gh run view 39064746392 --repo leynos/whitaker --log | grep -E 'DF12_CI_TELEMETRY|[0-9]+ passed|test-workflow-contracts' | head -n 45Length of output: 9097
I checked only this repository and this PR’s CI evidence:
The successful run does not demonstrate failed-job or fallback execution. However, those implementation behaviours do not justify a duplicate test requirement in this caller-only change. The contract tests verify the caller’s reporting condition and status input. I found no remaining code or test blocker in this assessment. GitHub reports Thanks for clarifying the test boundary. Linked revisions not inspected for this answer: ✏️ Learnings added
You are interacting with an AI system. |
3 participants
Part of DF12-98
Step 2.3.1 (whitaker half) of the CI observability plan: whitaker's
ci.ymljobs print the runner telemetry record that the collector reads. whitaker usessetup-rust(so it exercisessccache-status) and already writes sccache's JSON statistics, which makes it the pilot for the sccache half of the record. The actions are leynos/shared-actions#634 and leynos/shared-actions#636, pinned at leynos/shared-actions@81374b6. The collector side is https://github.com/leynos/grafana/pull/12, and the axinite half is leynos/axinite#444.What changes
coverage-check,linux-fullandwindows-compatstartci-telemetry-startright after the checkout and reportci-telemetry-finishunderif: always(), after the sccache health check and before the disk record and cache saves.windows-compatcarries the pair too; its record saysunsupported-osand has no CPU figures.setup-rust'ssccache-statusand reads thesccache-stats.jsonthatscripts/record-sccache-effectiveness.shalready writes (the action's default file), so sccache is not asked a second time and a fallback run carries null counts, never zeros. The existing record, upload and health-check steps are untouched.tests/workflow_contracts/ci_telemetry_test.pyholds the placement (start second, after the checkout),if: always(), that only housekeeping follows the report, the status wiring, that the recorder runs before the report, and the pin.ci.ymlhas no roll-up job, so there are no exemptions.Evidence
make test-workflow-contracts: 665 passed.actionlint .github/workflows/ci.ymlis clean.make check-fmt,make markdownlintandmake spellingpass. No Rust changed.Overlapping open pull requests
None is touched. Dependabot: #486, #477, #476. User: #372. Robot-house: #352. Those that edit
ci.ymlwill need a rebase.Summary by Sourcery
Instrument all ci.yml jobs with pinned shared-actions telemetry for centralized runner and sccache observability.
New Features:
Enhancements:
Documentation:
Tests: