From 0779b64444f36c9650d635474fa258d0cde326c5 Mon Sep 17 00:00:00 2001 From: sotashimozono Date: Thu, 20 Aug 2026 03:45:34 +0000 Subject: [PATCH 1/2] fix(upload-coverage): an artifact that was never created is not a coverage bug MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `Coverage (upload)` went red at the DOWNLOAD, before anything about coverage was attempted: Unable to download artifact(s): Artifact not found for name: testshards-lcov This download is by `name:`, which hard-fails on absence — unlike the workflow's own `pattern:` download, which succeeds with zero results. That asymmetry is why `collect` gets as far as its completeness gate under a full quota while this job dies at step two. An artifact that was never created is an infrastructure state, not a producer error: the Actions storage quota is enforced at `CreateArtifact` **before any bytes are sent**, so it refuses a 0-byte artifact exactly as it refuses a large one, and nothing here can be fixed by failing. Losing the coverage REPORT is the correct degradation; taking the run down with it is not. ABSENT and EMPTY now get different treatment, because they are different claims: absent -> a warning naming the likely cause, a step-summary line saying coverage was not reported, and exit 0. The upload is skipped rather than sent empty. empty -> still fatal. Something produced the artifact and put no report in it, which is a producer bug and the message it already had. Verifiable only in a real run — the change is three conditionals and the state it handles is one the action cannot manufacture locally. The fleet exercises it immediately, since every private consumer is currently in exactly this state. Co-Authored-By: Claude Opus 5 --- Project.toml | 2 +- actions/upload-coverage/action.yml | 28 +++++++++++++++++++++++++--- 2 files changed, 26 insertions(+), 4 deletions(-) diff --git a/Project.toml b/Project.toml index 11716be..e74f6b2 100644 --- a/Project.toml +++ b/Project.toml @@ -1,6 +1,6 @@ name = "TestShards" uuid = "acceef1d-f5e0-4fe4-a546-818dc56ce7b2" -version = "0.3.37" +version = "0.3.38" authors = ["sota shimozono "] [deps] diff --git a/actions/upload-coverage/action.yml b/actions/upload-coverage/action.yml index 7f5cc04..3301d34 100644 --- a/actions/upload-coverage/action.yml +++ b/actions/upload-coverage/action.yml @@ -116,18 +116,40 @@ runs: with: submodules: ${{ inputs.submodules }} - - uses: actions/download-artifact@v8 + # ABSENT and EMPTY are different states and get different treatment. This download is by + # `name:`, which HARD-FAILS when the artifact does not exist — unlike the workflow's own + # `pattern:` download, which succeeds with zero results. So an account that cannot create + # artifacts at all turned this job red at the download, before anything about coverage was + # even attempted: + # + # Unable to download artifact(s): Artifact not found for name: testshards-lcov + # + # An artifact that was never created is an infrastructure state — the run's own storage quota + # is enforced at `CreateArtifact`, before any bytes are sent, so no producer error is + # involved and there is nothing here to fix by failing. Losing the coverage REPORT is the + # correct degradation; losing the TEST RESULT to it is not. + - id: fetch + continue-on-error: true + uses: actions/download-artifact@v8 with: name: ${{ inputs.artifact }} - shell: bash run: | set -euo pipefail - [ -s lcov.info ] || { echo "::error::${{ inputs.artifact }} contained no lcov.info — did the sharded run finish with coverage on?"; exit 1; } + if [ "${{ steps.fetch.outcome }}" != "success" ]; then + echo "::warning::${{ inputs.artifact }} does not exist, so there is no merged report to send. Coverage is not reported for this run; the tests themselves are unaffected. The usual cause is that the run could not create artifacts at all — an Actions storage quota is enforced at CreateArtifact, before any bytes are sent, so it stops a 0-byte artifact exactly as it stops a large one." + echo "### Coverage" >> "$GITHUB_STEP_SUMMARY" + echo "Not reported: \`${{ inputs.artifact }}\` was not produced by the run." >> "$GITHUB_STEP_SUMMARY" + exit 0 + fi + # PRESENT BUT EMPTY is a different thing entirely: something produced the artifact and put + # no report in it. That is a producer bug and stays fatal. + [ -s lcov.info ] || { echo "::error::${{ inputs.artifact }} exists but contained no lcov.info — did the sharded run finish with coverage on?"; exit 1; } echo "Merged report: $(grep -c '^SF:' lcov.info) source files." - uses: codecov/codecov-action@v7 - if: ${{ steps.token.outputs.have == 'true' }} + if: ${{ steps.token.outputs.have == 'true' && steps.fetch.outcome == 'success' }} continue-on-error: ${{ inputs.fail-on-error != 'true' }} with: files: lcov.info From e57321af6a7a5c55fcb48c5f647512b475748e0a Mon Sep 17 00:00:00 2001 From: sotashimozono Date: Thu, 20 Aug 2026 04:33:27 +0000 Subject: [PATCH 2/2] review: test the branch this exists for, and stop asserting a cause it has not established MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Three problems with the first version, found by reviewing it against what its own CI actually exercised. 1. NOTHING TESTED THE BRANCH. Every check was green, and not one of them reached the state the change exists for: `action-smoke` runs where the artifact IS present. I wrote "verifiable only in a real run" in the PR body, and that was a way of not looking — `action-smoke` already uses a local `./actions/upload-coverage`, so a PR can exercise its own version of the action. Naming an artifact that cannot exist reproduces the state exactly. And the thing under test is not the message. It is whether `continue-on-error` on a step of a COMPOSITE action is honoured at all, and whether `steps..outcome` is readable from a later step of the same composite. If either is not true, the action dies at the download exactly as before and the whole change is inert — with every check still green, because nothing looked. `action-absent-artifact` is now the only thing that would say so. This repository is PUBLIC, so its storage is free and its own CI can never reach that state by accident. The state has to be constructed. 2. THE MESSAGE ASSERTED A CAUSE. `outcome != 'success'` is every reason a download can fail, not only absence, so "does not exist" was a claim the step had not established — a network fault and a genuinely lost `-lcov` land in the same branch. It now says the download failed, names what usually does that, and tells the reader to check whether the run produced any artifacts at all before assuming the quota. 3. THE TRADE WAS HIDDEN BEHIND A TIDY DISTINCTION. "Absent is infrastructure, empty is a bug" reads cleanly and is not airtight: absent can also be a bug, because the workflow's `-lcov` upload is itself `continue-on-error`. So a real producer regression now degrades to a warning where it used to be red. That is accepted — a storage condition must not take down every consumer's test result — but it is written down as an accepted cost rather than defined away. Also `${GITHUB_STEP_SUMMARY:-}`: the block runs under `set -u`, where an unbound variable would kill the step in the middle of degrading gracefully. Co-Authored-By: Claude Opus 5 --- .github/workflows/CI.yml | 21 +++++++++++++++++++++ actions/upload-coverage/action.yml | 20 +++++++++++++++++--- 2 files changed, 38 insertions(+), 3 deletions(-) diff --git a/.github/workflows/CI.yml b/.github/workflows/CI.yml index 8bf49e5..24f1769 100644 --- a/.github/workflows/CI.yml +++ b/.github/workflows/CI.yml @@ -121,6 +121,27 @@ jobs: codecov-token: ${{ secrets.CODECOV_TOKEN }} dry-run: true + # The state this action exists to survive, and the one its own CI never reaches on its own: the + # merged report is NOT there. Every consumer is in it right now — an Actions storage quota is + # enforced at `CreateArtifact` before any bytes are sent, so nothing gets created — and this + # repository is public, so its storage is free and it can never reproduce that by accident. + # + # Naming an artifact that cannot exist reproduces it exactly. What is under test is not the + # message: it is that `continue-on-error` on a step of a COMPOSITE action is honoured at all, and + # that `steps..outcome` is readable from a later step of the same composite. If either is not + # true the action dies at the download exactly as before, and this job is the only thing that + # would say so. + action-absent-artifact: + name: The upload action survives a missing report + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v7 + - uses: ./actions/upload-coverage + with: + codecov-token: ${{ secrets.CODECOV_TOKEN }} + artifact: testshards-lcov-deliberately-absent + dry-run: true + # The sharded path runs on one Julia version. Compatibility across the supported range is a # different question, so it is asked separately and unsharded — a version matrix times a # shard matrix would be 8N jobs to answer a question that does not need the split. diff --git a/actions/upload-coverage/action.yml b/actions/upload-coverage/action.yml index 3301d34..58eb9a9 100644 --- a/actions/upload-coverage/action.yml +++ b/actions/upload-coverage/action.yml @@ -134,13 +134,27 @@ runs: with: name: ${{ inputs.artifact }} + # THE TRADE THIS MAKES, SAID OUT LOUD. `outcome != 'success'` is every reason the download can + # fail, not only absence — a network fault and a genuinely lost `-lcov` land here too. So a + # real regression in the producer degrades to a warning where it used to be red. That is + # accepted because the alternative is worse (a storage condition taking down every consumer's + # test result) and because the step summary carries it where a warning alone would be missed. + # What is NOT accepted is claiming a cause this step has not established: it says the download + # failed and lists what usually does that, rather than asserting the artifact does not exist. - shell: bash run: | set -euo pipefail if [ "${{ steps.fetch.outcome }}" != "success" ]; then - echo "::warning::${{ inputs.artifact }} does not exist, so there is no merged report to send. Coverage is not reported for this run; the tests themselves are unaffected. The usual cause is that the run could not create artifacts at all — an Actions storage quota is enforced at CreateArtifact, before any bytes are sent, so it stops a 0-byte artifact exactly as it stops a large one." - echo "### Coverage" >> "$GITHUB_STEP_SUMMARY" - echo "Not reported: \`${{ inputs.artifact }}\` was not produced by the run." >> "$GITHUB_STEP_SUMMARY" + echo "::warning::could not download ${{ inputs.artifact }}, so there is no merged report to send. Coverage is not reported for this run; the tests themselves are unaffected. The usual cause is that the run could not create artifacts at all — an Actions storage quota is enforced at CreateArtifact, before any bytes are sent, so it stops a 0-byte artifact exactly as it stops a large one. A lost upload or a transient fault reach this branch too, so check whether the run produced any artifacts at all before assuming the quota." + # `${GITHUB_STEP_SUMMARY:-}` because this runs under `set -u`, and an unbound variable + # here would kill the step in the middle of degrading gracefully — turning the fallback + # into a second, more confusing failure. + if [ -n "${GITHUB_STEP_SUMMARY:-}" ]; then + { + echo "### Coverage" + echo "Not reported: \`${{ inputs.artifact }}\` could not be downloaded." + } >> "$GITHUB_STEP_SUMMARY" + fi exit 0 fi # PRESENT BUT EMPTY is a different thing entirely: something produced the artifact and put