Skip to content

the refusal to explain a run that has not concluded is measured by no test #2712

Closed

Description

@how2how2how2-arch

scripts/read-run-failure.py reads a run's own conclusion to decide whether a failure exists at all. A run that has not finished carries conclusion: null, and the tool raises rather than accepting that as a verdict, so a still-running run is answered as unmeasured — exit 2 — which is the right answer.

Nothing measures that branch. Deleting the null check leaves every one of the file's 31 tests green: the single test that reaches this path hands the reader a transport error (run_rc=1, an HTTP 502) and never a payload whose conclusion is null.

What the deletion would cost. str(None) is the string "None", which is not "success", so the run would fall into the branch that explains a failed run. A run that is still queued or in progress would then be reported as "the run itself concluded None, and the job(s) that account for it are: …" — a determination about a run that has reached none. That is the defect this area just had removed, one step further along: an unmeasured question answered as a verdict.

Observed live on run 41938836857 while it was still running: the tool exits 2 and says "whether this run failed is exactly the question, so it is unmeasured rather than 'nothing to explain'". The behaviour is correct; the measurement of it is what is missing.

What finishing this needs:

  • A test whose run payload carries conclusion: null, asserting exit 2, the word unmeasured and the "no conclusion yet" reason on stderr, and that no explanation of a failure is printed on stdout.
  • The fixture must be able to model a run that has not finished, so the reason the tool prints names the state such a run actually has (in_progress) rather than completed.
  • Deleting the null check must fail that test — that mutation is the acceptance criterion, not a courtesy.

Activity

  1. how2how2how2-arch commented on Nov 5, 2026

    @how2how2how2-arch
    CollaboratorAuthor

    Handled by #2713

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions