Part I — Quarterly reruns: history on the page, PR instead of push, staleness report - #11
Merged
Merged
Conversation
…report (Part I)
`.github/workflows/matrix.yml`: the scheduled run now covers every suite this
repository publishes (`otlp-logs`, `es-bulk`, `remote-write`, `otlp-metrics`,
`loki-push`), and instead of only uploading an artifact, opens a pull request
adding `results/<date>/<suite>/matrix.{json,md,html}` — never pushed straight
to main, since bumping every image tag and re-confirming each adapter by hand
is a maintainer's own step, not something a workflow can do for them.
`Matrix::to_html` (`src/matrix.rs`) gains `history: Option<&History>`: given
the most recent prior run of the same suite, a cell whose verdict changed
says so and links back to it — the strongest evidence this project can offer
that a finding was acted on is a cell moving from `ALTER` to `PASS` with the
version it changed at. `find_previous_run` (`src/main.rs`) locates that prior
run under `results/`, picking the most recent one strictly before the run
being rendered, so a same-day rerun never compares against itself. Wired into
both `render` and `matrix`.
`tools/check_confirmation_age.py`: reads the "Confirmed by hand against
`<image>` on `<date>`" comment every adapter already opens with (rather than
adding a second machine-only field that could drift from it) and reports how
old each one is. Never fails the job — a stale adapter needs a maintainer's
eyes, not a red build — and its report goes into the quarterly pull request
body, so a backend that could not be re-confirmed this quarter is shown with
its last date rather than silently carried forward as current. Run against
this repository's own adapters, it correctly flagged `parseable.yaml`: an
0.2-era adapter that predates the confirmation-comment convention and
genuinely has no date to report — left as found rather than backfilled with
a guessed date, which this project's own rule against confirming from memory
forbids.
Verified: unit tests for `find_previous_run` (most-recent-before-today
chosen, same-day never compares against itself, wrong suite never matches,
no `results/` directory at all is simply no history) and for the HTML diff
annotation; and an end-to-end run through the actual `render` CLI against two
hand-built quarters, confirming the link and the "changed from ALTER" text
appear in the real output, not only in a unit test's assertions.
174 tests, 0 clippy warnings, corpus gate passes.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018k65nFSzuwsHwYnpSHNaeK
DeviousCardi
force-pushed
the
quarterly-reruns
branch
from
September 9, 2026 08:30
62445de to
8f99197
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Part I of PLAN-1.0.md. Branched from
maindirectly (not fromci-action),so this PR is independent of #10 and can be reviewed/merged in either order.
.github/workflows/matrix.yml— the scheduled run now covers everysuite this repository publishes, and instead of only uploading an
artifact, opens a pull request adding
results/<date>/<suite>/matrix.*—never pushed straight to main, since bumping every image tag and
re-confirming each adapter by hand is a maintainer's own step.
Matrix::to_html(src/matrix.rs) gainshistory: Option<&History>:a cell whose verdict changed from the most recent prior run of the same
suite says so and links back to it.
find_previous_run(src/main.rs)locates that prior run under
results/, picking the most recent onestrictly before the run being rendered.
tools/check_confirmation_age.py— reads the "Confirmed by handagainst
<image>on<date>" comment every adapter already opens with,and reports how old each one is. Never fails the job; its report goes into
the quarterly pull request body. Run against this repository's own
adapters, it correctly flagged
parseable.yaml: an 0.2-era adapter thatpredates the confirmation-comment convention and genuinely has no date to
report — left as found rather than backfilled with a guessed date.
Verified: unit tests for
find_previous_runand the HTML diff annotation,plus an end-to-end run through the actual
renderCLI against two hand-builtquarters, confirming the link and the "changed from ALTER" text in the real
output.
172 tests (this branch, standalone), 0 clippy warnings, corpus gate passes.
🤖 Generated with Claude Code
https://claude.ai/code/session_018k65nFSzuwsHwYnpSHNaeK