release: v1.0.4 + fix write_firstrun_report version stamp - #12
Merged
Conversation
write_firstrun_report.py had its own "ACTION_VERSION = v1.0.0-rc1" that never got bumped, so every FirstRun gate_report.json shipped with a version string that was never released. Same bug class that scripts/_version.py was introduced to fix -- but the guard test only covered compare_runs and detect_test_set_drift, so the drift here slipped through v1.0.0..v1.0.3. Route write_firstrun_report through scripts/_version like the other two writers, and extend tests/test_action_version.py to enforce the invariant on all three so it cannot regress.
Cutting v1.0.4 (patch). Since v1.0.3 all changes are fixes or additive input options -- no breaking changes: #11 fix(compare): prefer newest per-generation dir; skip baseline gen output in warm-cache #10 fix(cache): warm assert-ai's artifact cache from the baseline so paired McNemar can reach PASS #9 fix(baseline): anchor _find_run_dir on suite.json for nested layouts #8 feat(baseline): event-aware default for baseline-branch #7 feat(gate): add run-timeout-minutes input to cap eval-step wall clock #6 feat(install): add assert-ai-ref and assert-ai-repo inputs for git installs Plus the v1.0.0-rc1 hardcode fix in the previous commit. tests/test_action_version.py will fail against local tags until v1.0.4 is tagged from this commit; that is expected per docs/release-procedure.md (bump then tag).
There was a problem hiding this comment.
Pull request overview
This PR cuts release v1.0.4 and fixes a latent bug where scripts/write_firstrun_report.py stamped FirstRun gate reports with a hardcoded, stale version instead of the shared action version constant.
Changes:
- Route
scripts/write_firstrun_report.pyto importACTION_VERSIONfromscripts/_version.py(matching the other report writers). - Extend
tests/test_action_version.pyto assert version-constant agreement across all three scripts and ensure the version stamp is written into FirstRun/compare gate reports. - Bump
scripts/_version.pyfromv1.0.3tov1.0.4.
Reviewed changes
Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.
| File | Description |
|---|---|
tests/test_action_version.py |
Expands version drift/stamping coverage to include write_firstrun_report and FirstRun stamping behavior. |
scripts/write_firstrun_report.py |
Removes hardcoded version and imports ACTION_VERSION from the shared version module. |
scripts/_version.py |
Updates the single source-of-truth action version to v1.0.4. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Cuts v1.0.4 (patch) and fixes a latent version-stamp bug so this bump actually reaches every gate report.
Changes
fix(version): stop hardcoding v1.0.0-rc1 in write_firstrun_reportscripts/write_firstrun_report.pythroughscripts/_version.ACTION_VERSIONlike its two siblings. Extendstests/test_action_version.pyso all three writers (compare_runs,detect_test_set_drift,write_firstrun_report) are covered — the exact drift that causedv1.0.0-rc1to survive across v1.0.0 → v1.0.3 in FirstRun reports can no longer happen.chore(release): bump ACTION_VERSION to v1.0.4v1.0.3 → v1.0.4.Why v1.0.4 (patch)
Since v1.0.3, six merged PRs — all fixes or additive input options, no breaking changes:
fix(compare): prefer newest per-generation dir; skip baseline gen output in warm-cachefix(cache): warm assert-ai's artifact cache from the baseline so paired McNemar can reach PASS_find_run_diron suite.json so nested layouts pair correctly #9fix(baseline): anchor_find_run_dironsuite.jsonfor nested layoutsbaseline-branch#8feat(baseline): event-aware default forbaseline-branchrun-timeout-minutesinput to cap eval-step wall clock #7feat(gate): addrun-timeout-minutesinputassert-ai-refandassert-ai-repoinputs #6feat(install): addassert-ai-refandassert-ai-repoinputs for git installsThe hardcoded-version bug
scripts/write_firstrun_report.pyhad its ownACTION_VERSION = "v1.0.0-rc1"that was never bumped. Every FirstRungate_report.jsonshipped stamped with a version that was never released — the same bug classscripts/_version.pywas introduced to eliminate. The guard test only coveredcompare_runsanddetect_test_set_drift, so this third writer drifted silently. Now covered.Testing
tests/test_action_version.py: 4/4 pass locally against commit 1.test_version_matches_the_newest_release_tagwill fail locally against commit 2 untilv1.0.4is tagged — that is the expected "bump then tag" invariant documented indocs/release-procedure.md. Will go green post-tag.Release checklist (after merge)
git tag v1.0.4 && git push origin v1.0.4(release workflow re-pointsv1automatically).gh release create v1.0.4 --repo responsibleai/assert-ai-action --title "ASSERT safety regression gate v1.0.4" --notes-file release-notes.md— from the exact semver tag, never fromv1.git rev-parse v1 == git rev-parse v1.0.4.Follow-up
tangym/fix-generation-alignment— its content was absorbed as squash-commit30ef6f5(PR fix(compare): prefer newest per-generation dir; skip baseline gen output in warm-cache #11).