test: stop hardcoding the version literal in tests; enforce lockstep (#29) - #37
Conversation
…29) The per-plugin smoke tests and three stacks tests hardcoded the version string (`assert x.__version__ == "0.7.0"`), so every release turned the suite red until all nine were hand-edited -- a check that fails on every legitimate bump and passes on every wrong-but-consistent one. They now assert __version__ is well-formed semver, naming no value. The three redundant stacks version tests are deleted (smoke covers the property). tests/test_marketplace.py gains test_all_plugins_report_the_same_version, enforcing lockstep -- the guarantee the nine literals were reaching for -- without naming a version. Verified: bumping all twelve genuine carriers to a fake version keeps the suite green with zero test edits; desyncing one plugin turns the lockstep and agreement tests red. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
submtd
left a comment
submtd
left a comment
There was a problem hiding this comment.
Reviewed against keel's rubric. Ready to land.
Correctness. Every == "0.7.0" in the six smoke tests and three stacks tests is gone; smoke now checks the semver property, naming no value, and keel's VERSION constant is removed. The three stacks version tests were exact duplicates of the smoke check and are deleted, not weakened.
The new guarantee has teeth. test_all_plugins_report_the_same_version enforces lockstep — which nothing did before. Verified both directions: desyncing one plugin's version turns it and the existing agreement test red; bumping all twelve carriers consistently to a fake version keeps the suite green with zero test edits.
Scope. Test-only plus a changelog Changed note. Suite 1406 → 1404 (−3 duplicate tests, +1 lockstep). Good to land.
1 participant
Closes #29.
The problem
Nine test files hardcoded the version string (
assert bosun.__version__ == "0.7.0"). Every release turned the suite red — 15 failures on the way to 0.6.0 — until all nine were hand-edited. I just re-paid that exact tax cutting 0.7.0. And the check is worthless: it fails on every legitimate bump and passes on every wrong-but-consistent one.The change
test_smoke.py: version assertions now check the property —__version__exists, is a string, matches^\d+\.\d+\.\d+$— naming no value. keel'sVERSION = "..."constant is gone.test_stacks.py: thetest_<plugin>_versionfunctions were exact duplicates of the smoke check — deleted.tests/test_marketplace.py: gainstest_all_plugins_report_the_same_version— enforces lockstep (all six equal), the guarantee the nine literals were groping toward, without naming a version. Nothing enforced lockstep before this.Net effect
A release now edits only the twelve files that genuinely carry the version (6
plugin.json+ 6__init__.py). Zero test edits.Verification (both run, not committed)
9.9.9→ suite stayed green (1404) with zero test edits.Suite: 1406 → 1404 (−3 deleted duplicate version tests, +1 lockstep test). Test-only change; changelog notes it under Changed since lockstep enforcement is a stated suite property.
🤖 Generated with Claude Code