test: add deterministic token-count baseline for command files (#87)#94
Merged
Merged
Conversation
Add tests/token-baseline.sh — a deterministic size census of commands/*.md (lines, bytes, ~tokens ≈ bytes/4) with a committed snapshot at tests/fixtures/token-baseline.txt, so the v0.14.0 compression milestone (#89-#92) can prove per-command reductions and catch accidental bloat without an LLM. - --check prints the table + drift vs snapshot, ALWAYS exits 0 (informational; a bloat hard-fail guard is deferred per gate1) - --update refreshes the snapshot - CR stripped before counting, and commands/*.md, the snapshot, and *.sh pinned to LF via .gitattributes, so byte counts are reproducible across Windows/WSL and Linux CI - tests/token-baseline-test.sh self-tests the census tool; both wired into .github/workflows/lint.yml (self-test gates, --check is informational) - CONTRIBUTING.md documents the refresh procedure Baseline at this commit: TOTAL 4573 lines / 313710 bytes / ~78424 tokens (start.md ~21277, ship.md ~12214 are the top cost centers). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Gate2 (qa-lead) flagged that the tool's core contract — AC #3, "snapshot drift must still exit 0 (informational) and warn" — was only verified in the match case, never under actual drift. Add a drift-case assertion: append a sentinel row to the snapshot, run --check, assert exit 0 AND a "WARN: size drift" line (captured from stderr), then restore the snapshot via --update. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The scripts were committed mode 100644 because `chmod +x` on Windows does not set git's executable bit. CI invokes them via `bash` (which works), but the self-test's `[ -x ]` contract assertion correctly failed on a clean Linux checkout — the repo convention is executable tests/*.sh (rwxr-xr-x). Set the git exec bit via `git update-index --chmod=+x` on both scripts (blob content unchanged, mode 100644 -> 100755). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
JFK
added a commit
that referenced
this pull request
Jun 13, 2026
The v0.14.0 "token + precision optimization" milestone was re-audited against the actual files (using the token-baseline tool from #94). The premise turned out to be largely invalid: - The "~28% / ~6,400 token" compression is not achievable: slash commands load whole (no runtime include / conditional load), so relocating sections to an appendix saves nothing, and the bulk of start.md/ship.md is load-bearing executable spec that must not be compressed. - The claimed precision bugs were phantom: step-18b precedence is already an If/Else-if chain; verdict last-wins is already explicit (and now test-guarded by #95); the propose.md "parallel Skill" instruction is correct (batched Skill calls are supported); the propose.md "regex mismatch" is a harmless subset, not a contradiction. This commit ships the ONLY verified-safe, genuinely-beneficial residue: - start.md: delete a verbatim-redundant `lang != "en"` localization line (649) that duplicated line 647. - goal.md: convert the red-verdict force-continue prose (phase-aware bullets) into a compact decision table, preserving every load-bearing detail (the gate2.binary_gate `fail` exception, phase routing, continue-to steps). Net effect (per tests/token-baseline.sh): TOTAL ~78,424 -> ~78,355 tokens (-69 tokens, -0.09%). The negligible number is itself the finding — it demonstrates the milestone's compression premise was unfounded, and the token-baseline tool (#87/#94) measuring it is working as intended. Closes #89 Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #87
Summary
tests/token-baseline.sh— a deterministic size census ofcommands/*.md(lines / bytes / ~tokens ≈ bytes/4) with a committed snapshot attests/fixtures/token-baseline.txt. This is the measurement floor for the v0.14.0 compression milestone (refactor: optimize start.md — compress ~28% + fix dry-run/verdict/step-18b precision bugs #89–refactor: optimize config/doctor/status — PMRP->appendix, schema->bullets #92): prove per-command reductions and catch accidental bloat, with no LLM/tokenizer.--checkprints the table + drift vs the snapshot and always exits 0 (informational; a bloat hard-fail guard is deferred per gate1).--updaterefreshes the snapshot.tests/token-baseline-test.shself-tests the tool (non-vacuous: row-count,~tokens==bytes/4,--updatesha1 idempotence, OK-match, and the drift→exit-0+WARN contract).Implementation notes
CRbefore counting, andcommands/*.md, the snapshot, and*.share pinned to LF via.gitattributes. (The working tree was CRLF — rawwc -cdiffered from the normalized count, so this is load-bearing for the milestone's "prove reduction" goal.).github/workflows/lint.yml: the self-test gates CI;--checkruns as an informational step.CONTRIBUTING.mddocuments the refresh procedure.start.md~21277,ship.md~12214).Pre-PR review summary
gate2.binary_gate= none)Full reviews are saved in the plugin cache (
<branch-flat>.gate1.md/.gate2.md).🤖 Generated via /gh-issue-driven:ship (autonomous=red-only, milestone v0.14.0)