docs(testing): master test plan — 17 chapters, 1,186 rows, a three-tier release gate - #60
Open
wshallwshall wants to merge 1 commit into
Open
docs(testing): master test plan — 17 chapters, 1,186 rows, a three-tier release gate#60wshallwshall wants to merge 1 commit into
wshallwshall wants to merge 1 commit into
Conversation
…three-tier gate The umbrella test plan the two existing artifacts plug into. FEATURE-COVERAGE-PLAN.md owns the per-subsystem coverage-gap audit and WIN2025-TEST-PLAN.md owns what only a real Server 2025 host can prove; neither covers the product as a whole, and neither covers the IDE, the tray, the Steps editor or the publish path at all. This states what the product is, what has to be true before it ships, and who proves it — delegating rather than restating, with foreign IDs prefixed FCP:/W25: because both predecessors use IDs that collide with this plan's own (HA-20 and API-13 exist in both spaces). Three things are load-bearing: Row classes. Only class-T rows gate a release. A criterion of "Pass = report received" or "Pass = a published number" cannot fail, so it cannot be a gate — those are class C (characterisation, tracked) and class A (external assurance, gates production exposure only). The draft had a penetration-test procurement event sitting at P0. The gate is now 196 blocking T-rows + 9 campaign-gated + 3 C + 3 A = 211 P0-marked, reconciled row-for-row against an independent parse. The publishing boundary. docs/BACKLOG.md is a published baseline ending at #231 and docs/security/ is gitignored post-cutover, so a citation above the baseline or into a withheld tree is valid evidence, not a dangling reference. An earlier pass had this backwards and was filing withheld material as coverage defects, including one that reached a release-gate criterion. Two owner-named items were structurally unprovable. The tray had zero P0 rows — all eight sat on NSSM/service/install — so the icon-state mapping, action routing, Win32 message pump and the lying-tray case are now blocking. HA had no P0 that proved a failover works; six now do. Also folds in surfaces nothing owned: last_resort.py (the PHI-egress backstop), netaddr.py (the shared allow-list decision the co-location exists to keep consistent), pki.py, the shipped PySide6 harness GUI, disk-full/ENOSPC, licence/copyleft compliance, destructive operator errors and non-ASCII patient data end to end. Ships as an index plus 20 chapter files: the assembled document is 1.8 MB, past GitHub's markdown render limit. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This was referenced Jul 30, 2026
wshallwshall
added a commit
that referenced
this pull request
Jul 30, 2026
PR #80 appended eight Steps-view items to docs/BACKLOG.md without the status banner every numbered item must carry, so tests/test_backlog_status_check.py::test_the_real_backlog_satisfies_the_invariant began failing on main itself (8 errors, first at line 6905). Because GitHub tests each PR merged into main, every open PR inherited the failure: #81, #74, #71, #66 and #60 were all blocked, three of them with auto-merge armed and unable to fire. Adds exactly one leading banner per item. Seven use the open/prioritized form; #239 uses the partial form, because its measurement ran and is recorded on PR #81 while the re-runnable scripts/quality/lens_coverage.py is still unmerged -- the number is not yet reproducible from main. The banners are deliberately minimal and do not re-litigate any item's verdict. #234 in particular is left explicitly unsettled rather than entrenched: it was filed as "revisit, not a bug" after the owner asked for a fix, and that framing is still open. Verified: scripts/docs/backlog_status_check.py exits 0 (237 items) and tests/test_backlog_status_check.py is 15 passed, was 14 passed 1 failed. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
wshallwshall
added a commit
that referenced
this pull request
Aug 2, 2026
That is the exact defect the backlog-hygiene gate exists for -- #60 shipped while its banner still read "PRE-RESERVED", and the stale banner was then repeated as fact in a merged PR. The gate did not fire here because it only inspects engine/IDE paths and this item is script-and-test work, which is precisely why the banner had to be corrected by hand rather than trusted to CI. Records what actually shipped (Half A) against what did not (Half B), and why Half B was backed out: claim.ps1 had three sessions in it at once, so the deferral is a coordination decision, not an estimate of difficulty. Someone reading this item needs that distinction to know it is pickup-able. The first draft of this named the contending BRANCH and the leak gate blocked it -- a worktree slug is internal naming and this ledger is public. Described by the code path it touches instead.
wshallwshall
added a commit
that referenced
this pull request
Aug 2, 2026
…OG #345) (#141) * fix(coord): a pruned worktree stranded its work claims forever (BACKLOG #345) A claim (scripts/coord/claim.ps1) is a JSON file under <git-common-dir>/mefor-coord/claims/. It lives beside the SHARED object store, so it outlives the worktree that took it. prune-merged.ps1 removed worktrees and, across 1,108 lines, never touched claims -- and `claim.ps1 -Take` hard-blocks on any claim file that exists. So every prune could leave a key unclaimable by every future session until a human happened to run `-Release <key> -Force`. Nothing surfaced the condition and nothing could: the registry has no way to observe that a holder ceased to exist. Released on EVIDENCE, never on a timer. claim.ps1 is right that an auto-expiring claim silently re-opens the race it exists to prevent; this hooks only into the branch that has already PROVEN the directory is gone and deregistered ("OUTCOME, NOT EXIT CODE"), so there is no session left to collide with. A merely-quiet holder is never touched. Dry-run safety is inherited, not re-asserted: the whole apply loop is gated on -Apply. The false positive is worse than the bug. Releasing a claim held by a different, LIVING worktree hands its key away and invites the duplicate build the registry exists to stop, so the match is full normalised equality via ConvertTo-Norm -- no leaf name, no prefix, no StartsWith. Two tests pin it: a living sibling's claim survives, and the PRIMARY checkout's claim survives a sibling prune even though `<primary>` is a strict prefix of `<primary>-clean`. Both carry a positive control in the same invocation, so neither can pass on a run that released nothing. Two defects found by the tests rather than by review, both of the shape this file already guards against elsewhere: * an unreadable claim was counted once per REMOVED worktree, so one blocked key reported as 2. It belongs to the registry, not to any worktree -- not being able to read it is precisely not knowing whose it is -- so it is now surveyed once per run and reported at run level. * that survey ran only inside the removal branch, making it invisible to a dry run, and its Set-Exit sat AFTER the -Json block that emits the receipt and exits. The receipt would have carried exitCode 0 over a key nothing can claim -- the surface CI reads. `claims.scanned` distinguishes "read the registry, found it clean" from "never looked" (no git-common-dir, or no claims directory): an empty `unreadable` list is not a green light. Verified: the 8 new tests FAIL against the unpatched script before they pass against the patched one, so they can see the class. Full file green, 71 tests. * docs(backlog): #345's banner said "not started" on the PR that builds it That is the exact defect the backlog-hygiene gate exists for -- #60 shipped while its banner still read "PRE-RESERVED", and the stale banner was then repeated as fact in a merged PR. The gate did not fire here because it only inspects engine/IDE paths and this item is script-and-test work, which is precisely why the banner had to be corrected by hand rather than trusted to CI. Records what actually shipped (Half A) against what did not (Half B), and why Half B was backed out: claim.ps1 had three sessions in it at once, so the deferral is a coordination decision, not an estimate of difficulty. Someone reading this item needs that distinction to know it is pickup-able. The first draft of this named the contending BRANCH and the leak gate blocked it -- a worktree slug is internal naming and this ledger is public. Described by the code path it touches instead. * docs(worktrees): the prune receipt grew a claims surface and nothing described it WORKTREES.md is where the pruner's outcome contract is written down -- exit codes, which counts are subsets of which, why `git worktree prune` is never run. A new field in that receipt that only the source explains is the same drift this section already guards against elsewhere. States the property that makes the release safe rather than just the behaviour: it fires on PROVEN deletion, so a quiet holder is untouched and a dry run releases nothing, and the match is full path equality because freeing a LIVING worktree's key is worse than the orphan being cleaned up. * style(worktree): my #345 comments put 13 non-ASCII em-dashes in a BOM-less script prune-merged.ps1 had exactly ZERO non-ASCII bytes before this branch and no BOM -- every comment in it uses the ASCII `--`. A BOM-less file whose encoding is only inferred is a bad place to start introducing multi-byte characters: pwsh 7 assumes UTF-8 and would be fine, Windows PowerShell 5.1 assumes the ANSI codepage and would not, and nothing in the repo pins which one an operator runs it with. The file is back to 0 non-ASCII bytes. Comments only; CRLF line endings verified byte-identical in count before and after, and the claim tests re-run green.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
The umbrella test plan the two existing artifacts plug into.
FEATURE-COVERAGE-PLAN.mdowns the per-subsystemcoverage-gap audit;
WIN2025-TEST-PLAN.mdowns what only a realServer 2025 host under the NSSM service identity can prove. Neither covers the product as a whole,
and neither covers the IDE, the tray, the Steps editor or the publish path at all.
Ships as
docs/testing/MASTER-TEST-PLAN.md(index) plus 20chapter files under
docs/testing/master-test-plan/— the assembled document is 1.8 MB, pastGitHub's markdown render limit. Every chapter carries scope, what is already covered and must not
be re-tested, a risk analysis, the test matrix, detailed scenarios, an automation disposition,
prerequisites, exit criteria and open questions.
Foreign IDs are prefixed
FCP:/W25:throughout, because both predecessors use IDs that collidewith this plan's own —
HA-20andAPI-13exist in both spaces.What is load-bearing
Row classes. Only class-T rows gate a release. A criterion of "Pass = report received" or
"Pass = a published number" cannot fail, so it cannot be a gate — those are class C
(characterisation, tracked) and class A (external assurance, gates production exposure only).
The draft had a penetration-test procurement event sitting at P0. The gate is now 196 blocking
T-rows + 9 campaign-gated + 3 C + 3 A = 211 P0-marked, reconciled row-for-row against an
independent parse of all 17 matrices.
The publishing boundary.
docs/BACKLOG.mdis a published baseline ending at## 231.anddocs/security/is gitignored post-cutover, so a citation above the baseline or into a withheld treeis valid evidence, not a dangling reference. An earlier pass had this backwards and was filing
withheld material as coverage defects — including one that had reached a release-gate criterion.
Two owner-named items were structurally unprovable. The tray application held zero P0 rows
(all eight sat on NSSM/install) even though ADR 0113 names the untested Win32 layer as the design's
main risk; the state→icon map, action routing, message-pump dispatch and the "lying tray" are now
blocking. HA had no P0 proving a failover works — six now do, all on
container-CIwith aservice container, none needing a second host.
Surfaces nothing previously owned
last_resort.py(the process-level PHI-egress backstop),netaddr.py(the shared allow-listdecision its co-location exists to keep consistent),
pki.py, the shipped PySide6 harness GUI —which is both a distribution and the plan's only route to hostile-input injection — plus disk-full /
ENOSPC, licence and copyleft compliance, destructive operator errors, and non-ASCII patient data
end to end.
Review notes
whether
load_settingsshould refuse unknown keys, which transports are deliberately notauthorable from
connections.toml).messagefoundry checkis invoked by no workflow, and onemeta-test plus one
ci.ymlregex fix turns 36 already-written tests from dormant intoexecuting. That is items 1–2 of §24's prioritised ten.
🤖 Generated with Claude Code