Repository navigation
docs: narrow the v0.5 Layer B gate and accept the attestation cost - #22
Conversation
S7 scoping for PDN-0004, proposed for owner review: - The Layer B release gate keeps one real-host scenario per invariant: F1, F2, F11, F18 and F27. Layer A already runs every scenario in CI against the reference host. Layer B only has to show that a real host follows the prose. F4, F8, F10 and F12-F15 move to post-release evidence. - Owner attestation is the v0.5 cost. The S6 measurements show that no shipped executor gives safe machine stop evidence on macOS, so every delegated attempt ends with a person confirming no process remains. v0.5 documents this instead of weakening §8.3; lowering it is v0.6 work. - In Layer B the owner attests by hand; the harness never records owner-attested itself. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
vunm-dev
left a comment
There was a problem hiding this comment.
One release-gate coverage issue needs adjustment. Keeping explicit human stop attestation and documenting the limitations of historical evidence is consistent with the existing protocol. Validation: all 32 documentation checks passed, the diff has no whitespace errors, and GitHub CI is green. Recommendation: narrow the gate to six scenarios, retaining F14 alongside the proposed five.
| The other Layer B scenarios (F4, F8, F10, F12–F15) move to post-release | ||
| evidence under `docs/evidence/v0.5.x/`. |
There was a problem hiding this comment.
[P2] Keep F14 in the real-host release gate
C6 recovery relies on host behavior that the helper does not enforce: projected checks the persisted accepted verdict, checkbox and attempt line, but does not recompute the task/artifact digests or rerun verification. If the task or working tree changes after a crash between verdict and projection, a real host that skips the prose's rechecks can still project the historical verdict and mark the task complete. None of F1, F2, F11, F18 or F27 exercises this recovery branch; Layer A only proves that the reference host performs the rechecks. Retain F14 in the gate, including changed-task and changed-artifact cases after the verdict, so the narrower gate still tests this distinct false-acceptance path.
There was a problem hiding this comment.
Agreed; fixed in ef0bd54. I confirmed on the helper that cmd_projected checks only the persisted verdict, the checkbox and the Dispatched: … attempt: line. It neither recomputes the digests nor reruns a check, so the §13.3 re-checks are host prose that only a real host can be shown to follow.
- F14 is back in the gate, which now has six scenarios: F1, F2, F11, F14, F18, F27.
- The real-host run covers both the changed-task case (the verdict stays historical) and the changed-artifact case (digest mismatch), as the reference host already does in F14b and F14c.
- The post-release list is now F4, F8, F10, F12, F13, F15.
A deterministic alternative would be for projected to recompute both digests itself. That would move this guard out of prose, but it is a helper change, so I would take it as a separate follow-up only if you want it.
Review on #22: C6 recovery depends on host prose the helper does not enforce. `projected` checks the persisted verdict, the checkbox and the attempt line, but it does not recompute the task or artifact digest or rerun any check. A real host that skipped the §13.3 re-checks could project a historical verdict, and none of the other five gate scenarios exercises that path. F14 goes back into the gate, with the changed-task and changed-artifact cases, making six scenarios. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven tasks for the release slice under the gate narrowed in #22: isolate the Layer B host, build the Layer B fixture, stage markers and file oracles, write the scenario scripts, run Layer B with the owner, write the release docs, and cut the release. Task 1.1 is done: a claude -p host loaded both the 0.4.0 plugin and the direct v0.5 copies, so the owner moved Claude to the beta plugin channel pinned to release/v0.5.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/harness/layer-b/ runs the gate scenarios of #22 (F1, F2, F11, F14a, F14b, F18, F27) against a real `claude -p` host with the v0.5 skills: - fixture.sh builds each scenario's workspace: a plan, an AGENTS.md with an owner routing policy, and an executor card for the test double in the scenario's mode. F27 routes to an executor that has no card. - run.sh kills the host at the scenario's crash marker (after-arm, after-result, after-verdict), applies the scenario's edit, and opens a new session for pickup. It relays the owner's answers verbatim and never answers or attests for them. - collect.sh and oracle.sh judge a run from files only: contract, acceptance by ground truth, final-tree digests, stop evidence, no second writer, no overlapping worker lifetimes. A pilot F1 run showed the host stopping to ask for attestation, and the oracle failed that unfinished state as it should. A smoke F27 run passed all checks. Neither counts as evidence; plan task 3.1 runs the gate with the owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Review on #22: C6 recovery depends on host prose the helper does not enforce. `projected` checks the persisted verdict, the checkbox and the attempt line, but it does not recompute the task or artifact digest or rerun any check. A real host that skipped the §13.3 re-checks could project a historical verdict, and none of the other five gate scenarios exercises that path. F14 goes back into the gate, with the changed-task and changed-artifact cases, making six scenarios. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Seven tasks for the release slice under the gate narrowed in #22: isolate the Layer B host, build the Layer B fixture, stage markers and file oracles, write the scenario scripts, run Layer B with the owner, write the release docs, and cut the release. Task 1.1 is done: a claude -p host loaded both the 0.4.0 plugin and the direct v0.5 copies, so the owner moved Claude to the beta plugin channel pinned to release/v0.5.0. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
tests/harness/layer-b/ runs the gate scenarios of #22 (F1, F2, F11, F14a, F14b, F18, F27) against a real `claude -p` host with the v0.5 skills: - fixture.sh builds each scenario's workspace: a plan, an AGENTS.md with an owner routing policy, and an executor card for the test double in the scenario's mode. F27 routes to an executor that has no card. - run.sh kills the host at the scenario's crash marker (after-arm, after-result, after-verdict), applies the scenario's edit, and opens a new session for pickup. It relays the owner's answers verbatim and never answers or attests for them. - collect.sh and oracle.sh judge a run from files only: contract, acceptance by ground truth, final-tree digests, stop evidence, no second writer, no overlapping worker lifetimes. A pilot F1 run showed the host stopping to ask for attestation, and the oracle failed that unfinished state as it should. A smoke F27 run passed all checks. Neither counts as evidence; plan task 3.1 runs the gate with the owner. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
PDN-0004, S7 scoping. This is a design-only change for the owner to decide on before S7 is built. Design: §20.2 release gate, §26.2 S7. Issue #12.
Proposal
Narrow the Layer B release gate from 13 scenarios (F1, F2, F4, F8, F10–F15, F18, F27) to 6, one per invariant (F14 added back in review):
acceptedline after a crash;Layer A already runs all of them in CI against the reference host. Layer B only has to show that a real host follows the prose. The rest become post-release evidence.
Accept owner attestation as the v0.5 cost. The S6 measurements (feat: measured executor cards for kiro-cli, claude and codex #21) show that no shipped executor gives safe machine stop evidence on macOS: Claude Code and Codex tool shells survive
SIGKILL, and kiro-cli could detach through a trusted shell. So every delegated attempt ends with a person confirming no process remains. v0.5 documents this in the README and release notes instead of weakening §8.3. Lowering it is v0.6 work (a macOS containment scope, or measured normal-exit behavior).In Layer B the owner attests by hand. The harness never records
owner-attesteditself.Why now
v0.5 has grown to roughly 50k lines. About 40k of those are fixtures, recorded evidence and byte-identical copies of the helper; about 8.5k are hand-written. The full 13-scenario Layer B gate, with a person attesting every attempt, would make S7 the largest slice.
Tests
tests/docs.sh: 32 (design text only).🤖 Generated with Claude Code