From f34aa9b7d132cf580f38dbce2be17c3840618889 Mon Sep 17 00:00:00 2001 From: Hieu Phung Date: Sat, 5 Sep 2026 19:53:08 +0700 Subject: [PATCH 1/3] Delete the Review dispatch-record lookup so the reviewer reproduces the counterexample instead of searching a window that cannot hold it. Co-authored-by: Cursor --- .claude-plugin/plugin.json | 2 +- .codex-plugin/plugin.json | 2 +- skills/delivery/SKILL.md | 24 +++++------------------- tests/contracts.sh | 10 +++++----- 4 files changed, 12 insertions(+), 26 deletions(-) diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 81cfba1..3470d6c 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dely", - "version": "0.17.5", + "version": "0.17.6", "description": "An automation-first, Orca-supervised delivery protocol for coding agents.", "license": "MIT", "keywords": ["delivery", "workflow", "review", "orca", "automation"], diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index 910d98a..768bbfc 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dely", - "version": "0.17.5", + "version": "0.17.6", "description": "An automation-first, Orca-supervised delivery protocol for coding agents.", "license": "MIT", "keywords": ["delivery", "workflow", "review", "orca", "automation"], diff --git a/skills/delivery/SKILL.md b/skills/delivery/SKILL.md index bae0d4e..8c600cf 100644 --- a/skills/delivery/SKILL.md +++ b/skills/delivery/SKILL.md @@ -265,11 +265,7 @@ thing under check rather than as the check, and say so. Review independence is role independence: a fresh session that did not implement and does not edit the candidate. It gets the decision record (or -Bounded design), the baseline, and the diff. The reviewer reads the -implementer's dispatch record itself — not the implementer's reasoning. -`worker-read` returns the hook-reported transcript when Orca can prove the -worker session, and otherwise returns bounded terminal output, with a typed -`fallbackReason` when the plane supplies one. The phase adds no sandbox +Bounded design), the baseline, and the diff. The phase adds no sandbox by default; `AGENTS.md` may pin one for a concrete risk. Review depth is adaptive: Bounded work gets one independent whole-change @@ -286,20 +282,10 @@ gates in that tree, so a concurrent edit makes another task's work look like this one's result. **Reproduce, do not accept.** Run the gates yourself. A claim you did not -reproduce is not evidence. - -A disposition is not reachable until the reviewer has located, in the -implementer's dispatch record, the run where the counterexample was -observed red — an implementation that is present, runs, returns a pass, -and is wrong. An instrument red because the behaviour was absent is not -that run. Where Orca cannot recover the record, the reviewer says so with -the reason the plane gave, and treats the implementer's own account as -the thing under check rather than as the check. Where the implementer -had no authority to produce the counterexample, locating that run in the -implementer's dispatch record is not required; the observation is valid -only when a role that does have that authority recorded it as that -role's observation, not as the implementer's. The reviewer verifies as -much as its own seat reaches, and records the remainder as not verified. +reproduce is not evidence. The reviewer observes the counterexample +discriminate for itself — an implementation that is present, runs, returns +a pass, and is wrong. An instrument red only because the behaviour was +absent is not that observation. Classify findings: **Blocking** — contract failure, regression, data or security risk. **Important** — missing required behaviour, test, or diff --git a/tests/contracts.sh b/tests/contracts.sh index dca781f..944fb04 100755 --- a/tests/contracts.sh +++ b/tests/contracts.sh @@ -20,8 +20,8 @@ skill="$root/skills/delivery/SKILL.md" claude_version="$(jq -r .version "$claude_manifest" 2>/dev/null)" codex_version="$(jq -r .version "$codex_manifest" 2>/dev/null)" -if [ "$claude_version" != "0.17.5" ] || [ "$codex_version" != "0.17.5" ]; then - fail_with "manifest versions must both be 0.17.5 (claude=$claude_version codex=$codex_version)" +if [ "$claude_version" != "0.17.6" ] || [ "$codex_version" != "0.17.6" ]; then + fail_with "manifest versions must both be 0.17.6 (claude=$claude_version codex=$codex_version)" fi root_name="$(jq -r .name "$root_manifest" 2>/dev/null)" @@ -62,14 +62,14 @@ grep -Fq 'do not infer it from reading' "$skill" && ! grep -Fq 'proves readiness review="$(awk '/^## Review[[:space:]]*$/{p=1;next} p && /^## /{exit} p' "$skill")"; [ -n "$review" ] || fail_with "$skill is missing a ## Review section" printf '%s\n' "$review" | grep -Fq 'shared mutable state' && printf '%s\n' "$review" | grep -Fq 'did not verify' && printf '%s\n' "$review" | grep -Fq 'not implementing the candidate' && printf '%s\n' "$review" | grep -Fq 'per finding, not per review' || fail_with "$skill ## Review does not carry the sequencing, unverified, Control-owned-amendment, and one-pass-per-finding rules" -# Reviewer dispatch-record obligation is matched verbatim so a weakened 'should' or a dropped locator fails. -expected_review_section='Review independence is role independence: a fresh session that did not implement and does not edit the candidate. It gets the decision record (or Bounded design), the baseline, and the diff. The reviewer reads the implementer'"'"'s dispatch record itself — not the implementer'"'"'s reasoning. `worker-read` returns the hook-reported transcript when Orca can prove the worker session, and otherwise returns bounded terminal output, with a typed `fallbackReason` when the plane supplies one. The phase adds no sandbox by default; `AGENTS.md` may pin one for a concrete risk. Review depth is adaptive: Bounded work gets one independent whole-change review. Each Architectural task gets an independent task review. After all tasks are accepted, a different fresh reviewer performs one integration review of task interactions, complete-contract coverage, deferred findings, candidate identity, and release readiness. Only that final review is release-binding for Architectural work; Bounded work has no earlier task review and no duplicate integration review. No worker runs while a review of the same working tree runs. The working tree and its gate surface are shared mutable state, and a review reproduces gates in that tree, so a concurrent edit makes another task'"'"'s work look like this one'"'"'s result. **Reproduce, do not accept.** Run the gates yourself. A claim you did not reproduce is not evidence. A disposition is not reachable until the reviewer has located, in the implementer'"'"'s dispatch record, the run where the counterexample was observed red — an implementation that is present, runs, returns a pass, and is wrong. An instrument red because the behaviour was absent is not that run. Where Orca cannot recover the record, the reviewer says so with the reason the plane gave, and treats the implementer'"'"'s own account as the thing under check rather than as the check. Where the implementer had no authority to produce the counterexample, locating that run in the implementer'"'"'s dispatch record is not required; the observation is valid only when a role that does have that authority recorded it as that role'"'"'s observation, not as the implementer'"'"'s. The reviewer verifies as much as its own seat reaches, and records the remainder as not verified. Classify findings: **Blocking** — contract failure, regression, data or security risk. **Important** — missing required behaviour, test, or reconciliation. **Minor** — useful, does not block. **Out of scope** — recorded, not absorbed. Return exactly one role disposition: `ACCEPT`, `CHANGES_REQUESTED`, or `BLOCKED`. State what the review did not verify — what it did not reproduce or read. A contradiction you cannot resolve is `CHANGES_REQUESTED`. Do not open remediation over wording when deterministic checks already prove the contract.' +# Reviewer reproduce-the-counterexample obligation is matched verbatim so a weakened 'should' or a dropped discriminator fails. +expected_review_section='Review independence is role independence: a fresh session that did not implement and does not edit the candidate. It gets the decision record (or Bounded design), the baseline, and the diff. The phase adds no sandbox by default; `AGENTS.md` may pin one for a concrete risk. Review depth is adaptive: Bounded work gets one independent whole-change review. Each Architectural task gets an independent task review. After all tasks are accepted, a different fresh reviewer performs one integration review of task interactions, complete-contract coverage, deferred findings, candidate identity, and release readiness. Only that final review is release-binding for Architectural work; Bounded work has no earlier task review and no duplicate integration review. No worker runs while a review of the same working tree runs. The working tree and its gate surface are shared mutable state, and a review reproduces gates in that tree, so a concurrent edit makes another task'"'"'s work look like this one'"'"'s result. **Reproduce, do not accept.** Run the gates yourself. A claim you did not reproduce is not evidence. The reviewer observes the counterexample discriminate for itself — an implementation that is present, runs, returns a pass, and is wrong. An instrument red only because the behaviour was absent is not that observation. Classify findings: **Blocking** — contract failure, regression, data or security risk. **Important** — missing required behaviour, test, or reconciliation. **Minor** — useful, does not block. **Out of scope** — recorded, not absorbed. Return exactly one role disposition: `ACCEPT`, `CHANGES_REQUESTED`, or `BLOCKED`. State what the review did not verify — what it did not reproduce or read. A contradiction you cannot resolve is `CHANGES_REQUESTED`. Do not open remediation over wording when deterministic checks already prove the contract.' actual_review_section="$(awk '/^## Review$/{flag=1; next} flag && /^#/{exit} flag' "$skill" | tr '\n' ' ' | tr -s ' ')" actual_review_section="${actual_review_section# }" actual_review_section="${actual_review_section% }" -[ "$actual_review_section" = "$expected_review_section" ] || fail_with "review section in $skill no longer matches the approved reviewer-reads-dispatch-record and counterexample-reproduction contract verbatim" +[ "$actual_review_section" = "$expected_review_section" ] || fail_with "review section in $skill no longer matches the approved reproduce-do-not-accept and counterexample-observation contract verbatim" for f in '--permission-mode bypassPermissions' '--dangerously-skip-permissions' '--trust-all-tools' '--force' 'claude -p' 'codex exec' 'grok --prompt-file' 'agy -p' 'kiro-cli chat --no-interactive' 'cursor-agent -p' 'copilot -p' '--allow-all'; do grep -Fq -- "$f" "$harnesses" || fail_with "$harnesses does not name: $f" From 0aab5e34048ffbc9af678b250b8584a3847f340d Mon Sep 17 00:00:00 2001 From: Hieu Phung Date: Sat, 5 Sep 2026 20:19:47 +0700 Subject: [PATCH 2/3] Record the deletion and supersede the lookup decision. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01RFPwA3GEzovwEoeohLLPjM --- docs/decisions.md | 62 +++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 62 insertions(+) diff --git a/docs/decisions.md b/docs/decisions.md index 0fa500c..c16b48c 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -9,6 +9,64 @@ Last updated 2026-09-05. ## Settled +### 2026-09-05 — The reviewer reproduces the counterexample; the dispatch-record lookup is deleted + +#### Context + +The record below required a disposition to wait until the reviewer had located +the counterexample's red run in the implementer's dispatch record. Two further +deliveries each added a branch to rescue that lookup: one for a counterexample +the implementer had no authority to produce, one proposed for a record recovered +only in part. + +A Spike then measured the lookup instead of extending it. Across all ten settled +dispatches of those deliveries, `worker-read` returned one of two shapes. Cursor +dispatches gave `source: terminal`, 120 lines, `truncated: true`, `cursor: null` +— no way to page backwards. Claude Code dispatches gave `source: transcript` +with a cursor, warning that older messages were omitted from the bounded archive. + +Searching a Cursor implementer's retained window for its own evidence found the +fixture command zero times and the failing output zero times. The window holds +the worker's closing summary, not the middle of the session where the +counterexample ran. `AGENTS.md` pins Cursor for `implement`, so in this +repository the implementer's dispatch record structurally cannot hold the red +run. + +Across the five reviews written under the rule, none located the run. All five +reproduced the counterexample themselves and said so. + +#### Decision + +The lookup is deleted, with both of its escape branches. The surviving +obligation joins the rule it always belonged to: the reviewer observes the +counterexample discriminate for itself, and an instrument red only because the +behaviour was absent is not that observation. Review's opening paragraph no +longer describes `worker-read`; `## Evidence` and the `### Handoff` +`Verification` clause already carry that machinery, including what to do when +Orca cannot recover a dispatch item. + +This removes the dependency on plane retention entirely, so the partial-recovery +case needs no branch. `tests/contracts.sh` pins the shortened prose verbatim. + +#### Consequences + +The `## Review` section falls from 353 words to 206. It remains 26 words larger +than the 180 it was before the lookup was introduced, so this is a reduction +against the peak and not a return to the starting point; the residue is the +discriminator sentence, which is the part that was load-bearing. + +What the deletion does not do is make a reviewer look. Nothing in this +repository can observe that. The change moves the obligation to something a +reviewer can discharge from its own seat rather than one it provably could not. + +The cost of the original rule was three deliveries of repair after it, because +it was written asserting what the plane retained without measuring it first. +That is the reusable part. + +This decision ships in **0.17.6**. + +--- + ### 2026-09-05 — The reviewer reads the implementer's dispatch record, and the contract-script ceiling is 280 #### Context @@ -46,6 +104,10 @@ Orca cannot recover the record, the reviewer says so with the reason the plane gave, and treats the implementer's own account as the thing under check rather than as the check. +**Superseded on 2026-09-05.** A Spike measured the dispatch record and found +it structurally cannot hold the red run under this repository's harness pin; +the lookup and its branches are deleted. The ceiling raise stands. + `tests/contracts.sh` pins that Review prose verbatim. The ceiling rises from 250 to 280 so the pin can exist. The raise is deliberate room for a pin the contract needs; it is not permission to grow the script into a test suite. From bf7ced8435c7ec1ba4810d565a79be2874683013 Mon Sep 17 00:00:00 2001 From: Hieu Phung Date: Sat, 5 Sep 2026 20:36:27 +0700 Subject: [PATCH 3/3] Correct the record's history: one delivery added one branch, not two. 0.17.3 shipped the lookup with its recoverability escape attached; 0.17.5 added the authority branch; partial recovery was queued and never shipped. Co-Authored-By: Claude Opus 5 Claude-Session: https://claude.ai/code/session_01RFPwA3GEzovwEoeohLLPjM --- docs/decisions.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index c16b48c..a626062 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -14,10 +14,10 @@ Last updated 2026-09-05. #### Context The record below required a disposition to wait until the reviewer had located -the counterexample's red run in the implementer's dispatch record. Two further -deliveries each added a branch to rescue that lookup: one for a counterexample -the implementer had no authority to produce, one proposed for a record recovered -only in part. +the counterexample's red run in the implementer's dispatch record, with an +escape for a record Orca cannot recover. A later delivery added a second escape, +for a counterexample the implementer had no authority to produce. A third case — +a record recovered only in part — was found and queued, and is not shipped. A Spike then measured the lookup instead of extending it. Across all ten settled dispatches of those deliveries, `worker-read` returned one of two shapes. Cursor