diff --git a/.claude-plugin/plugin.json b/.claude-plugin/plugin.json index 76531cf..7d80505 100644 --- a/.claude-plugin/plugin.json +++ b/.claude-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dely", - "version": "0.23.0", + "version": "0.23.1", "description": "An automation-first, Orca-supervised delivery protocol for coding agents.", "license": "MIT", "keywords": ["delivery", "workflow", "review", "orca", "automation"], diff --git a/.codex-plugin/plugin.json b/.codex-plugin/plugin.json index 8d6dea5..c57cb69 100644 --- a/.codex-plugin/plugin.json +++ b/.codex-plugin/plugin.json @@ -1,6 +1,6 @@ { "name": "dely", - "version": "0.23.0", + "version": "0.23.1", "description": "An automation-first, Orca-supervised delivery protocol for coding agents.", "license": "MIT", "keywords": ["delivery", "workflow", "review", "orca", "automation"], diff --git a/AGENTS.md b/AGENTS.md index e98ad72..cc0c85f 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -57,8 +57,8 @@ delivery run. | Phase | Harness | Model | Effort | | --- | --- | --- | --- | -| `implement` | Cursor Agent CLI | cursor-grok-4.6-high | default | -| `review` | Claude Code | claude-opus-5-5 | high | +| `implement` | Claude Code | claude-sonnet-5-5 | medium | +| `review` | Codex CLI | gpt-6.1-sol | high | The table is this repository's deployment selection, not the portable @@ -92,9 +92,9 @@ git ls-files -z '*.sh' 'skills/delivery/scripts/dely' | xargs -0 -n1 bash -n ``` ```bash -test "$(jq -r .version .claude-plugin/plugin.json)" = 0.23.0 -test "$(jq -r .version .codex-plugin/plugin.json)" = 0.23.0 -test "$(jq -r .version package.json)" = 0.23.0 +test "$(jq -r .version .claude-plugin/plugin.json)" = 0.23.1 +test "$(jq -r .version .codex-plugin/plugin.json)" = 0.23.1 +test "$(jq -r .version package.json)" = 0.23.1 ``` ```bash diff --git a/README.md b/README.md index 0facb9a..7d7fbc8 100644 --- a/README.md +++ b/README.md @@ -64,7 +64,7 @@ question; Pi asks only when the project or a parent folder has `.pi/` or | Kiro CLI | ✗ | ✗ | ✗ | Workers never start in Orca. | | Cursor Agent CLI | ✓ | ✓ | ✓ | Choose the effort through the model name, for example cursor-grok-4.6-high. | | GitHub Copilot CLI | ✓ | ⚠ | ⚠ | At the first launch, answer No, thanks to the app prompt; until then a launch can leave the task unsent. Answer the folder trust question once in every project and choose to remember the folder; Orca does not see it, so a missed one shows as NO_ACK. As a worker, you cannot choose the model. | -| OMP | ✓ | ⚠ | ⚠ | As a worker, OMP 18.4 does not start under Orca 1.4.217 or later (stablyai/orca#24068) until that issue closes, and a crash is noticed after about 10 minutes. It never asks for trust, so use it only in repositories you trust. As a worker, to pin a model, install Dely in OMP. | +| OMP | ✓ | ⚠ | ⚠ | It never asks for trust, so use it only in repositories you trust. As a worker, use Orca 1.4.220 or newer, which does not show OMP's status (stablyai/orca#24436), so Dely reports a working OMP worker as needing attention. As a worker, to pin a model, install Dely in OMP, then check the model in OMP's session log. | | Pi | ⚠ | ⚠ | ⚠ | As Control, run Dely's commands with no time limit, and release leftover workers. As a worker, to pin a model, install Dely in Pi, then check the model in Pi's session log. | Other agents Orca can launch may work too; @@ -222,10 +222,10 @@ Update by installing the new tag. At startup, check that Skills names Invoke `/skill:delivery` or `/skill:setup`. Check a pinned worker's model in `~/.pi/agent/sessions/`. -Checked versions (observations, not minimums): Orca 1.4.218; Claude Code +Checked versions (observations, not minimums): Orca 1.4.218, 1.4.220 (OMP worker); Claude Code 2.1.286; Codex CLI 0.159.3; Cursor Agent CLI 2026.09.28-64d2043; Copilot CLI 1.0.89; Antigravity CLI 1.2.13 (install), 1.2.14 (worker, Control); Grok -Build 1.0.44; OMP 18.4.8 (Control); Pi 0.99.1. +Build 1.0.44; OMP 18.4.8 (Control), 18.6.1 (worker); Pi 0.99.1. ## Choose models diff --git a/docs/decisions.md b/docs/decisions.md index 0dce3be..3aacec1 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -3,12 +3,116 @@ What has been settled, what is still open, and what was rejected and why. Rationale is kept because the reasons are the reusable part. -Last updated 2026-10-01. +Last updated 2026-10-05. --- ## Settled +### 2026-10-05 — OMP's pin is unchecked while Orca reports no OMP status + +#### Context + +Measured on 2026-10-05 (evidence `~/dely-probe/omp-ready-1420/NOTES.md`): + +- Orca 1.4.220 with OMP 18.6.1: `worker-start --agent omp` passed agent + readiness in about 8 s, four of four. stablyai/orca#24068 is closed + (PR #24295). +- Every OMP worker's `worker-show` projection had `liveness` + `{verdict: unverifiable, reason: missing_status}` and no `provider`, pinned + or not (stablyai/orca#24436; fix PR stablyai/orca#24196 is open). +- Dely 0.23.0 `dely dispatch` with an OMP pin `google-vertex/gemini-3.8-flash` + `medium` printed `FAILED pin not applied: expected + google-vertex/gemini-3.8-flash, saw none`, although the OMP session log + shows the extension applied the pin (`thinking_level_change` from `high` to + `medium` before the first user message). +- `worker-start --agent omp --model …` is refused on 1.4.220 ("does not + support launch-time model selection"); stablyai/orca#24796 (merged + 2026-10-03) is not in that release. +- `omp --model google-vertex/gemini-3.8-flash:low` started with thinking + `low` (session log), so an effort can travel in the model name. + +#### Decision + +OMP's `modelPin` is `spec-unchecked`: the pin still travels as a spec line, +and dispatch skips the post-acknowledgement model check. The extension still +applies the pin and fails closed on a bad one. The README role row and the +checklist follow; the user checks the model in OMP's session log. + +This amends the 2026-09-27 record's post-acknowledgement check and its +rejected alternative in the 2026-09-28 record, whose premise "Orca does +report the model" no longer holds. It supersedes the two OMP items in the +2026-09-27 Deferred list. + +#### Alternatives considered + +**Keep `spec` and wait for stablyai/orca#24196.** Rejected: every pinned OMP +dispatch fails today. + +**Read OMP's session log from the helper.** Rejected: it puts a +harness-private file format in the helper. + +#### Consequences + +- No harness uses `spec` now; its check in `dely.js` stays dormant. +- An OMP worker that did not load the extension runs OMP's default unseen, as + for Pi. + +#### Verification, 2026-10-05 + +A separate Claude Code session ran `probe/checklist.md` on a snapshot of +`c7e96e4` (Orca 1.4.220, OMP 18.6.1), in the scope the owner chose: OMP rows +plus stubs. Report: `~/dely-probe/checklist-omp-c7e96e4.md`; raw evidence +`~/dely-probe/live-0231/`. + +- The OMP install from the snapshot resolved to the linked package and passed + the hash chain. +- The rejected-message stub passed (RED, LOG PASS, GREEN). The argv stub + against `v0.23.0` passed with `DELY_NAMED_INTENDED` empty: `spec` and + `spec-unchecked` launch the same argv. +- OMP parameterized worker row, pin `google-vertex/gemini-3.5-flash` + `medium`: preflight PASS in 9 s; implement and review both `DISPATCHED` + and settled with `worker_done`; the review returned `ACCEPT`. In both + session logs the pinned model and level precede the first user message, + and every assistant message ran the pinned model. +- Row 8, pin `google-vertex/gemini-3.5-flash` `low` (model and level both + differ from the default): `DISPATCHED` without a model check; 19 of 19 + assistant messages ran the pin. +- Row 9, an unknown selector: `NO_ACK` quoting `DELY-PIN-FAIL`; OMP wrote no + session for it. +- Row 10, Model `default`: no `dely-pin:` line in the first user message; + the worker ran OMP's default. + +Finding: `dely wait` returned `ATTENTION` 15 s after it started for a healthy +OMP worker (liveness `unverifiable` / `missing_status`, `nextAction` `none`, +`requiresAction` `true`), which settled about 20 s later. That is the +projection a killed worker shows (row 4 table), and the behaviour the +2026-09-28 record saw under Orca 1.4.215. The `limits` sentence "a crash is +noticed after about 10 minutes" was therefore replaced in `4d7d581` by "Dely +reports a working OMP worker as needing attention". `limits` and `notes` do +not change a launch, so no row reran. + +Not run, because the owner scoped this delivery to OMP rows plus stubs: the +Claude Code and Codex CLI worker rows (upgraded to 2.1.289 and 0.160.0), the +Control rows per wake mode and for OMP and Pi, rows 4 to 7, and Pi rows 13 to +16 (Pi upgraded to 1.0.2). Row 11 did not run because neither `start()` nor +the extension changed. OMP crash timing on Orca 1.4.220 was not measured. + +#### Deferred + +Pinning OMP by flag. Trigger: an Orca release containing +stablyai/orca#24796. `modelFlag: true`, no `modelPin`, effort as +`:` in the Model cell; then delete the `spec` mode and its +check from `dely.js`, `SKILL.md`, `skills/setup/SKILL.md` and the `modelPin` +gate. Pi keeps the extension. + +Returning OMP to `spec`: only if a later decision keeps the spec pin and Orca +reports OMP's model again (stablyai/orca#24196). + +Telling a live OMP worker from a dead one in `dely wait`. Trigger: an Orca +release containing stablyai/orca#24196, or a decision to read a signal other +than the projection; until then the projection cannot tell them apart. + ### 2026-10-01 — README shows a role table derived from `harnesses.json`; `limits` are user caveats; the checklist runs rows by what changed #### Context diff --git a/harnesses.json b/harnesses.json index a961560..89acab5 100644 --- a/harnesses.json +++ b/harnesses.json @@ -194,7 +194,7 @@ "binary": "omp", "status": "supported", "controlWake": "waker", - "limits": "As a worker, OMP 18.4 does not start under Orca 1.4.217 or later (stablyai/orca#24068) until that issue closes, and a crash is noticed after about 10 minutes. It never asks for trust, so use it only in repositories you trust. As a worker, to pin a model, install Dely in OMP.", + "limits": "It never asks for trust, so use it only in repositories you trust. As a worker, use Orca 1.4.220 or newer, which does not show OMP's status (stablyai/orca#24436), so Dely reports a working OMP worker as needing attention. As a worker, to pin a model, install Dely in OMP, then check the model in OMP's session log.", "trust": "none", "discovery": { "models": "omp models --json", @@ -204,7 +204,7 @@ }, "modelFlag": false, "effortFlag": false, - "modelPin": "spec", + "modelPin": "spec-unchecked", "permissionDefault": "none", "forbiddenHeadless": [ "omp -p", @@ -217,7 +217,7 @@ "readsAgentsMd": true, "needsImport": false }, - "notes": "Trust is none: no workspace-trust gate. The pin travels as a spec line. OMP 18.4 titles an idle pane 'π > ', which Orca does not read as idle (stablyai/orca#24068). A second-turn dely-pin line in an interactive OMP 18.4.4 session was ignored (row 11). Decision sections: 2026-09-27, 2026-09-30." + "notes": "Trust is none: no workspace-trust gate. The pin travels as a spec line. Orca 1.4.220 reports no OMP status or model (stablyai/orca#24436), so dispatch skips the model check; the extension still applies the pin and fails closed on a bad one. Without status, dely wait reports a live OMP worker as ATTENTION (unverifiable/missing_status, requiresAction true), the projection a killed worker also shows. A second-turn dely-pin line in an interactive OMP 18.4.4 session was ignored (row 11). Decision sections: 2026-09-27, 2026-09-30, 2026-10-05." }, { "name": "Pi", diff --git a/package.json b/package.json index a6d5b01..f10f045 100644 --- a/package.json +++ b/package.json @@ -1,6 +1,6 @@ { "name": "dely", - "version": "0.23.0", + "version": "0.23.1", "private": true, "pi": { "extensions": ["./extensions/dely-pin.ts"], diff --git a/probe/checklist.md b/probe/checklist.md index 22e1cf1..4be0e43 100644 --- a/probe/checklist.md +++ b/probe/checklist.md @@ -47,10 +47,8 @@ deployment uses it as Control. The release floor is the argv stub row, the rejected-message stub row, the live worker rows the rules above require, the Control rows those rules -require, and rows 4–7. OMP rows 8, 9 and 10 and the OMP parameterized -worker row are a standing named skip until stablyai/orca#24068 is closed. -Rows 10–11 run when `start()` or the pin extension changes, except row 10 -stays under that OMP skip while the issue is open. Rows 13–16 run when +require, and rows 4–7. Rows 10–11 run when `start()` or the pin extension +changes. Rows 13–16 run when `start()`, the extension or the Pi entry changes, after a Pi upgrade, and after an Orca upgrade. After an Orca upgrade, run the live worker row for each harness a deployment uses, the Control row once per wake mode, rows 4 @@ -430,8 +428,7 @@ recording the result. Run for each supported entry that changed and for each harness upgrade a deployment uses. Unchanged entries are covered by the argv stub row. -Skip a deferred entry. The OMP parameterized worker row is a standing named -skip until stablyai/orca#24068 is closed. +Skip a deferred entry. Vary exactly: `id`, `name`, `discovery`, Model and Effort from the candidate's entry; the probe path `worker-`; and phase @@ -778,41 +775,23 @@ second. ## Step 8 — row 8, OMP worker with a valid pin -Standing named skip until stablyai/orca#24068 is closed. - Build a probe repository under `~/dely-probe/` whose `implement` pin is OMP with a Model `selector` from `omp models --json` and an Effort that is one of -that model's `thinking` levels, and with the pinned model different from +that model's `thinking` levels. The pinned model and level must differ from OMP's configured default. `probe/mkrepo.sh` writes whatever harness names it is given; OMP needs no trust step. From that repository run `dely dispatch` for `implement`. -**Pass:** `dely dispatch` prints `DISPATCHED ` — the post-acknowledgement -check passed. Record `worker-show` `result.projection.provider.model` at -the acknowledgement; if that value is not already the pinned selector, -record how long after the acknowledgement it became the pin. The last -`model_change` (and `thinking_level_change`, when an Effort is pinned) -before the first user message is the pinned model and level; every -assistant message, the first included, is the pinned model; and -`worker-show` `projection.provider.model` before release shows it. -If the helper prints `DISPATCHED` but the first request ran OMP's -configured default, the check compared a different field or format. - -Then keep the same pin and stop this dispatch from loading `extensions/dely-pin.ts` -without touching any user configuration beyond this plugin: -`omp plugin disable dely`. Dispatch `implement` again. Re-enable with -`omp plugin enable dely` before later rows. - -**Pass:** `dely dispatch` prints -`FAILED pin not applied: expected , saw ` -and does not print `DISPATCHED`. A print of `DISPATCHED`, or a warning -that still dispatches, is a fail: the helper did not stop a worker that -was not running the pin. +**Pass:** `dely dispatch` prints `DISPATCHED ` without a model check, +because Orca does not report OMP's model (stablyai/orca#24436). In the OMP +session JSONL under `~/.omp/agent/sessions/`: the last `model_change` before +the first user message and every assistant message is the pinned model, and +the last `thinking_level_change` before the first user message is the pinned +level. If the first request ran OMP's configured default, the extension did +not apply the pin. ## Step 9 — row 9, OMP worker with an invalid pin -Standing named skip until stablyai/orca#24068 is closed. - Same probe-repository setup as row 8's first dispatch (plugin enabled), with a selector `omp models --json` does not offer. @@ -823,8 +802,7 @@ called `ctx.shutdown()` instead of exiting the process. ## Step 10 — row 10, OMP worker with Model `default` -Standing named skip until stablyai/orca#24068 is closed. After that, run this -row when `start()` in `dely.js` or `extensions/dely-pin.ts` changes. +Run this row when `start()` in `dely.js` or `extensions/dely-pin.ts` changes. Same probe-repository setup as row 8's first dispatch, with Model `default` and Effort `default`. @@ -925,8 +903,8 @@ replayed batch in a live session (the stub covers only its fixtures). A quota ex the model was having a bad day. A shape change between two Orca releases, until the rows are run again. A Model `default` dispatch whose spec itself carries a pin line. A stuck live Pi worker before `DEADLINE`. Pi versions other than -0.87.1 and 0.99.1. Pi's trust layout on other machines. A pinned Pi worker that did not -load the extension, which runs its own default unseen. A preflight that leaks +0.87.1 and 0.99.1. Pi's trust layout on other machines. A pinned Pi or OMP worker that did +not load the extension, which runs its own default unseen. A preflight that leaks its worker. A heartbeat-only acknowledgement, because the argv stub's fixture returns one batch. An Orca upgrade, which reruns live workers only for deployment harnesses, so the other supported harnesses are not re-verified