Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "dely",
"version": "0.23.0",
"version": "0.23.1",
"description": "An automation-first, Orca-supervised delivery protocol for coding agents.",
"license": "MIT",
"keywords": ["delivery", "workflow", "review", "orca", "automation"],
Expand Down
2 changes: 1 addition & 1 deletion .codex-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "dely",
"version": "0.23.0",
"version": "0.23.1",
"description": "An automation-first, Orca-supervised delivery protocol for coding agents.",
"license": "MIT",
"keywords": ["delivery", "workflow", "review", "orca", "automation"],
Expand Down
10 changes: 5 additions & 5 deletions AGENTS.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,8 +57,8 @@ delivery run.

| Phase | Harness | Model | Effort |
| --- | --- | --- | --- |
| `implement` | Cursor Agent CLI | cursor-grok-4.6-high | default |
| `review` | Claude Code | claude-opus-5-5 | high |
| `implement` | Claude Code | claude-sonnet-5-5 | medium |
| `review` | Codex CLI | gpt-6.1-sol | high |
<!-- dely:end -->

The table is this repository's deployment selection, not the portable
Expand Down Expand Up @@ -92,9 +92,9 @@ git ls-files -z '*.sh' 'skills/delivery/scripts/dely' | xargs -0 -n1 bash -n
```

```bash
test "$(jq -r .version .claude-plugin/plugin.json)" = 0.23.0
test "$(jq -r .version .codex-plugin/plugin.json)" = 0.23.0
test "$(jq -r .version package.json)" = 0.23.0
test "$(jq -r .version .claude-plugin/plugin.json)" = 0.23.1
test "$(jq -r .version .codex-plugin/plugin.json)" = 0.23.1
test "$(jq -r .version package.json)" = 0.23.1
```

```bash
Expand Down
6 changes: 3 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -64,7 +64,7 @@ question; Pi asks only when the project or a parent folder has `.pi/` or
| Kiro CLI | ✗ | ✗ | ✗ | Workers never start in Orca. |
| Cursor Agent CLI | ✓ | ✓ | ✓ | Choose the effort through the model name, for example cursor-grok-4.6-high. |
| GitHub Copilot CLI | ✓ | ⚠ | ⚠ | At the first launch, answer No, thanks to the app prompt; until then a launch can leave the task unsent. Answer the folder trust question once in every project and choose to remember the folder; Orca does not see it, so a missed one shows as NO_ACK. As a worker, you cannot choose the model. |
| OMP | ✓ | ⚠ | ⚠ | As a worker, OMP 18.4 does not start under Orca 1.4.217 or later (stablyai/orca#24068) until that issue closes, and a crash is noticed after about 10 minutes. It never asks for trust, so use it only in repositories you trust. As a worker, to pin a model, install Dely in OMP. |
| OMP | ✓ | ⚠ | ⚠ | It never asks for trust, so use it only in repositories you trust. As a worker, use Orca 1.4.220 or newer, which does not show OMP's status (stablyai/orca#24436), so Dely reports a working OMP worker as needing attention. As a worker, to pin a model, install Dely in OMP, then check the model in OMP's session log. |
| Pi | ⚠ | ⚠ | ⚠ | As Control, run Dely's commands with no time limit, and release leftover workers. As a worker, to pin a model, install Dely in Pi, then check the model in Pi's session log. |

Other agents Orca can launch may work too;
Expand Down Expand Up @@ -222,10 +222,10 @@ Update by installing the new tag. At startup, check that Skills names
Invoke `/skill:delivery` or `/skill:setup`. Check a pinned worker's model in
`~/.pi/agent/sessions/`.

Checked versions (observations, not minimums): Orca 1.4.218; Claude Code
Checked versions (observations, not minimums): Orca 1.4.218, 1.4.220 (OMP worker); Claude Code
2.1.286; Codex CLI 0.159.3; Cursor Agent CLI 2026.09.28-64d2043; Copilot CLI
1.0.89; Antigravity CLI 1.2.13 (install), 1.2.14 (worker, Control); Grok
Build 1.0.44; OMP 18.4.8 (Control); Pi 0.99.1.
Build 1.0.44; OMP 18.4.8 (Control), 18.6.1 (worker); Pi 0.99.1.

## Choose models

Expand Down
106 changes: 105 additions & 1 deletion docs/decisions.md
Original file line number Diff line number Diff line change
Expand Up @@ -3,12 +3,116 @@
What has been settled, what is still open, and what was rejected and why.
Rationale is kept because the reasons are the reusable part.

Last updated 2026-10-01.
Last updated 2026-10-05.

---

## Settled

### 2026-10-05 — OMP's pin is unchecked while Orca reports no OMP status

#### Context

Measured on 2026-10-05 (evidence `~/dely-probe/omp-ready-1420/NOTES.md`):

- Orca 1.4.220 with OMP 18.6.1: `worker-start --agent omp` passed agent
readiness in about 8 s, four of four. stablyai/orca#24068 is closed
(PR #24295).
- Every OMP worker's `worker-show` projection had `liveness`
`{verdict: unverifiable, reason: missing_status}` and no `provider`, pinned
or not (stablyai/orca#24436; fix PR stablyai/orca#24196 is open).
- Dely 0.23.0 `dely dispatch` with an OMP pin `google-vertex/gemini-3.8-flash`
`medium` printed `FAILED <id> pin not applied: expected
google-vertex/gemini-3.8-flash, saw none`, although the OMP session log
shows the extension applied the pin (`thinking_level_change` from `high` to
`medium` before the first user message).
- `worker-start --agent omp --model …` is refused on 1.4.220 ("does not
support launch-time model selection"); stablyai/orca#24796 (merged
2026-10-03) is not in that release.
- `omp --model google-vertex/gemini-3.8-flash:low` started with thinking
`low` (session log), so an effort can travel in the model name.

#### Decision

OMP's `modelPin` is `spec-unchecked`: the pin still travels as a spec line,
and dispatch skips the post-acknowledgement model check. The extension still
applies the pin and fails closed on a bad one. The README role row and the
checklist follow; the user checks the model in OMP's session log.

This amends the 2026-09-27 record's post-acknowledgement check and its
rejected alternative in the 2026-09-28 record, whose premise "Orca does
report the model" no longer holds. It supersedes the two OMP items in the
2026-09-27 Deferred list.

#### Alternatives considered

**Keep `spec` and wait for stablyai/orca#24196.** Rejected: every pinned OMP
dispatch fails today.

**Read OMP's session log from the helper.** Rejected: it puts a
harness-private file format in the helper.

#### Consequences

- No harness uses `spec` now; its check in `dely.js` stays dormant.
- An OMP worker that did not load the extension runs OMP's default unseen, as
for Pi.

#### Verification, 2026-10-05

A separate Claude Code session ran `probe/checklist.md` on a snapshot of
`c7e96e4` (Orca 1.4.220, OMP 18.6.1), in the scope the owner chose: OMP rows
plus stubs. Report: `~/dely-probe/checklist-omp-c7e96e4.md`; raw evidence
`~/dely-probe/live-0231/`.

- The OMP install from the snapshot resolved to the linked package and passed
the hash chain.
- The rejected-message stub passed (RED, LOG PASS, GREEN). The argv stub
against `v0.23.0` passed with `DELY_NAMED_INTENDED` empty: `spec` and
`spec-unchecked` launch the same argv.
- OMP parameterized worker row, pin `google-vertex/gemini-3.5-flash`
`medium`: preflight PASS in 9 s; implement and review both `DISPATCHED`
and settled with `worker_done`; the review returned `ACCEPT`. In both
session logs the pinned model and level precede the first user message,
and every assistant message ran the pinned model.
- Row 8, pin `google-vertex/gemini-3.5-flash` `low` (model and level both
differ from the default): `DISPATCHED` without a model check; 19 of 19
assistant messages ran the pin.
- Row 9, an unknown selector: `NO_ACK` quoting `DELY-PIN-FAIL`; OMP wrote no
session for it.
- Row 10, Model `default`: no `dely-pin:` line in the first user message;
the worker ran OMP's default.

Finding: `dely wait` returned `ATTENTION` 15 s after it started for a healthy
OMP worker (liveness `unverifiable` / `missing_status`, `nextAction` `none`,
`requiresAction` `true`), which settled about 20 s later. That is the
projection a killed worker shows (row 4 table), and the behaviour the
2026-09-28 record saw under Orca 1.4.215. The `limits` sentence "a crash is
noticed after about 10 minutes" was therefore replaced in `4d7d581` by "Dely
reports a working OMP worker as needing attention". `limits` and `notes` do
not change a launch, so no row reran.

Not run, because the owner scoped this delivery to OMP rows plus stubs: the
Claude Code and Codex CLI worker rows (upgraded to 2.1.289 and 0.160.0), the
Control rows per wake mode and for OMP and Pi, rows 4 to 7, and Pi rows 13 to
16 (Pi upgraded to 1.0.2). Row 11 did not run because neither `start()` nor
the extension changed. OMP crash timing on Orca 1.4.220 was not measured.

#### Deferred

Pinning OMP by flag. Trigger: an Orca release containing
stablyai/orca#24796. `modelFlag: true`, no `modelPin`, effort as
`<selector>:<level>` in the Model cell; then delete the `spec` mode and its
check from `dely.js`, `SKILL.md`, `skills/setup/SKILL.md` and the `modelPin`
gate. Pi keeps the extension.

Returning OMP to `spec`: only if a later decision keeps the spec pin and Orca
reports OMP's model again (stablyai/orca#24196).

Telling a live OMP worker from a dead one in `dely wait`. Trigger: an Orca
release containing stablyai/orca#24196, or a decision to read a signal other
than the projection; until then the projection cannot tell them apart.

### 2026-10-01 — README shows a role table derived from `harnesses.json`; `limits` are user caveats; the checklist runs rows by what changed

#### Context
Expand Down
6 changes: 3 additions & 3 deletions harnesses.json
Original file line number Diff line number Diff line change
Expand Up @@ -194,7 +194,7 @@
"binary": "omp",
"status": "supported",
"controlWake": "waker",
"limits": "As a worker, OMP 18.4 does not start under Orca 1.4.217 or later (stablyai/orca#24068) until that issue closes, and a crash is noticed after about 10 minutes. It never asks for trust, so use it only in repositories you trust. As a worker, to pin a model, install Dely in OMP.",
"limits": "It never asks for trust, so use it only in repositories you trust. As a worker, use Orca 1.4.220 or newer, which does not show OMP's status (stablyai/orca#24436), so Dely reports a working OMP worker as needing attention. As a worker, to pin a model, install Dely in OMP, then check the model in OMP's session log.",
"trust": "none",
"discovery": {
"models": "omp models --json",
Expand All @@ -204,7 +204,7 @@
},
"modelFlag": false,
"effortFlag": false,
"modelPin": "spec",
"modelPin": "spec-unchecked",
"permissionDefault": "none",
"forbiddenHeadless": [
"omp -p",
Expand All @@ -217,7 +217,7 @@
"readsAgentsMd": true,
"needsImport": false
},
"notes": "Trust is none: no workspace-trust gate. The pin travels as a spec line. OMP 18.4 titles an idle pane 'π > <dir>', which Orca does not read as idle (stablyai/orca#24068). A second-turn dely-pin line in an interactive OMP 18.4.4 session was ignored (row 11). Decision sections: 2026-09-27, 2026-09-30."
"notes": "Trust is none: no workspace-trust gate. The pin travels as a spec line. Orca 1.4.220 reports no OMP status or model (stablyai/orca#24436), so dispatch skips the model check; the extension still applies the pin and fails closed on a bad one. Without status, dely wait reports a live OMP worker as ATTENTION (unverifiable/missing_status, requiresAction true), the projection a killed worker also shows. A second-turn dely-pin line in an interactive OMP 18.4.4 session was ignored (row 11). Decision sections: 2026-09-27, 2026-09-30, 2026-10-05."
},
{
"name": "Pi",
Expand Down
2 changes: 1 addition & 1 deletion package.json
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
{
"name": "dely",
"version": "0.23.0",
"version": "0.23.1",
"private": true,
"pi": {
"extensions": ["./extensions/dely-pin.ts"],
Expand Down
50 changes: 14 additions & 36 deletions probe/checklist.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,10 +47,8 @@ deployment uses it as Control.

The release floor is the argv stub row, the rejected-message stub row, the
live worker rows the rules above require, the Control rows those rules
require, and rows 4–7. OMP rows 8, 9 and 10 and the OMP parameterized
worker row are a standing named skip until stablyai/orca#24068 is closed.
Rows 10–11 run when `start()` or the pin extension changes, except row 10
stays under that OMP skip while the issue is open. Rows 13–16 run when
require, and rows 4–7. Rows 10–11 run when `start()` or the pin extension
changes. Rows 13–16 run when
`start()`, the extension or the Pi entry changes, after a Pi upgrade, and
after an Orca upgrade. After an Orca upgrade, run the live worker row for
each harness a deployment uses, the Control row once per wake mode, rows 4
Expand Down Expand Up @@ -430,8 +428,7 @@ recording the result.

Run for each supported entry that changed and for each harness upgrade a
deployment uses. Unchanged entries are covered by the argv stub row.
Skip a deferred entry. The OMP parameterized worker row is a standing named
skip until stablyai/orca#24068 is closed.
Skip a deferred entry.

Vary exactly: `id`, `name`, `discovery`, Model
and Effort from the candidate's entry; the probe path `worker-<id>`; and phase
Expand Down Expand Up @@ -778,41 +775,23 @@ second.

## Step 8 — row 8, OMP worker with a valid pin

Standing named skip until stablyai/orca#24068 is closed.

Build a probe repository under `~/dely-probe/` whose `implement` pin is OMP
with a Model `selector` from `omp models --json` and an Effort that is one of
that model's `thinking` levels, and with the pinned model different from
that model's `thinking` levels. The pinned model and level must differ from
OMP's configured default.
`probe/mkrepo.sh` writes whatever harness names it is given; OMP needs no
trust step. From that repository run `dely dispatch` for `implement`.

**Pass:** `dely dispatch` prints `DISPATCHED <id>` — the post-acknowledgement
check passed. Record `worker-show` `result.projection.provider.model` at
the acknowledgement; if that value is not already the pinned selector,
record how long after the acknowledgement it became the pin. The last
`model_change` (and `thinking_level_change`, when an Effort is pinned)
before the first user message is the pinned model and level; every
assistant message, the first included, is the pinned model; and
`worker-show` `projection.provider.model` before release shows it.
If the helper prints `DISPATCHED` but the first request ran OMP's
configured default, the check compared a different field or format.

Then keep the same pin and stop this dispatch from loading `extensions/dely-pin.ts`
without touching any user configuration beyond this plugin:
`omp plugin disable dely`. Dispatch `implement` again. Re-enable with
`omp plugin enable dely` before later rows.

**Pass:** `dely dispatch` prints
`FAILED <id> pin not applied: expected <selector>, saw <value or none>`
and does not print `DISPATCHED`. A print of `DISPATCHED`, or a warning
that still dispatches, is a fail: the helper did not stop a worker that
was not running the pin.
**Pass:** `dely dispatch` prints `DISPATCHED <id>` without a model check,
because Orca does not report OMP's model (stablyai/orca#24436). In the OMP
session JSONL under `~/.omp/agent/sessions/`: the last `model_change` before
the first user message and every assistant message is the pinned model, and
the last `thinking_level_change` before the first user message is the pinned
level. If the first request ran OMP's configured default, the extension did
not apply the pin.

## Step 9 — row 9, OMP worker with an invalid pin

Standing named skip until stablyai/orca#24068 is closed.

Same probe-repository setup as row 8's first dispatch (plugin enabled), with
a selector `omp models --json` does not offer.

Expand All @@ -823,8 +802,7 @@ called `ctx.shutdown()` instead of exiting the process.

## Step 10 — row 10, OMP worker with Model `default`

Standing named skip until stablyai/orca#24068 is closed. After that, run this
row when `start()` in `dely.js` or `extensions/dely-pin.ts` changes.
Run this row when `start()` in `dely.js` or `extensions/dely-pin.ts` changes.
Same probe-repository setup as row 8's first dispatch, with Model `default`
and Effort `default`.

Expand Down Expand Up @@ -925,8 +903,8 @@ replayed batch in a live session (the stub covers only its fixtures). A quota ex
the model was having a bad day. A shape change between two Orca releases, until
the rows are run again. A Model `default` dispatch whose spec itself carries
a pin line. A stuck live Pi worker before `DEADLINE`. Pi versions other than
0.87.1 and 0.99.1. Pi's trust layout on other machines. A pinned Pi worker that did not
load the extension, which runs its own default unseen. A preflight that leaks
0.87.1 and 0.99.1. Pi's trust layout on other machines. A pinned Pi or OMP worker that did
not load the extension, which runs its own default unseen. A preflight that leaks
its worker. A heartbeat-only acknowledgement, because the argv stub's fixture
returns one batch. An Orca upgrade, which reruns live workers only for
deployment harnesses, so the other supported harnesses are not re-verified
Expand Down