Skip to content

Retire the structural suite and narrow Dely to the harnesses it can support (0.19.0) - #55

Merged
hieuphung97 merged 78 commits into
mainfrom
feat/verify-event-driven-dispatch
Sep 16, 2026
Merged

hieuphung97 merged 78 commits into
mainfrom
feat/verify-event-driven-dispatch

Conversation

@hieuphung97

@hieuphung97 hieuphung97 commented Sep 12, 2026 •

Copy link
Copy Markdown
Owner

Summary

0.19.0 keeps the protocol and the helper that live probes actually exercised, and deletes the rest.

  • No structural suite, no CI. tests/ (contracts.sh, scripts.test.js, the fake Orca) and
    .github/workflows/contracts.yml are gone. The closure gates in AGENTS.md are now the whole structural
    check: whitespace, JSON validity, syntax enumerated from git ls-files, absence commands, an identifier
    grep over skills/, the disclosure greps, and a version gate.
  • The version pin moved into AGENTS.md. It names the literal 0.19.0 and both manifests must match it.
    That is what keeps the Claude and Codex installs from splitting now that no workflow compares them.
  • The helper is 478 lines, from 688. Gone: the Antigravity adopt path and its six temp-file helpers, the
    nine-string GATES classifier and the two preflight polls on it, pinWhy, dely.cmd, and the launcher's
    Electron branch.
  • Install and discovery name Claude Code, Codex CLI and Cursor Agent CLI. Grok Build, Antigravity CLI,
    Kiro CLI and GitHub Copilot CLI keep their measured launch mechanics in references/harnesses.md and lose
    their install sections and discovery commands. This narrows what is documented; it does not declare a closed
    set.
  • probe/ replaces the suite. checklist.md (seven live rows, the snapshot install with its hash check,
    the fixed probe paths, cleanup), mkrepo.sh, and trust.sh, which acts as the human on a Claude trust
    dialog and reports trust from ~/.claude.json rather than from the screen. Both scripts resolve their
    target and refuse anything outside ~/dely-probe/. No skill references probe/.
  • Record. The durable record is the 2026-09-16 entry in docs/decisions.md.

Why now: measured in this repository on 2026-09-16, the contracts workflow ran 89 times — 86 green, one
cancelled, two red. One red was the logo-assets commit, where the disclosure grep matched coordinates inside
an SVG path. The other was an external contributor's branch, whose logs have expired. No defect in this
repository's own shipped code was first found by CI, while tests/contracts.sh took 56 commits on main
against 39 for everything under skills/.

Live verification

Candidate installed for real from git archive de50905 into ~/dely-probe/.snap-de50905, added as a
marketplace and installed in Claude Code and Codex CLI. sha256 of SKILL.md matched the snapshot at every
Claude and Codex location. Orca 1.4.203, Claude Code 2.1.273, Codex CLI 0.154.0.

Row Result
1 — full delivery (Claude Code Control, Codex CLI implement, Claude Code review) Pass. 474 s wall, cf6cfba on the remote, review ACCEPT, zero human steps after the opening prompt, all four dispatches released
4 — worker killed after its acknowledgement Fail, cause outside the candidate (Orca 1.4.203 projection; see below). Recorded as a failed row in docs/decisions.md, not as a pass
5 — preflight on an untrusted Claude pin Pass. PREFLIGHT implement claude FAIL worker never started a turn: turn_start_unobserved in 56 s, quote carried the whole dialog ("Quick safety check: Is this a project you created or one you trust? … Security guide"), no terminal left behind, path confirmed untrusted in ~/.claude.json first
7 — full trust-intervention loop (Codex CLI Control, Claude Code implement pin) Pass. Details below

Rows 2, 3 and 6 were not run for this release.

Row 7, step by step

  1. Control reached preflight and stopped 58 s later (179 s from the opening prompt) on a message naming Claude
    Code, the failing pin and the action required. PREFLIGHT implement claude FAIL … ; PREFLIGHT review codex PASS 16s. Both preflight dispatches released, no leftover terminal.
  2. probe/trust.sh ~/dely-probe/t-de50905 answered the dialog and verified it: TRUSTED … verified in ~/.claude.json, 9.9 s.
  3. One line sent to Control — "Đã trust Claude Code trong repo này. Chạy lại preflight và tiếp tục." — accepted
    on the first try, no agent_prompt_blocked.
  4. The same Run got a second preflight, both pins passed, implement and review ran, review returned ACCEPT,
    and d436a86 reached the remote. Nothing else was sent. The Codex Control used wait-bg and notify for
    both waits, as its waker wake mode requires.

Row 5 was red first, and was fixed

The first run of row 5 on 4e0448d refused the untrusted pin correctly and quoted the dialog, but took 192 s
instead of the 39 s the deleted gate classifier used to manage: the classifier had been the only early-failure
path, and deleting it left only the 150 s timeout.

Orca already reports this without any string matching. Measured on an untrusted pin: the worker-list
projection holds stage.worker: "starting" for about 40 s, then flips to "start_unknown" with
detail: "turn_start_unobserved", while dispatchStatus stays "pending" throughout — which is why the
failed-row poll never fired. a51a247 drops on that projection as well as on a failed row, and removes the
trailing sleep in the poll loop, which existed only to space the two gate observations that no longer happen.
Row 5 then passed at 56 s.

Row 4 failed, and not here

A dispatched worker was killed after it acknowledged. Its process was gone and its terminal was back at a shell
prompt, but for 6 minutes and 8 seconds Orca 1.4.203 kept reporting terminalState: "active",
stage.worker: "ready", liveness: unverifiable/missing_status and nextAction: "none", so dely wait had
nothing to report and correctly kept waiting. The same scenario measured 16 s to ATTENTION on Orca 1.4.200.

dely wait's ATTENTION path is byte-identical to 0.18.0 — diff of the whole wait function between
3ee4e1a and this head shows one hunk, the removal of the adopted-terminal close after SETTLED, which runs
nowhere near it. worker-show offers no better signal: its lastHeartbeatAt was set 10 s after dispatch and
never updated, so it cannot distinguish a dead worker from a quiet one without a threshold that would
false-positive on the two-minute gaps seen in row 1.

This is an execution-plane finding, not a candidate defect, and adding a new liveness detector is outside this
delivery. probe/checklist.md now says whose signal that row reads and asks for the Orca version next to the
result. It also retires the "ATTENTION 16 s after the kill" line this pull request used to carry.

Also measured, not fixed here

  • Codex CLI 0.154.0 now shows its own trust dialog on a fresh directory ("Do you trust the contents of this
    directory? … 1. Yes, continue"). references/harnesses.md still records Codex's Setup as Orca preflight.
    That file is deliberately untouched here; it is replaced by harnesses.json in the next delivery, which is
    where the row should be corrected.
  • Claude Code resolved scripts/dely from the marketplace source directory, not from
    ~/.claude/plugins/cache. The hash check has to cover both; probe/checklist.md now says so.
  • Cursor Agent CLI could not be put on the candidate. cursor-agent plugin marketplace add takes a git URL
    only, and installing needs the interactive /plugin panel. Two stale copies remain on this machine, with
    hashes that differ from the candidate and from each other: ~/.cursor/plugins/cache/dely/dely/5690982… and
    ~/.cursor/plugins/marketplaces/github.com/hieuphung97/dely/8062f35…. No row here needed Cursor, so the
    rows above are unaffected, but a Cursor row cannot be run until those are replaced.
  • codex plugin marketplace add silently kept a stale marketplace and installed 0.18.0 while reporting
    success. Removing and re-adding fixed it. The hash check caught it; the report did not.
  • Uninstalling left ~/.claude/plugins/cache/dely/dely/0.19.0 behind after its marketplace was removed.
  • The row 1 Control noted two protocol weaknesses worth carrying into the next delivery: a worker_done body
    capped at a short summary is a lossy channel for review findings, where a report file in the worktree is
    reliable; and orchestration inbox is not Run-scoped, so a coordinator reading the inbox rather than the
    wait result could attribute a stale report to the current Run.

Closure gates on exact HEAD de50905

All nine blocks exit 0: git diff --check; jq -e . over five manifests; node --check on the helper; shell
syntax over the three shipped scripts enumerated by git ls-files; both version gates; the fifteen absence
commands; the identifier grep over skills/; the two disclosure greps.

What this does not do

harnesses.json, a harness-agnostic setup skill, a shorter SKILL.md, the JSONL log, preflight moved to setup
and after a NO_ACK, and dely with no arguments printing version and hashes are all the next delivery's
contract, on main.

Review

CHANGES_REQUESTED at de50905, remediated at 984fa22.

The independent session reproduced all ten closure-gate blocks on exact de50905, and reproduced the three
gates discriminating against seeded wrong implementations: the Codex manifest left at 0.18.0, a hasGate
caller left in skills/ without its definition, and a broken probe/mkrepo.sh. It also reproduced the one
runtime change against a stub Orca — stage.worker: start_unknown with dispatchStatus: pending and a
reworded dialog matching no old gate string — where this head fails in 0.4 s and calls worker-stop and
worker-release once each, while 3ee4e1a waits out the budget. It did not reinstall the candidate or rerun
the live rows.

Every finding was a contradiction between this delivery's documentation and this delivery's own measurements.
984fa22 is documentation only; nothing under skills/ changes, which the diff shows:

  1. docs/decisions.md claimed it "verified rows 1, 4, 5 and 7". It now says rows 1, 5 and 7 passed, that row 4
    ran and failed on Orca 1.4.203 with the projection numbers and the note that wait's ATTENTION path is
    unchanged from 0.18.0, and that four of seven rows was a recorded exception rather than the floor.
  2. probe/checklist.md said row 1 was the release floor, contradicting the record's own Deferred section and
    decision 15 of 2026-09-15. The floor is all seven rows; a release that runs fewer names which it skipped.
  3. probe/checklist.md step 1 told the next operator to install the snapshot into Cursor with the README
    commands, which this delivery had already measured to be impossible. It now states that Cursor runs off the
    Claude plugin cache, gives the command that finds older Cursor copies, and says a Cursor row that cannot be
    trusted is reported rather than run. It also warns about codex plugin marketplace add keeping a stale
    marketplace while reporting success.
  4. probe/trust.sh now reads with --screen, as the checklist tells everyone else to.
  5. .github/ISSUE_TEMPLATE/bug_report.yml marks the four deferred harnesses.

One further note the reviewer recorded for the next delivery, not fixed here: the plan was deleted at
4e0448d and three candidate commits followed it, one of them a runtime change. The record carries that
change, so nothing is lost, but the skill puts the plan deletion in the release commit.

Closure gates rerun on exact 984fa22: all nine blocks exit 0.

Awaiting re-review of the de50905..984fa22 diff.

🤖 Generated with Claude Code

hieuphung97 and others added 30 commits September 11, 2026 16:12
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnfpfr9czVFGiXVksmrFDB
…vents.

Control can no longer compose worker launches by hand: dely.js owns ACK, adopt quiescence, silence, and deadline so later verify and skill tasks have a stable command surface.

Co-authored-by: Cursor <cursoragent@cursor.com>
…tch cannot swallow settlement.

Co-authored-by: Cursor <cursoragent@cursor.com>
…le-batch consumer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnfpfr9czVFGiXVksmrFDB
A heartbeat-typed wait can no longer hide a batched worker_done: dispatch never consumes, wait and sidecar judge the frozen FIFO batch, and collect reads acknowledged history.

Co-authored-by: Cursor <cursoragent@cursor.com>
… of this delivery.

Codex's weekly limit was 97% used with about five reviews left; the human chose to
move the review pin rather than pause until 2026-09-15.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnfpfr9czVFGiXVksmrFDB
A consuming typed wait must replay the frozen Delivery when a newer matching type arrives, and collect must fail closed on a draining check instead of collapsing distinct settles.

Co-authored-by: Cursor <cursoragent@cursor.com>
Dispatch needs a PASS verdict for this repo, these pins and this Control harness; verify proves that path and restores the previously bound Run so a later worker-start can still run.

Co-authored-by: Cursor <cursoragent@cursor.com>
…immediately on a failed launch.

Co-authored-by: Cursor <cursoragent@cursor.com>
…ute, as decision 8 of the 2026-09-11 record requires.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnfpfr9czVFGiXVksmrFDB
…ses launches or retries into the same terminal.

Co-authored-by: Cursor <cursoragent@cursor.com>
… sidecar into its remediation; the adopt terminal leak into task 4.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnfpfr9czVFGiXVksmrFDB
…at inverted sentences were passing.

Collect now reports DEADLINE from dispatchedAt and keeps one sidecar per Run so nudge-mode recovery can see those exits.

Co-authored-by: Cursor <cursoragent@cursor.com>
… be woken by any message and Orca already reports a dead worker.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…unch failed on a shell prompt and left its terminal behind.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…erminal it cannot find or judge live.

Co-authored-by: Cursor <cursoragent@cursor.com>
…dispatch, pin the sentence that states what removing the sidecar costs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…and pin the sidecar-removal limit in both skills.

Co-authored-by: Cursor <cursoragent@cursor.com>
…5 to say collect reads liveness for a dispatch that is no longer open.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…forget it and a live dispatch can still settle.

Co-authored-by: Cursor <cursoragent@cursor.com>
…eport-once must be Dely's own memory, not a terminalState Orca may never set.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
… so collect forgets a dead worker even when Orca leaves it retained.

Co-authored-by: Cursor <cursoragent@cursor.com>
…outside a git repository, and the rest of the review's findings are deferred.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…repository, so collect still reports a dead worker once.

Co-authored-by: Cursor <cursoragent@cursor.com>
… to the case it covers, and defer the unwritable git directory.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…became ready.

A failed launch must not keep launching pins or leave the created terminal, and a throw after launch must reuse verify's cleanup and FAIL verdict before restoring the previous Run.

Co-authored-by: Cursor <cursoragent@cursor.com>
hieuphung97 and others added 7 commits September 14, 2026 11:48
…ompt; it retries --enter, then leaves the result file.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcUecPMViKUf1gJRBLnsJc
… the block clears, then give up.

Co-authored-by: Cursor <cursoragent@cursor.com>
…val env values are not positive.

A non-numeric retry used to hang Atomics.wait; the skill still described the removed bare-return send.

Co-authored-by: Cursor <cursoragent@cursor.com>
…lure lines quote the worker's screen before release, preflight waits longer.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WcUecPMViKUf1gJRBLnsJc
…rs, and wait 150s for each preflight pin.

Co-authored-by: Cursor <cursoragent@cursor.com>
…d transcript failure text comes from message blocks.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hieuphung97 hieuphung97 changed the title Prove the dispatch path with dely:verify and wake Control on events (0.18.0) Coordinate on Orca's supervised loop with a small dispatch helper (0.18.0) Sep 14, 2026
@hieuphung97
hieuphung97 marked this pull request as ready for review September 14, 2026 08:28
hieuphung97 and others added 4 commits September 14, 2026 19:18
… for waker Controls, quote the gate, fail preflight early, fix the skill and Grok Setup.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Tnfpfr9czVFGiXVksmrFDB
…ker Controls, and fail preflight when a gate stays on screen.

Co-authored-by: Cursor <cursoragent@cursor.com>
…DELY_WAITER is set, and ignore non-blocking usage footers.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hieuphung97
hieuphung97 marked this pull request as draft September 14, 2026 13:30
hieuphung97 and others added 2 commits September 14, 2026 20:40
…, and drop a failed worker-list row instead of waiting out the budget.

Co-authored-by: Cursor <cursoragent@cursor.com>
… NO_ACK.

Co-authored-by: Cursor <cursoragent@cursor.com>
@hieuphung97
hieuphung97 marked this pull request as ready for review September 14, 2026 14:17
hieuphung97 and others added 9 commits September 16, 2026 08:47
…supported harnesses, and plan the deletions.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…s the whole structural check.

The suite was a change detector: 89 CI runs, 86 green, and no defect in this
repository's shipped code first found by it, against 56 commits spent on
contracts.sh versus 39 on everything under skills/. The fake Orca it ran on
diverged from the real CLI in four shapes and stayed green through every probe
round that failed.

The version pin moves out of contracts.sh into a literal gate in AGENTS.md, so
a skills/ change that forgets one manifest is a red gate rather than a silent
split between the Claude and Codex installs.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Gone: the Antigravity adopt path and the six temp-file helpers behind it, the
nine-string GATES classifier and the two preflight polls built on it, pinWhy,
dely.cmd, and the launcher's Electron fallback. The helper is 478 lines, from
688.

A failure quote is now the tail of the worker's own screen rather than the
lines that matched a gate string. Two of those strings had already been removed
as footer false positives, and the quote is checked live by the untrusted-pin
row of the probe checklist, not by a string list.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The README keeps Claude Code, Codex CLI and Cursor Agent CLI, and the setup
skill discovers models and effort for those three. Grok Build, Antigravity CLI,
Kiro CLI and GitHub Copilot CLI keep their measured launch mechanics in
references/harnesses.md and lose their install commands, their discovery
commands and their checked-version rows.

This narrows what is documented. It does not declare a closed set: the wording
that was open stays open, and a deferred harness returns as a status change.

Checked versions are refreshed to what is installed here: Claude Code 2.1.273,
Codex CLI 0.154.0, Cursor Agent CLI 2026.09.10-fd3934a, Orca 1.4.203.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
probe/checklist.md carries the seven live rows, the snapshot install with its
hash check, the fixed probe paths, and what each row leaves behind.
probe/mkrepo.sh builds a probe repository with a bare remote and a
delivery-sized task. probe/trust.sh acts as the human on a Claude trust dialog
and reports trust from ~/.claude.json, not from the screen: the earlier probe
script reported success twice while hasTrustDialogAccepted stayed false.

Both scripts resolve their target and refuse anything outside ~/dely-probe/
before creating a directory or an Orca terminal. No skill references probe/.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Everything durable is in the 2026-09-16 decision record; the live results go in
the pull request body.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Row 5 of the probe checklist failed on the pushed candidate: an untrusted
Claude pin was refused correctly and quoted the dialog, but took 192 s instead
of the 39 s the deleted gate classifier used to manage. The classifier was the
only early-failure path, and deleting it left only the 150 s timeout.

Orca already reports this without any string matching. On an untrusted pin the
worker-list projection holds stage.worker "starting" for about 40 s and then
flips to "start_unknown" with detail "turn_start_unobserved", while
dispatchStatus stays "pending" the whole time, which is why the failed-row poll
never fired. Preflight now drops on that projection as well as on a failed row.

The trailing sleep in the poll loop goes with it: it existed so that the two
consecutive gate observations were spaced, and there are no gate observations
left. The loop now costs one check --wait per turn instead of two.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude Code 2.1.273 asks "Quick safety check: Is this a project you created or
one you trust?", so the script's "do you trust" pattern never fired and it
reported NOT_TRUSTED after 60 s with the dialog plainly on screen. It now
matches several parts of the dialog.

The wording is a detector for when to answer, never for whether the answer took:
that stays the store check, which is the reason this script exists.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Three things the first live run needed and the file did not say: a terminal read
needs --screen or it returns stacked fragments and, after one read, an empty
tail; worker-list can report an empty taskTitle, so a dispatch is identified by
id and order; and the hash check has to cover the marketplace source directory,
because a Claude Control was seen running scripts/dely from there rather than
from the plugin cache.

The killed-worker row now says whose signal it is. It reads Orca's
projection.nextAction, that projection has changed between Orca releases, and a
row where the agent process is gone while terminalState stays active is a
finding about the execution plane rather than about the helper.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@hieuphung97 hieuphung97 changed the title Coordinate on Orca's supervised loop with a small dispatch helper (0.18.0) Retire the structural suite and narrow Dely to the harnesses it can support (0.19.0) Sep 16, 2026
Remediation for the CHANGES_REQUESTED review of de50905. Documentation only;
nothing under skills/ changes.

The record claimed it verified rows 1, 4, 5 and 7. Row 4 ran and failed: on Orca
1.4.203 a killed worker kept terminalState "active" and nextAction "none" for
6 min 8 s, against 16 s to ATTENTION on 1.4.200, while wait's ATTENTION path is
unchanged from 0.18.0. The record now says that, names the Orca version, and
says running four of seven rows was a recorded exception rather than the floor.
It also carries the real cost of dropping the dialog classifier: 56 s now
against the classifier's 39 s, with 192 s as the timeout-only case in between.

The checklist said row 1 was the release floor, which contradicted the record's
own Deferred section and the decision it came from. The floor is all seven rows;
a release that runs fewer names which it skipped.

Step 1 told the next operator to install the snapshot into Cursor with the
README commands. That is not possible: cursor-agent plugin marketplace add takes
a git URL only and installing needs the interactive /plugin panel. Step 1 now
says Cursor runs off the Claude plugin cache, gives the command that checks for
older Cursor copies, and says a Cursor row that cannot be trusted is reported
rather than run. It also warns that codex plugin marketplace add was seen
keeping a stale marketplace and installing the previous version while reporting
success.

trust.sh reads with --screen, which is what the checklist tells everyone else to
do. The bug report template marks the four deferred harnesses so a reporter is
not promised support that does not exist.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@hieuphung97
hieuphung97 merged commit 36ef5cf into main Sep 16, 2026
@hieuphung97
hieuphung97 deleted the feat/verify-event-driven-dispatch branch September 18, 2026 01:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant