fix: reliable slash-command initial prompt delivery for RunPane creates#360
Conversation
QA proof — command-shaped verification on final tree
|
type: wrap-up-report
|
type: postmortem
|
| Step / dispatch | Start | Dur | Tokens | Note |
|---|---|---|---|---|
| Step 0 preflight + load + branch FF | 22:53 | ~7m | (overseer) | branch fast-forwarded v2.4.5→v2.4.32 |
| Step 1 direct research + plan write | 23:00 | ~7m | (overseer) | zone 2: no dossier |
| plan-reviewer | 23:07 | ~4m | 62,566 | 3 MF + 1 SF → folded into plan |
| implementer (initial, full slice) | 23:11 | ~13m | 189,577 | DONE, 381/381 |
| backend-verifier #1 | 23:24 | ~3m | 74,898 | fail: 3 assertion-strength gaps |
| implementer fix 1 (tests) | 23:27 | ~3m | 94,632Δ | resume session |
| backend-verifier #2 | 23:30 | ~5m | 41,143 | pass |
| build gate + commit + rebase + PR open | 23:43 | ~6m | (overseer) | PR #360 |
| code-reviewer (aborted) | 23:35 | ~5m | 15,578 | wasted: repo missing .claude/agents/code-reviewer.md |
| PR diagram author + host | 23:45 | ~10m | (overseer) | overlapped with review lane |
| code-reviewer pass 1 (retry) | 23:41 | ~30m | 53,910 | 3 MF (1 security) |
| implementer fix 2 (3 MF) | 00:14 | ~5m | 88,044Δ | zone escalated 2→1 |
| review pass 2 (scoped) | 00:19 | ~3m | 56,143 | MF-2 fixed; MF-1/MF-3 persist, sharper |
| implementer fix 3 | 00:22 | ~5m | 248,409Δ | output-only signal + race-safe handoff |
| review pass 3 (scoped) | 00:27 | ~2m | 45,822 | MF-3 fixed; MF-1 narrowed to same-ms edge |
| implementer fix 4 (gen counter) | 00:29 | ~3m | 39,630Δ | post-cap, duplicate-safe direction |
| backend-verifier #3 (final QA) | 00:32 | ~3m | 58,013 | pass, 388/388 |
| body/QA comment + wrap-up + postmortem | 00:36 | ~9m+ | (overseer) |
Phase aggregates: plan ≈16% of wall-clock, implement+verify ≈33%, post-PR review loop
≈43%, wrap-up ≈8%. Summed turnaround gaps (Overseer picking up completed dispatches):
small — the in-turn wait loops and notifications kept pickup within ~1 min of each
marker.
Ranked stalls (none needed a human nudge):
- code-reviewer pass 1 ran ~30 min against low-effort expectations (~10 min) — the
single biggest wall-clock block; partly queue/model variance, no action beyond noting. - Wasted code-reviewer dispatch (~5 min + 15.6k tokens): the repo's synced
.claude/agents/set lackscode-reviewer.md; the role charter correctly aborted.
Preflight had verified the plan-reviewer file but not all roles the run would use. - Baseline
pnpm --filter main testran in Vitest watch mode and never exited
(killed ~40 min later) — no wall-clock cost (backgrounded) but an occupied slot and a
misleading "still running" signal;CI=1was the known fix, applied everywhere after. - Node ABI mismatch (default Node 26 vs native modules built for Node 22) cost each
fresh Codex dispatch a false-fail + re-run of the DB tests until it re-derived the
PATH=/opt/homebrew/opt/node@22/binrecipe from AGENTS.md.
Blocker inventory: 0 AskUserQuestion gates, 0 rate-limit hits, 0 red gates; all
waits were background-dispatch waits. The single change that removes the biggest
avoidable stall: preflight-verify the instruction files for every role the run's
zone will dispatch (see proposals) — it converts the wasted reviewer dispatch and its
retry latency into a Step 0 one-liner.
What we asked for
Reliable slash-command initial prompt delivery for RunPane creates: per agent × shape
routing, earned verifiedSubmitted, evidence-gated duplicate-free retry, structured
submission_unverified blocker (item #358, AC1–AC7).
Outcome vs intended
On-target — no outcome gap known at wrap-up. All ACs carry quoted automated evidence;
the outcome half of this postmortem is deferred until the human's PR review.
Why the gap happened
N/A — operational-only postmortem.
What to change so it doesn't recur
claude/skills/codex/SKILL.md(orchestra) — path-resolution fallback. In
"Path resolution", after "Confirm both files exist before dispatching", add:
"When the repo-local.claude/agents/<role>.mdis absent, fall back to the
canonical~/.claude/agents/<role>.mdand note the sync gap in the wrap-up —
never dispatch a role whose instructions resolve nowhere." (This run burned a
dispatch exactly here.)claude/skills/do/SKILL.md(orchestra) — preflight the full role roster. In
Step 0 preflight, add: "Resolve the instruction + format file for every sub-agent
role this run's zone will dispatch (plan-reviewer, implementer, verifiers,
code-reviewer) and confirm each exists — a role file discovered missing at dispatch
time burns the dispatch."claude/skills/do/SKILL.md(orchestra) — baseline tests in CI mode. In Step 0's
environment-readiness paragraph, add: "Any baseline or gate test run uses the
repo's non-interactive invocation (CI=1/vitest run/ equivalent) — watch-mode
scripts never exit and poison background-task signals."references/zones.md(orchestra) — escalator addition. Add to the escalator
list: "construction of shell command lines from user- or agent-supplied text"
forces zone 1 minimum. This run's zone-2 cap of 1 post-PR pass would have shipped a
command-injection bypass; only a discretionary in-run escalation bought the passes
that caught and fixed it. Also sync the repo: addcode-reviewer.md(and
frontend-verifier.md,discussant.md) to the agent set synced intodcouple/Pane.
Dial record & right-sizing
(Full block in wrapup.md; key figures)
zone: 2 (escalated in-run to 1)
lanes: single-codex
passes: {plan: 1/1, post_pr: 3/3}
findings: {plan: {pass1: {codex: 3}}, post_pr: {pass1: {codex: 3}, later: {codex: 0 new}}}
verifiers: {frontend: skipped, qa_pass: trimmed}
qa_findings: 0
wall_clock: 1:52
pr_size: {files_changed: 13, additions: 1062, deletions: 88}
tokens: {codex: 1068365, overseer: 438500, claude_subagents: 0, total: 1506865}
spend_ratio: 1310.3Judgment: underdone at the captured zone, right-sized after escalation. Every
review pass paid: plan pass caught 3 real MF pre-implement; post-PR pass 1 caught a
security bug plus two duplicate/loss mechanisms; passes 2–3 (scoped, ~50k tokens each)
each converted a "claimed fixed" into a sharper persisting finding before confirming.
The dial that mattered: the item's zone: 2 — the injection-adjacent surface
(prompt text → shell command line) warranted zone 1 at capture (proposal 4).
Acceptance
Still awaiting review (operations-only postmortem; PR labeled
awaiting-human-review).
System changes
Proposals 1–4 above — recorded, not applied; verdicts pending the human.
Comment URLs: filled after publishing (see below).
e10cd0d to
44ae7ce
Compare
1 participant
Summary
What — RunPane pane creates now choose initial-input delivery per agent × prompt shape, and the create result's
verifiedSubmittedis earned from evidence instead of assumed: fresh Claude creates deliver the prompt as a CLI argument (probe-verified to execute slash commands), Codex slash-command input routes through the semantic composer path with an evidence-gated retry, and the unconditionalverifiedSubmitted: truefor Codex argument delivery is gone.Why — Two silent failure modes broke slash-command startup (issue #358, peeled from #353): (1) in a four-pane dogfood run,
/do TM-xsat staged in 3 of 4 composers because the autocomplete menu swallowed the bare\r; (2) Codex creates reportedverifiedSubmitted: trueunconditionally whilecodex "/status"provably bypasses the TUI command layer — the model improvised on literal text and the orchestrator was told everything succeeded.How — The routing decision lives in
createTerminalPanelForSession(argument mode for Claude and non-slash Codex; composer mode with thecodex-ctrl-entersequence for slash-shaped Codex input). The create-only composer loop (submitCreateComposerInput) verifies with a new pure evidence predicate (assessComposerEvidence:staged/cleared/unknown) and retries only on staged-text evidence confirmed stable across two fresh readbacks; anything ambiguous degrades to a structuredsubmission_unverifiedblocker with attempt count andnextCommand— never a duplicate submit, never a hang, never a false success. Contract changes are additive and shipped across all parity surfaces (daemon contract JSON, generated npm/shared/Python contracts, hand-written CLI types, docs). The publicpanels submit-composercommand is untouched.Done means: every create shape (
{claude, codex, custom}×{slash, prose, multiline}×{wait-ready, no-wait-ready}) delivers through its correct mechanism,verifiedSubmitted: trueappears only with delivery evidence, a swallowed submit is retried exactly once on stable staged evidence, and an unverifiable submission returnsok: falsewith the full AC5 blocker payload in bounded time.Visual overview
Before: both failure paths were silent — the composer's bare
\rgot eaten by autocomplete (text stayed staged, nothing ran), and Codex argument delivery reportedverifiedSubmitted: trueunconditionally while the model received the slash command as literal text. After: delivery routes per agent × prompt shape; the composer lane verifies against screen-readback evidence, retries at most on two stable staged readbacks, and returns a structuredsubmission_unverifiedblocker when evidence is ambiguous.User journeys
Journey map — orchestrator creates a pane with a slash command: an agent runs
runpane panes create --agent claude --prompt "/do TM-x" --wait-ready --yes --json→ Pane launchesclaude --session-id <uuid> "/do TM-x"(the command executes as a command — no composer, nothing to swallow) → the JSON result reportsstrategy: "argument",verifiedSubmitted: true→ the orchestrator proceeds on trustworthy state. On Codex the same prompt routes through the composer with the semantic Ctrl+Enter sequence; if verification cannot be earned the orchestrator getsok: false+blocked.kind: "submission_unverified"+ a ready-to-runnextCommandinstead of a silent stall.Flow map — the delivery decision forks at
main/src/ipc/runpane.ts(shouldUseArgumentDelivery/createTerminalPanelForSession):Gap flags: J4 is proven by fixture-driven unit tests only — deliberately, since a real swallowed-autocomplete race can't be staged on demand; the fixtures encode the dogfood failure signature.
Verification
Automated evidence from the backend-verifier passes (full logs quoted in the verification runs; suite: 43 files / 381 tests passing, plus
pnpm typecheck,pnpm lint0 errors,pnpm run test:runpane-contractparity green):claude ... --session-id ... "/do TM-x"; create result assertsstrategy: 'argument',verifiedSubmitted: true,writeToTerminalnever called for delivery.'\x1b[13;5u\r'asserted); success only after cleared+activity evidence; forced-failure fixture returnsverifiedSubmitted: false(the old unconditionaltrueis structurally unreachable).toHaveBeenCalledTimes(3)total (1 input write + exactly 2 submit writes),verifiedSubmitted: true,attempts: 2.submitted: false,attempts: 1.ok: false,staged: true,verifiedSubmitted: false,attempts: 3,blocked.kind: 'submission_unverified', exactnextCommand; bounded-duration assertion≤ 3 × (3000 + 500)ms.initialInput.verifiedSubmitted === truefor every wait-ready prose/multiline combination on both agents./frobnicate xpreserved byte-exact as the first write; unverified outcome asserts the full AC5 blocker shape.confirmationVerdict === 'staged'AND an unchanged composer region across two fresh samples;unknownnever authorizes retry.quoteCommandArgument, dropped prompt on wait-ready timeout), all fixed across three commits — final state: output-only generation-counter freshness gate for retries, backslash-safe quoting, race-safe premark clearing with explicit exactly-once delivery handoff. Final QA verification on the finished branch: 388/388 tests, typecheck, lint 0 errors, contract parity — all pass with the fix behaviors pinned by named tests. The last commit (generation counter,e10cd0d) landed after the reviewer cap; it was verified by tests + Overseer inspection, and its only failure direction is duplicate-safe (a missed retry degrades to the blocker). CLI rubric blockers (exit codes, idempotency, help accuracy) all pass.Manual tests
Must (breaks orchestrator trust if wrong):
runpane panes create --repo <r> --agent codex --prompt "/status" --wait-ready --yes --json→ result is eitherverifiedSubmitted: truewith the command actually executed in the pane, orok: falsewithblocked.kind: "submission_unverified"and a runnablenextCommand— never a silenttruewith literal text sent to the model. — left to human: needs a live Pane daemon + real agent CLIsstaged,attempts,blocked.message,nextCommand; running thenextCommandshows the composer state. — left to human: needs a live Pane daemon + real agent CLIsImportant (user-facing behavior):
runpane panes create --agent claude --prompt "/do <item>" --wait-ready --yes --json→ pane's launch command carries the quoted prompt argument; slash command expands in the Claude TUI; result reportsstrategy: "argument". — left to human: needs a live Pane daemon + real agent CLIsverifiedSubmitted: true). — left to human: needs a live Pane daemon + real agent CLIs--initial-input-file) → prompt arrives intact, session starts on it. — left to human: needs a live Pane daemon + real agent CLIs--wait-ready→ pane receives the text + Ctrl+Enter sequence (not a bare Enter); result makes no delivery claim. — left to human: needs a live Pane daemon + real agent CLIsNice (cosmetic):
--commandtool with initial input → unchanged raw delivery, no new fields in output. — left to human: needs a live Pane daemon + real agent CLIsafter N attempts; staged: yes/nophrasing only when those fields exist. — left to human: needs a live Pane daemon + real agent CLIsAreas not affected: GUI-created sessions and panels, resumed Claude/Codex sessions,
panels submit-composer(public behavior untouched), git/worktree operations, frontend renderer.QA results
Command-shaped QA executed (zone dial: no user-visible surface → command-shaped items only): full main suite 388/388, typecheck, lint (0 errors), and npm/Python contract parity all pass on the final tree (
e10cd0d), with the three review-fix behaviors each pinned by named tests — quoted evidence in the QA proof comment. 0 of the 8 Manual-test items were driven automatically: all require a live Pane daemon with realclaude/codexCLIs attached, which this run's environment does not exercise — they remain for the human, starting with the two Must items.Residual risks
unknown→ blocker rather than retry. Worst case is an actionableok: false, never a duplicate — but some genuinely swallowed submits may reportsubmission_unverifiedinstead of self-healing.cleared + activity≈clearedduring CLI boot (byte-silence activity heuristic is noisy); real semantic activity detection is epic feat: Event-driven RunPane orchestration #356 phase 1.Closes #358