From f0ecd77a76d8357f935526da2308dc82d13888cc Mon Sep 17 00:00:00 2001 From: khaliqgant Date: Sun, 20 Sep 2026 17:45:04 -0700 Subject: [PATCH 1/3] docs(orchestrating-agent-relay): 12.4 PR-owner GitHub subscribe Audit the orchestrating skill against Agent Relay 12.4 and make subscribing the live PR-owner agent to the PR's GitHub provider resource the durable default. Cover rebinding after respawn, unsubscribe/release hygiene, no duplicate writable helper subscriptions, and verifying actual event delivery. Salvage still-valid #56 workflow lessons (idle detection, per-turn interactive spawn, LLM-as-oracle, repair-step counterweight) into writing-agent-relay-workflows. --- README.md | 4 +- prpm.json | 6 +- skills/orchestrating-agent-relay/SKILL.md | 166 ++++++++++++++++-- skills/writing-agent-relay-workflows/SKILL.md | 22 +++ 4 files changed, 180 insertions(+), 18 deletions(-) diff --git a/README.md b/README.md index 82abf45d..f5a7f9c8 100644 --- a/README.md +++ b/README.md @@ -9,11 +9,11 @@ Package metadata lives in [prpm.json](prpm.json). The repo currently publishes ` | Skill | Version | Description | |-------|---------|-------------| | [choosing-swarm-patterns](skills/choosing-swarm-patterns/SKILL.md) | 1.1.5 | Pick the right Agent Relay orchestration pattern across the 10 core swarm patterns plus specialized patterns. | -| [writing-agent-relay-workflows](skills/writing-agent-relay-workflows/SKILL.md) | 1.6.24 | Build multi-agent workflows with WorkflowBuilder, DAG dependencies, Relayfile-backed Slack human assistance, integration subscriptions, waitFor gates, review-depth review/fix loops, channels, and chat-native coordination recipes. | +| [writing-agent-relay-workflows](skills/writing-agent-relay-workflows/SKILL.md) | 1.6.25 | Build multi-agent workflows with WorkflowBuilder, DAG dependencies, Relayfile-backed Slack human assistance, integration subscriptions, waitFor gates, review-depth review/fix loops, channels, and chat-native coordination recipes. | | [setting-up-relayfile](skills/setting-up-relayfile/SKILL.md) | 1.1.1 | Set up Relayfile mounts and writeback for provider files through local filesystem access. | | [writing-relayflows](skills/writing-relayflows/SKILL.md) | 1.0.0 | Author a Relayflows v2 flow (`@relayflows/surface`/`@relayflows/sdk`, the `flows` CLI) in TypeScript or YAML/JSON — the run/llm/agent ladder, verification gates, cli/model resolution, flows.json, and real `flows check`/`run` refusal shapes. Not the older `@relayflows/core` WorkflowBuilder engine. | | [using-agent-relay](skills/using-agent-relay/SKILL.md) | 1.4.0 | Participant-side MCP reference for a **registered** relay agent (spawned worker / registered lead): messaging, channels, threads, reactions, search, inbox, actions, and worker spawn/release. Counterpart to `orchestrating-agent-relay`. | -| [orchestrating-agent-relay](skills/orchestrating-agent-relay/SKILL.md) | 2.3.1 | The canonical way to run agent-relay: self-bootstrap the broker (`agent-relay node up`) and autonomously spawn, monitor, and coordinate a worker team over the relay MCP without human intervention. | +| [orchestrating-agent-relay](skills/orchestrating-agent-relay/SKILL.md) | 2.4.0 | The canonical way to run agent-relay: self-bootstrap the broker (`agent-relay node up`) and autonomously spawn, monitor, and coordinate a worker team over the relay MCP without human intervention, including GitHub PR-owner integration subscriptions. | | [relay-80-100-workflow](skills/relay-80-100-workflow/SKILL.md) | 1.0.9 | Author workflows that close the 80-to-100 validation gap with repair-aware test, verify, review-depth review/fix with test hardening, and commit gates. | | [review-fix-signoff-loop](skills/review-fix-signoff-loop/SKILL.md) | 1.0.2 | Loop review, repair, validation, and fresh-context dual-agent signoff until independent reviewers both satisfy the verdict contract. | | [trigger-autocomplete-catalog](skills/trigger-autocomplete-catalog/SKILL.md) | 1.0.0 | Enforce webhook/event trigger autocomplete coverage through KNOWN_TRIGGER_CATALOG in @relayfile/adapter-core. | diff --git a/prpm.json b/prpm.json index 55cf53e5..51bdfd4a 100644 --- a/prpm.json +++ b/prpm.json @@ -47,7 +47,7 @@ }, { "name": "writing-agent-relay-workflows", - "version": "1.6.24", + "version": "1.6.25", "description": "Use when building multi-agent workflows with relay broker-sdk. Covers conversation vs pipeline coordination, WorkflowBuilder/DAG steps, agents, {{steps.X.output}} chaining, Relayfile-backed human assistance, Slack answer injection, integration subscriptions, waitFor gates, repairable verification, review-depth fresh-eyes review/fix loops with test hardening, channels, chat-native recipes, error handling, event listeners, step sizing, lead+workers teams, and parallel waves.", "format": "claude", "subtype": "skill", @@ -136,8 +136,8 @@ }, { "name": "orchestrating-agent-relay", - "version": "2.3.1", - "description": "The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, and team coordination.", + "version": "2.4.0", + "description": "The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, GitHub PR-owner integration subscriptions, and team coordination.", "format": "claude", "subtype": "skill", "tags": [ diff --git a/skills/orchestrating-agent-relay/SKILL.md b/skills/orchestrating-agent-relay/SKILL.md index 23b8d937..491d40e6 100644 --- a/skills/orchestrating-agent-relay/SKILL.md +++ b/skills/orchestrating-agent-relay/SKILL.md @@ -1,6 +1,6 @@ --- name: orchestrating-agent-relay -description: The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, and team coordination. +description: The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, GitHub PR-owner integration subscriptions, and team coordination. --- # Orchestrating Agent Relay @@ -66,7 +66,12 @@ reference is the **`using-agent-relay`** skill. | Read worker replies (MCP) | `check_inbox(limit: 20)` / `list_messages(channel: "general")` | | Give a human a follow-along link | `agent-relay observer` | | Inspect a worker's TTY | `agent-relay node agent attach Worker1 --mode view` | -| Release worker | `agent-relay node agent release Worker1` | +| Fleet-wide agent inventory | `agent-relay fleet agent list` | +| Subscribe PR-owner to GitHub PR | `agent-relay integration subscribe github --resource '/github/repos///pulls//**' --to @Owner` | +| List integration bindings | `agent-relay integration subscribe --list` | +| Unsubscribe a provider resource | `agent-relay integration unsubscribe github --resource '...'` | +| Failed deliveries | `agent-relay node deadletters` | +| Release worker | `agent-relay node agent release Worker1` / `agent-relay fleet release Worker1` | | Stop broker | `agent-relay node down` | ## Bootstrap Flow @@ -146,9 +151,12 @@ add_agent( ``` `node agent spawn` takes the provider as a positional argument -(`claude`, `codex`, `gemini`, `droid`, …) and `--name` / `--task` / `--channels` -/ `--model` / `--cwd` flags. By default the agent joins the `general` channel and -runs in `interactive` spawn mode; pass `--exit-after-task` for a one-shot worker. +(`claude`, `codex`, `gemini`, `grok`, `opencode`, `muse`, `devin`, `droid`, …) +and `--name` / `--task` / `--channels` / `--model` / `--cwd` flags. By default +the agent joins the `general` channel and runs in `interactive` spawn mode; pass +`--exit-after-task` for a one-shot worker. Fleet placement uses +`agent-relay fleet spawn ` (or MCP `spawn` / `add_agent`); `droid` remains +high-risk for delegation. > **Expect a 30–60s gap between spawn and the first ACK.** A worker shows in > `node agent list` within ~5s (the process is up), but the underlying CLI @@ -226,13 +234,19 @@ agent-relay node metrics ### Step 4: Release Workers +Unsubscribe any provider resources that worker owned (see +[Subscribe the live PR owner](#subscribe-the-live-pr-owner)) before release. + ```text remove_agent(name: "Worker1", reason: "Work accepted") ``` -CLI equivalent: +CLI equivalent — prefer the control-plane form when the worker may not be on +this broker's default `connection.json`: ```bash +agent-relay fleet release Worker1 +# local-only fallback: agent-relay node agent release Worker1 ``` @@ -425,11 +439,19 @@ Quick Reference. Then enforce this protocol: over the relay MCP — never `node tail` (that streams broker debug events, not worker messages). See the "Channel vs DM" section for the full reading model -- Poll `agent-relay node agent list` for worker liveness; set a wall-clock - fallback so a silently-dead worker can't hang the loop +- Poll `agent-relay node agent list` (local) or `agent-relay fleet agent list` + (cross-node) for worker liveness; set a wall-clock fallback so a + silently-dead worker can't hang the loop +- The moment a worker opens or owns a GitHub PR, subscribe **that live + PR-owner identity** with `agent-relay integration subscribe github + --resource '/github/repos///pulls//**' --to @`. + Do not subscribe helpers to the same glob. After respawn under a new + name, rebind. Verify a real GitHub event arrives in `check_inbox`, not + just that `--list` shows a binding. Unsubscribe owned resources before + release - If a human is watching, give them a follow-along link with - `agent-relay observer` and print the URL it returns. Never print the - workspace key or put it in a URL + `agent-relay observer` (or MCP `get_observer_url`) and print the URL it + returns. Never print the workspace key or put it in a URL ``` ## Multi-Round Review Loops (DONE → NO-GO → fix → re-review) @@ -476,7 +498,8 @@ Do NOT self-remove — stay alive for re-review." Always pass branch + commit SHA + the **complete** reviewer verdict. A fresh worker has none of the loop's history; a summarized verdict loses the -specifics it needs to fix. +specifics it needs to fix. Rebind any GitHub PR subscription to the new +live identity (see below) — a new name does not inherit the old route. ### Detecting a silently-dead worker @@ -492,6 +515,116 @@ worker still thinking. Defenses: - Always set a wall-clock fallback (e.g. a ScheduleWakeup ~30 min out) so a silently-dead worker can't hang the loop forever waiting on a message that will never arrive. +- Across nodes, `agent-relay fleet agent list` joins live broker agents + against the workspace roster. `idle` means the harness is at its prompt, + not that the assigned work is complete. Roster-only presence is not a live + worker. + +## Subscribe the live PR owner + +Whenever a worker **opens or owns a PR**, subscribe **that live PR-owner +agent** to the PR's GitHub provider resource immediately — the moment the PR +number exists, not after the first review comment. Review comments, pushes, +check changes, and discussion then wake the owner instead of waiting on the +next poll. + +```bash +agent-relay integration subscribe github \ + --resource '/github/repos///pulls//**' \ + --to @ +``` + +`--to` is `@agent` or `#channel`. Point it at the owner handle. `--spawn ` +can launch and confirm a recipient before binding; do not use it to replace an +already-live owner. `--events` defaults to Relay `message.created,thread.reply`. + +`subscribe` is not a chat listener. It provisions three artifacts: + +1. A Relay inbound webhook (`webhookId`) delivering into the recipient's + identity-bound channel +2. A Relayfile webhook subscription (`webhookSubscriptionId`) filtered by + `path_glob` +3. A Relayfile binding keyed by `(provider, resolved path glob)` + +`--resource` is a Relayfile VFS glob. `owner/repo` resolves to repository +scope; provider URLs are not accepted. A PR glob does **not** include GitHub +issue-comment paths (`/issues//comments/**`) even when those comments sit +on the PR. Reviews, root review-comments, and checks are inventoried at +repository-level `reviews/`, `comments/`, and `checks/` paths — a PR directory +does not cover them. Subscribe the PR glob first; add those extra globs only +when the owner must wake on those exact records. + +List bindings with `agent-relay integration subscribe --list`. This is +distinct from `agent-relay integration subscription list`, which lists Relay +workspace event subscriptions (`message.created`, …), and from +`agent-relay integration webhook create `, which mints an inbound +webhook URL that posts into a channel (it does **not** take a URL or `--event`; +that older shape is stale). + +### Ownership transfer / rebinding after respawn + +Bindings follow the **recipient identity**, not the OS process. + +- **Same name, identity kept** (`fleet release` / `node agent release` without + `--delete-agent`): the binding still points at `@Owner`. Respawn that name, + confirm it is live, and re-subscribe only if `--list` no longer shows + `@Owner` on that resource. +- **New name or deleted identity** (`--delete-agent`, or a fresh + `Implementer2`): the new agent does **not** inherit the old subscription. + Subscribe the new live identity to the same `--resource` immediately. The + binding key is `(provider, resolved path)` — a re-subscribe **replaces** the + previous route (create-first). Record the prior `webhookId` / + `webhookSubscriptionId` so you can confirm they retired. +- Do not repair a failed spawn by inviting a helper onto the owner's channel + or by pointing `--to` at a disposable worker. + +### Helper agents: no second writable subscribe + +Only the PR owner gets `--to @Owner` on that PR glob. A second `subscribe` to +the same `(github, path glob)` **replaces** the owner's binding. Reviewers, +shadows, and one-shot helpers read the PR through git/`gh` or channel traffic; +they do not get their own writable subscription to the owner's resource. + +### Unsubscribe / release hygiene + +Unsubscribe **owned** resources before releasing the owner, then verify the +IDs are gone: + +```bash +agent-relay integration subscribe --list +agent-relay integration webhook list +agent-relay integration webhook list-inbound +agent-relay integration unsubscribe github \ + --resource '/github/repos///pulls//**' +# confirm the binding, webhookId, and webhookSubscriptionId disappeared +agent-relay fleet release +``` + +Unsubscribe does not delete agents or channels. Do not unsubscribe by guessed +name, and do not delete a webhook another binding still references. A failed +re-subscribe must leave the prior working binding in place — do not tear it +down first. + +### Verify delivery, not just binding creation + +A successful `subscribe` (or a 2xx webhook create) is **not** proof that +GitHub events will wake the agent. Verify in this order: + +1. **Binding** — `agent-relay integration subscribe --list` shows + `(github, resolved PR glob) → @Owner`. +2. **Webhook inventory** — `integration webhook list` / `list-inbound` shows + the new `webhookId` (and the prior id retired after a replacement). +3. **Live recipient** — `fleet agent list` or `node agent list` shows `@Owner` + live on a node. Roster-only is not enough. +4. **Actual event** — after a real GitHub comment, review, check, or push, the + owner receives a Relay message from `github` whose body names the event and + a Relayfile path under that glob. Read it with `check_inbox` / + `list_messages`. Optional broker corroboration: a delivery-injected event + for that agent; `node deadletters` must not hold it. + +If steps 1–3 pass and step 4 does not, the subscription is not working. Do not +treat Cloud/Nango ingress success, an HTTP 200, or a channel post you wrote +yourself as delivery proof. ## Lifecycle Events @@ -730,9 +863,16 @@ this whole workaround once `relay#1446` lands rather than letting it outlive the | Targeted `fleet spawn` fails with `Targeted Fleet spawn requires an agent token` | Pass `--token` or set `RELAY_AGENT_TOKEN`; mint one with `agent-relay agent register --type system` (capture it without echoing). `--task` is also mandatory and the error only surfaces one problem at a time | | Node shows `online` but never receives a spawn | `online` ≠ available. Check `capabilities` contains `spawn:*` — a record can be live with no spawn capacity. Confirm with a throwaway targeted spawn, verified by `pgrep` **on the target host**, then release | | Harness blocks `sleep 25; check_inbox ...` | Bare foreground `sleep` wait loops are disallowed in harnessed environments. Run the poll loop with `run_in_background` (or Monitor + until-loop); the inline `sleep` snippets show logic only | -| Worker self-removed; can't send review fixes | Instruct workers not to self-remove until told. If already gone, spawn a fresh worker and re-inject branch + commit SHA + full verdict (see Multi-Round Review Loops) | +| Worker self-removed; can't send review fixes | Instruct workers not to self-remove until told. If already gone, spawn a fresh worker and re-inject branch + commit SHA + full verdict; rebind any GitHub PR subscription to the new live identity (see [Subscribe the live PR owner](#subscribe-the-live-pr-owner)) | | Told the user to open an observer URL built from the workspace key | That is an admin credential in a query string, and the realtime endpoint rejects it. Run `agent-relay observer` (or `get_observer_url`) and share the `ot_live_` URL it returns | -| Worker died silently; loop hangs | Inbox polling fires on messages only. Poll `agent-relay node agent list` for liveness and set a wall-clock fallback (~30 min ScheduleWakeup) | +| Worker died silently; loop hangs | Inbox polling fires on messages only. Poll `agent-relay node agent list` / `fleet agent list` for liveness and set a wall-clock fallback (~30 min ScheduleWakeup) | +| Opened a PR but did not subscribe the live owner | Review comments, pushes, and checks will not wake anyone. Immediately `integration subscribe github --resource '/github/repos///pulls//**' --to @` | +| Subscribed a helper/reviewer to the same PR glob | Binding key is `(provider, resolved path)` — the second subscribe **replaces** the owner's writable route. Only the PR-owner identity gets `--to @Owner` | +| Treated `--list` / webhook create as delivery proof | Binding creation is not a wake. Confirm a real GitHub event arrives as a Relay message from `github` with a Relayfile path under that glob (`check_inbox` / `list_messages`); check `node deadletters` if it does not | +| Respawned the owner under a new name and left the old binding | New identities do not inherit subscriptions. Re-subscribe `--to @NewOwner` on the same `--resource`; confirm the prior webhook/subscription IDs retired | +| Released the owner without unsubscribing | Unsubscribe owned resources, verify IDs disappeared, then `fleet release`. Unsubscribe does not delete agents | +| Used `integration webhook create --event …` | Current CLI takes `` (inbound webhook into a channel). Provider wakes use `integration subscribe`, not that older URL/event shape | +| Used `integration subscription list` to inspect PR bindings | That lists Relay workspace event subscriptions. Provider bindings are `integration subscribe --list` | ## Prerequisites diff --git a/skills/writing-agent-relay-workflows/SKILL.md b/skills/writing-agent-relay-workflows/SKILL.md index 35445e8b..7e27f40c 100644 --- a/skills/writing-agent-relay-workflows/SKILL.md +++ b/skills/writing-agent-relay-workflows/SKILL.md @@ -64,6 +64,8 @@ The point of an agent team workflow is not to discover a red gate and stop. The Avoid hard-stop gates (`failOnError: true` with no repair step) in workflows that are supposed to be self-healing. Even cheap preconditions such as missing credentials, wrong repository, or an unsafe dirty worktree should normally write a clear `BLOCKED_*` artifact and exit cleanly. For implementation, build, test, lint, schema, artifact, and review failures, model the fix path in the workflow. +**Counterweight — don't reflexively add a repair step.** This rule applies when the gate has a real, agent-fixable failure surface. A deterministic gate that sits downstream of deterministic work (e.g. a transcript line-count check after a series of deterministic grader steps) is correct by construction — there is no failure mode for a repair agent to address. Wiring a repair step there just spawns a fresh agent for a guaranteed no-op, paying the spawn cost (and often a follow-on idle-timeout wait) for nothing. Keep the hard gate, skip the repair. Reserve repair branches for gates where a real agent action could plausibly turn red into green. + ## Human Assistance and Relayfile Events Relayflows can block on humans and provider events without shelling out to Slack, GitHub, Linear, or the Relayfile CLI. Declare the integration intent in the workflow and let the SDK/runtime use the existing local Relayfile/Pear connection. @@ -266,6 +268,7 @@ Before writing the workflow, decide *how the agents will coordinate*. The relay | Shape | What it is | Use when | |---|---|---| | **Conversation** (chat-native) | Interactive agents share a channel; messages, `@-mentions`, and ambient awareness drive coordination. Lead and workers spawn in parallel and self-organize. The relay is the coordination layer, not just transport. | Multi-file work, peer review loops, cross-agent feedback, dynamic re-planning, multi-PR coordination, anything with a human-in-the-loop escape, swarms where workers pick up each other's output. | +| **Per-turn interactive spawn** (bounded interactive) | Interactive PTY agents (no `preset`), but each step is bounded to *one* turn that exits as soon as its file/message is produced. Agents still join the channel — they just don't live across turns. | Turn-taking flows where each turn is short, well-defined, and reads its inputs from a deterministic source (file on disk, prior grade step). Exercises the PTY + channel-injection path without the idle-detection-vs-wait-for-event race that bites long-running interactive agents. | | **Pipeline** (one-shot DAG) | Each step runs as a one-shot subprocess (`claude -p`, `codex exec`); steps hand off via `{{steps.X.output}}` text injection. No agents are alive at the same time; no chat happens. | Linear, well-specified transformations; deterministic data passing; no live agent-to-agent coordination during implementation. The selected review-depth path and deterministic final gates still apply. | **Default to Conversation for any non-trivial work.** Pipeline DAGs are simpler to reason about but they do not exercise the relay primitive — they are a Unix pipe with extra steps. If you would happily write the same task as a single shell pipeline, pipeline-shape is fine. Otherwise, you almost certainly want a Conversation shape. @@ -1723,6 +1726,22 @@ Edit files as assigned. Report completion. Fix issues from feedback.`, Once you're in the Interactive Team shape, the channel is your coordination medium. These are recipes for using it well — they are *prompt-authoring patterns*, not new SDK surface. All of them assume interactive agents (no `preset`) sharing a `.channel('wf-...')`. +> **Idle detection beats "wait for X" prompts.** The runtime treats a silent interactive PTY as complete after a short idle window (~30 seconds). A task prompt that tells an agent to *wait for message Y* / *exit only on `GAME_OVER`* / *stay until @reviewer posts a verdict* will lose that race if the awaited event is more than ~30s away — the runtime will mark the agent complete and tear it down before the message arrives. This affects every recipe below: **Q/A, Broadcast/Ack, Peer Review, and Hand-Off all encode a wait.** +> +> **Two ways to handle it:** +> +> 1. **Per-agent escape hatch — set `idleThresholdSecs` on the agent.** The builder option `idleThresholdSecs` overrides the per-agent idle window (default 30s; `0` disables idle detection entirely). Use this when a specific agent reliably has long quiet stretches by design: +> ```typescript +> .agent('long-running-reviewer', { cli: 'claude', idleThresholdSecs: 0 }) +> ``` +> Pick the lowest value that comfortably exceeds your worst-case wait — `0` is safest but means a genuinely-hung agent will sit forever. +> +> 2. **Restructure to Per-turn interactive spawn.** For multi-turn flows where some turns are slow (LLM reasoning, manual review, paused work), use the **Per-turn interactive spawn** shape from the [coordination table](#choose-your-coordination-style--conversation-vs-pipeline): one bounded step per turn, with a deterministic step in between carrying state on disk. You still get interactive PTY + channel; the session is just short enough to never hit the idle window. +> +> The chat-native recipes below work as written when the next expected event reliably arrives well inside the idle window, or when you've extended the window per (1). +> +> Symptom to watch for in logs: a player exits with `Completion inferred from clean process exit (code 0) — no coordination signal was required` *before* the event it was supposed to wait for arrived; or a lead step that drives waits via shell `sleep` calls. + #### 1. Question / Answer (blocking ask) When agent A needs information only agent B has, instruct A to **post a direct question and wait for a reply** rather than guessing or proceeding. @@ -2230,6 +2249,9 @@ When you set `.pattern('supervisor')` (or `hub-spoke`, `fan-out`), the runner au |---------|-----| | Treating relay as transport, not as a coordination layer (every step is `preset: 'worker'`, every handoff is `{{steps.X.output}}`) | Default to **Conversation shape** for non-trivial work — interactive agents on a shared channel. Pipeline-shape is only correct when the work could be expressed as a `bash \| bash \| bash` pipe. | | Interactive agents on a channel whose task strings don't tell them to talk to each other | Pick a [Chat-Native Coordination Recipe](#chat-native-coordination-recipes) (Q/A, Broadcast/Ack, Peer Review, Standup, Hand-Off) and bake it into the task prompt — otherwise you're paying for a chat substrate you're not using | +| Task prompt that tells an interactive agent to wait for a later channel event | Idle detection (~30s) will complete the PTY before the event arrives. Set `idleThresholdSecs` (0 disables) or restructure to [Per-turn interactive spawn](#choose-your-coordination-style--conversation-vs-pipeline) | +| Using an LLM as a wait/grade oracle (`sleep 30` loops, "guess whether the secret matches") | If the comparison is computable (string match, test pass/fail, regex, schema validity), grade it in a **deterministic step**. LLMs should *produce*, not adjudicate fixed rules | +| Adding a repair step downstream of a gate that has no failure mode an agent can fix | "Workflows repair before they fail" applies when the gate has a real, agent-fixable failure surface. A deterministic gate downstream of deterministic work is correct by construction — a repair step there just burns a spawn cycle on a guaranteed no-op. Skip the repair, keep the hard gate | | All workflows run sequentially | Group independent workflows into parallel waves (4-7x speedup) | | Every step depends on the previous one | Only add `dependsOn` when there's a real data dependency | | Self-review step with no timeout | Set `timeout: 300_000` (5 min) — Codex hangs in non-interactive review | From 8f0f45fc9509f7a4c0edc156b8ee1e2ceddb8974 Mon Sep 17 00:00:00 2001 From: khaliqgant Date: Sun, 20 Sep 2026 17:49:56 -0700 Subject: [PATCH 2/3] fix(skills): match PR-owner subscribe and idle-window claims to live 12.4 delivery The PR glob wakes events whose resource_ref is that PR even when the inventory file lives under /reviews or /comments. Per-turn interactive spawn only beats idle detection when each turn itself finishes inside the window. --- skills/orchestrating-agent-relay/SKILL.md | 21 ++++++++++++------- skills/writing-agent-relay-workflows/SKILL.md | 4 ++-- 2 files changed, 15 insertions(+), 10 deletions(-) diff --git a/skills/orchestrating-agent-relay/SKILL.md b/skills/orchestrating-agent-relay/SKILL.md index 491d40e6..bd440116 100644 --- a/skills/orchestrating-agent-relay/SKILL.md +++ b/skills/orchestrating-agent-relay/SKILL.md @@ -524,8 +524,9 @@ worker still thinking. Defenses: Whenever a worker **opens or owns a PR**, subscribe **that live PR-owner agent** to the PR's GitHub provider resource immediately — the moment the PR -number exists, not after the first review comment. Review comments, pushes, -check changes, and discussion then wake the owner instead of waiting on the +number exists, not after the first review comment. Events whose +`resource_ref` is that PR (review submitted/edited, review comments, PR +pushes, PR-tied discussion) then wake the owner instead of waiting on the next poll. ```bash @@ -547,12 +548,16 @@ already-live owner. `--events` defaults to Relay `message.created,thread.reply`. 3. A Relayfile binding keyed by `(provider, resolved path glob)` `--resource` is a Relayfile VFS glob. `owner/repo` resolves to repository -scope; provider URLs are not accepted. A PR glob does **not** include GitHub -issue-comment paths (`/issues//comments/**`) even when those comments sit -on the PR. Reviews, root review-comments, and checks are inventoried at -repository-level `reviews/`, `comments/`, and `checks/` paths — a PR directory -does not cover them. Subscribe the PR glob first; add those extra globs only -when the owner must wake on those exact records. +scope; provider URLs are not accepted. The canonical PR glob is the owner +binding. Matching uses the event's `resource_ref` (for example +`/github/repos/__/pulls/by-id/.json`), not only the inventory +file path — so a review stored at `/github/repos///reviews/.json` +still wakes a `/pulls//**` owner when it is that PR's review. Extra globs +are for records that **never** carry this PR's `resource_ref` (a repo-wide +check on another SHA, an issue-comment path that only names `/issues//`, +or an independently inventoried pending review you must stage before its +terminal event). Do not give those extra globs to a helper; they are still +writable owner routes. One `(github, glob)` binding per resource. List bindings with `agent-relay integration subscribe --list`. This is distinct from `agent-relay integration subscription list`, which lists Relay diff --git a/skills/writing-agent-relay-workflows/SKILL.md b/skills/writing-agent-relay-workflows/SKILL.md index 7e27f40c..77dbfece 100644 --- a/skills/writing-agent-relay-workflows/SKILL.md +++ b/skills/writing-agent-relay-workflows/SKILL.md @@ -268,7 +268,7 @@ Before writing the workflow, decide *how the agents will coordinate*. The relay | Shape | What it is | Use when | |---|---|---| | **Conversation** (chat-native) | Interactive agents share a channel; messages, `@-mentions`, and ambient awareness drive coordination. Lead and workers spawn in parallel and self-organize. The relay is the coordination layer, not just transport. | Multi-file work, peer review loops, cross-agent feedback, dynamic re-planning, multi-PR coordination, anything with a human-in-the-loop escape, swarms where workers pick up each other's output. | -| **Per-turn interactive spawn** (bounded interactive) | Interactive PTY agents (no `preset`), but each step is bounded to *one* turn that exits as soon as its file/message is produced. Agents still join the channel — they just don't live across turns. | Turn-taking flows where each turn is short, well-defined, and reads its inputs from a deterministic source (file on disk, prior grade step). Exercises the PTY + channel-injection path without the idle-detection-vs-wait-for-event race that bites long-running interactive agents. | +| **Per-turn interactive spawn** (bounded interactive) | Interactive PTY agents (no `preset`), but each step is bounded to *one* turn that exits as soon as its file/message is produced. Agents still join the channel — they just don't live across turns. | Turn-taking flows where each turn is short, well-defined, finishes **inside the idle window**, and reads its inputs from a deterministic source (file on disk, prior grade step). Cuts the long-running wait-for-event race; a silent turn that itself exceeds ~30s still needs `idleThresholdSecs`. | | **Pipeline** (one-shot DAG) | Each step runs as a one-shot subprocess (`claude -p`, `codex exec`); steps hand off via `{{steps.X.output}}` text injection. No agents are alive at the same time; no chat happens. | Linear, well-specified transformations; deterministic data passing; no live agent-to-agent coordination during implementation. The selected review-depth path and deterministic final gates still apply. | **Default to Conversation for any non-trivial work.** Pipeline DAGs are simpler to reason about but they do not exercise the relay primitive — they are a Unix pipe with extra steps. If you would happily write the same task as a single shell pipeline, pipeline-shape is fine. Otherwise, you almost certainly want a Conversation shape. @@ -1736,7 +1736,7 @@ Once you're in the Interactive Team shape, the channel is your coordination medi > ``` > Pick the lowest value that comfortably exceeds your worst-case wait — `0` is safest but means a genuinely-hung agent will sit forever. > -> 2. **Restructure to Per-turn interactive spawn.** For multi-turn flows where some turns are slow (LLM reasoning, manual review, paused work), use the **Per-turn interactive spawn** shape from the [coordination table](#choose-your-coordination-style--conversation-vs-pipeline): one bounded step per turn, with a deterministic step in between carrying state on disk. You still get interactive PTY + channel; the session is just short enough to never hit the idle window. +> 2. **Restructure to Per-turn interactive spawn.** For multi-turn flows, use the **Per-turn interactive spawn** shape from the [coordination table](#choose-your-coordination-style--conversation-vs-pipeline): one bounded step per turn, with a deterministic step in between carrying state on disk. You still get interactive PTY + channel. This only beats idle detection when **each turn finishes well inside the window** — a slow reasoning turn still needs `idleThresholdSecs` (option 1). Do not treat per-turn spawn as a substitute for raising the threshold on a turn that is itself quiet for >30s. > > The chat-native recipes below work as written when the next expected event reliably arrives well inside the idle window, or when you've extended the window per (1). > From 19676901776cd775d50389c770db085c4d4e8660 Mon Sep 17 00:00:00 2001 From: khaliqgant Date: Sun, 20 Sep 2026 17:56:48 -0700 Subject: [PATCH 3/3] fix(orchestrating-agent-relay): default PR-owner recipe covers reviews, checks, comments Codex P1: the single /pulls//** command did not match the promised wake set. Subscribe the owner to the adapter nested reviews/status globs plus GitHub issue-comment paths as well. --- skills/orchestrating-agent-relay/SKILL.md | 84 ++++++++++++++--------- 1 file changed, 52 insertions(+), 32 deletions(-) diff --git a/skills/orchestrating-agent-relay/SKILL.md b/skills/orchestrating-agent-relay/SKILL.md index bd440116..b53a2e5f 100644 --- a/skills/orchestrating-agent-relay/SKILL.md +++ b/skills/orchestrating-agent-relay/SKILL.md @@ -67,7 +67,7 @@ reference is the **`using-agent-relay`** skill. | Give a human a follow-along link | `agent-relay observer` | | Inspect a worker's TTY | `agent-relay node agent attach Worker1 --mode view` | | Fleet-wide agent inventory | `agent-relay fleet agent list` | -| Subscribe PR-owner to GitHub PR | `agent-relay integration subscribe github --resource '/github/repos///pulls//**' --to @Owner` | +| Subscribe PR-owner to GitHub PR | `agent-relay integration subscribe github --resource '/github/repos///pulls//**' --to @Owner` (also `pulls//reviews/**`, `pulls//status/**`, `issues//comments/**`) | | List integration bindings | `agent-relay integration subscribe --list` | | Unsubscribe a provider resource | `agent-relay integration unsubscribe github --resource '...'` | | Failed deliveries | `agent-relay node deadletters` | @@ -443,9 +443,9 @@ Quick Reference. Then enforce this protocol: (cross-node) for worker liveness; set a wall-clock fallback so a silently-dead worker can't hang the loop - The moment a worker opens or owns a GitHub PR, subscribe **that live - PR-owner identity** with `agent-relay integration subscribe github - --resource '/github/repos///pulls//**' --to @`. - Do not subscribe helpers to the same glob. After respawn under a new + PR-owner identity** to the PR glob **and** `pulls//reviews/**`, + `pulls//status/**`, and `issues//comments/**` (same `--to @`). + Do not subscribe helpers to those globs. After respawn under a new name, rebind. Verify a real GitHub event arrives in `check_inbox`, not just that `--list` shows a binding. Unsubscribe owned resources before release @@ -523,16 +523,29 @@ worker still thinking. Defenses: ## Subscribe the live PR owner Whenever a worker **opens or owns a PR**, subscribe **that live PR-owner -agent** to the PR's GitHub provider resource immediately — the moment the PR -number exists, not after the first review comment. Events whose -`resource_ref` is that PR (review submitted/edited, review comments, PR -pushes, PR-tied discussion) then wake the owner instead of waiting on the -next poll. +agent** to the PR's GitHub provider resources immediately — the moment the +PR number exists, not after the first review comment. Conversation comments, +reviews, review comments, pushes, and check changes then wake the owner +instead of waiting on the next poll. + +The default recipe is four owner bindings. Each `--resource` is a distinct +binding key, so these do not replace each other. Paths match the GitHub +adapter layout used by `writing-agent-relay-workflows` (`pulls//reviews/**`, +`pulls//status/**`) plus GitHub's issue-comment path for PR conversation +comments: ```bash -agent-relay integration subscribe github \ - --resource '/github/repos///pulls//**' \ - --to @ +OWNER=@ +REPO='/github/repos//' +N= +for resource in \ + "$REPO/pulls/$N/**" \ + "$REPO/pulls/$N/reviews/**" \ + "$REPO/pulls/$N/status/**" \ + "$REPO/issues/$N/comments/**" +do + agent-relay integration subscribe github --resource "$resource" --to "$OWNER" +done ``` `--to` is `@agent` or `#channel`. Point it at the owner handle. `--spawn ` @@ -548,16 +561,16 @@ already-live owner. `--events` defaults to Relay `message.created,thread.reply`. 3. A Relayfile binding keyed by `(provider, resolved path glob)` `--resource` is a Relayfile VFS glob. `owner/repo` resolves to repository -scope; provider URLs are not accepted. The canonical PR glob is the owner -binding. Matching uses the event's `resource_ref` (for example -`/github/repos/__/pulls/by-id/.json`), not only the inventory -file path — so a review stored at `/github/repos///reviews/.json` -still wakes a `/pulls//**` owner when it is that PR's review. Extra globs -are for records that **never** carry this PR's `resource_ref` (a repo-wide -check on another SHA, an issue-comment path that only names `/issues//`, -or an independently inventoried pending review you must stage before its -terminal event). Do not give those extra globs to a helper; they are still -writable owner routes. One `(github, glob)` binding per resource. +scope; provider URLs are not accepted. Matching also uses the event's +`resource_ref` (for example `/github/repos/__/pulls/by-id/.json`), +not only the inventory file path — so a review whose file is +`/github/repos///reviews/.json` can still wake a `/pulls//**` +binding when `resource_ref` names that PR. Do **not** rely on that alone for +conversation comments or checks: GitHub conversation comments on a PR are +inventoried under `/issues//comments/**`, and status/check records under +`/pulls//status/**`. Subscribe those globs in the default recipe above. +Do not give any of them to a helper; they are still writable owner routes. +One `(github, glob)` binding per resource. List bindings with `agent-relay integration subscribe --list`. This is distinct from `agent-relay integration subscription list`, which lists Relay @@ -585,10 +598,11 @@ Bindings follow the **recipient identity**, not the OS process. ### Helper agents: no second writable subscribe -Only the PR owner gets `--to @Owner` on that PR glob. A second `subscribe` to -the same `(github, path glob)` **replaces** the owner's binding. Reviewers, -shadows, and one-shot helpers read the PR through git/`gh` or channel traffic; -they do not get their own writable subscription to the owner's resource. +Only the PR owner gets `--to @Owner` on those owner globs. A second +`subscribe` to the same `(github, path glob)` **replaces** that route. +Reviewers, shadows, and one-shot helpers read the PR through git/`gh` or +channel traffic; they do not get their own writable subscription to the +owner's resources. ### Unsubscribe / release hygiene @@ -599,9 +613,15 @@ IDs are gone: agent-relay integration subscribe --list agent-relay integration webhook list agent-relay integration webhook list-inbound -agent-relay integration unsubscribe github \ - --resource '/github/repos///pulls//**' -# confirm the binding, webhookId, and webhookSubscriptionId disappeared +for resource in \ + "/github/repos///pulls//**" \ + "/github/repos///pulls//reviews/**" \ + "/github/repos///pulls//status/**" \ + "/github/repos///issues//comments/**" +do + agent-relay integration unsubscribe github --resource "$resource" +done +# confirm each binding, webhookId, and webhookSubscriptionId disappeared agent-relay fleet release ``` @@ -623,8 +643,8 @@ GitHub events will wake the agent. Verify in this order: live on a node. Roster-only is not enough. 4. **Actual event** — after a real GitHub comment, review, check, or push, the owner receives a Relay message from `github` whose body names the event and - a Relayfile path under that glob. Read it with `check_inbox` / - `list_messages`. Optional broker corroboration: a delivery-injected event + a Relayfile path (or `resource_ref`) for this PR. Read it with `check_inbox` + / `list_messages`. Optional broker corroboration: a delivery-injected event for that agent; `node deadletters` must not hold it. If steps 1–3 pass and step 4 does not, the subscription is not working. Do not @@ -871,7 +891,7 @@ this whole workaround once `relay#1446` lands rather than letting it outlive the | Worker self-removed; can't send review fixes | Instruct workers not to self-remove until told. If already gone, spawn a fresh worker and re-inject branch + commit SHA + full verdict; rebind any GitHub PR subscription to the new live identity (see [Subscribe the live PR owner](#subscribe-the-live-pr-owner)) | | Told the user to open an observer URL built from the workspace key | That is an admin credential in a query string, and the realtime endpoint rejects it. Run `agent-relay observer` (or `get_observer_url`) and share the `ot_live_` URL it returns | | Worker died silently; loop hangs | Inbox polling fires on messages only. Poll `agent-relay node agent list` / `fleet agent list` for liveness and set a wall-clock fallback (~30 min ScheduleWakeup) | -| Opened a PR but did not subscribe the live owner | Review comments, pushes, and checks will not wake anyone. Immediately `integration subscribe github --resource '/github/repos///pulls//**' --to @` | +| Opened a PR but did not subscribe the live owner | Review comments, pushes, and checks will not wake anyone. Immediately subscribe `@Owner` to `/pulls//**`, `/pulls//reviews/**`, `/pulls//status/**`, and `/issues//comments/**` | | Subscribed a helper/reviewer to the same PR glob | Binding key is `(provider, resolved path)` — the second subscribe **replaces** the owner's writable route. Only the PR-owner identity gets `--to @Owner` | | Treated `--list` / webhook create as delivery proof | Binding creation is not a wake. Confirm a real GitHub event arrives as a Relay message from `github` with a Relayfile path under that glob (`check_inbox` / `list_messages`); check `node deadletters` if it does not | | Respawned the owner under a new name and left the old binding | New identities do not inherit subscriptions. Re-subscribe `--to @NewOwner` on the same `--resource`; confirm the prior webhook/subscription IDs retired |