diff --git a/README.md b/README.md index 82abf45d..b9a79cf4 100644 --- a/README.md +++ b/README.md @@ -2,19 +2,19 @@ Skills, slash commands, and a Claude Code plugin for building multi-agent systems with Agent Relay. -Package metadata lives in [prpm.json](prpm.json). The repo currently publishes `agent-workforce-skills` version `1.1.6`. +Package metadata lives in [prpm.json](prpm.json). The repo currently publishes `agent-workforce-skills` version `1.7.0`. ## Published Skills | Skill | Version | Description | |-------|---------|-------------| | [choosing-swarm-patterns](skills/choosing-swarm-patterns/SKILL.md) | 1.1.5 | Pick the right Agent Relay orchestration pattern across the 10 core swarm patterns plus specialized patterns. | -| [writing-agent-relay-workflows](skills/writing-agent-relay-workflows/SKILL.md) | 1.6.24 | Build multi-agent workflows with WorkflowBuilder, DAG dependencies, Relayfile-backed Slack human assistance, integration subscriptions, waitFor gates, review-depth review/fix loops, channels, and chat-native coordination recipes. | +| [writing-agent-relay-workflows](skills/writing-agent-relay-workflows/SKILL.md) | 2.0.0 | **DEPRECATED** - the superseded v1 `@relayflows/core` WorkflowBuilder engine. Use [`writing-relayflows`](skills/writing-relayflows/SKILL.md) for v2; keep this only to maintain an existing v1 workflow. | | [setting-up-relayfile](skills/setting-up-relayfile/SKILL.md) | 1.1.1 | Set up Relayfile mounts and writeback for provider files through local filesystem access. | -| [writing-relayflows](skills/writing-relayflows/SKILL.md) | 1.0.0 | Author a Relayflows v2 flow (`@relayflows/surface`/`@relayflows/sdk`, the `flows` CLI) in TypeScript or YAML/JSON — the run/llm/agent ladder, verification gates, cli/model resolution, flows.json, and real `flows check`/`run` refusal shapes. Not the older `@relayflows/core` WorkflowBuilder engine. | +| [writing-relayflows](skills/writing-relayflows/SKILL.md) | 1.4.0 | Author a Relayflows v2 flow (`@relayflows/surface`/`@relayflows/sdk`, the `flows` CLI) in TypeScript or YAML/JSON — the run/llm/agent ladder, verification gates, cli/model resolution, flows.json, and real `flows check`/`run` refusal shapes. Not the older `@relayflows/core` WorkflowBuilder engine. | | [using-agent-relay](skills/using-agent-relay/SKILL.md) | 1.4.0 | Participant-side MCP reference for a **registered** relay agent (spawned worker / registered lead): messaging, channels, threads, reactions, search, inbox, actions, and worker spawn/release. Counterpart to `orchestrating-agent-relay`. | -| [orchestrating-agent-relay](skills/orchestrating-agent-relay/SKILL.md) | 2.3.1 | The canonical way to run agent-relay: self-bootstrap the broker (`agent-relay node up`) and autonomously spawn, monitor, and coordinate a worker team over the relay MCP without human intervention. | -| [relay-80-100-workflow](skills/relay-80-100-workflow/SKILL.md) | 1.0.9 | Author workflows that close the 80-to-100 validation gap with repair-aware test, verify, review-depth review/fix with test hardening, and commit gates. | +| [orchestrating-agent-relay](skills/orchestrating-agent-relay/SKILL.md) | 2.4.0 | The canonical way to run agent-relay: self-bootstrap the broker (`agent-relay node up`) and autonomously spawn, monitor, and coordinate a worker team over the relay MCP without human intervention, including GitHub PR-owner integration subscriptions. | +| [relay-80-100-workflow](skills/relay-80-100-workflow/SKILL.md) | 2.1.0 | Close the 80-to-100 validation gap in a Relayflows v2 flow: the evidence recorder that turns a red check into work for a repair agent, repairable gates on the critical path, edit and hazard gates, when *not* to add a repair step, and the fresh-eyes review rounds that catch what green gates miss. | | [review-fix-signoff-loop](skills/review-fix-signoff-loop/SKILL.md) | 1.0.2 | Loop review, repair, validation, and fresh-context dual-agent signoff until independent reviewers both satisfy the verdict contract. | | [trigger-autocomplete-catalog](skills/trigger-autocomplete-catalog/SKILL.md) | 1.0.0 | Enforce webhook/event trigger autocomplete coverage through KNOWN_TRIGGER_CATALOG in @relayfile/adapter-core. | | [activity-summary](skills/activity-summary/SKILL.md) | 1.0.0 | Answer "what did I work on yesterday" questions by reading `digests/yesterday.md` first instead of crawling provider directories. | @@ -67,10 +67,11 @@ npx prpm install collections/agent-relay-starter --as codex,claude This collection includes: - `@agent-relay/choosing-swarm-patterns` -- `@agent-relay/writing-agent-relay-workflows` +- `@agent-relay/writing-relayflows` - `@agent-workforce/trail-snippet` - optional `@agent-relay/relay-80-100-workflow` - optional `@agent-relay/review-fix-signoff-loop` +- optional `@agent-relay/writing-agent-relay-workflows` (deprecated v1 engine) Install the `relayfile-workspace` collection when you want the full Relayfile workspace primitive stack: diff --git a/prpm.json b/prpm.json index 55cf53e5..46566a3b 100644 --- a/prpm.json +++ b/prpm.json @@ -1,6 +1,6 @@ { "name": "agent-workforce-skills", - "version": "1.1.6", + "version": "1.7.0", "description": "Skills for multi-agent coordination - swarm patterns, workflow building, relay usage, and orchestrating agent-relay", "author": "khaliqgant", "organization": "agent-relay", @@ -10,7 +10,7 @@ "packages": [ { "name": "migrating-persona-to-relayflow", - "version": "1.0.0", + "version": "1.1.0", "description": "Use when migrating an existing defineAgent persona (in AgentWorkforce/agents or AgentWorkforce/internal-agents) to run its LLM-heavy work through a Relayflow v1 workflow via ctx.workflow.run(). Covers scope decisions, the 4-step durable-workflow shape, colocation with the persona, ctx.workflow.run() invocation, red-first + resume tests, useSubscription semantics for realtime triggers, and the agentworkforce >= 4.1.44 CI-deploy bump. Reference migration: hn-monitor PR #126.", "format": "claude", "subtype": "skill", @@ -47,8 +47,8 @@ }, { "name": "writing-agent-relay-workflows", - "version": "1.6.24", - "description": "Use when building multi-agent workflows with relay broker-sdk. Covers conversation vs pipeline coordination, WorkflowBuilder/DAG steps, agents, {{steps.X.output}} chaining, Relayfile-backed human assistance, Slack answer injection, integration subscriptions, waitFor gates, repairable verification, review-depth fresh-eyes review/fix loops with test hardening, channels, chat-native recipes, error handling, event listeners, step sizing, lead+workers teams, and parallel waves.", + "version": "2.0.0", + "description": "DEPRECATED - the superseded v1 @relayflows/core WorkflowBuilder engine (failOnError, captureOutput, {{steps.X.output}}, .onError()). All new work uses writing-relayflows (v2: @relayflows/surface / @relayflows/sdk, CLI `flows`). Keep this only to read or maintain an existing v1 workflow: conversation vs pipeline coordination, DAG steps, idle detection, human assistance, waitFor gates, review-depth review/fix loops, channels, and parallel waves.", "format": "claude", "subtype": "skill", "languages": [ @@ -95,8 +95,8 @@ }, { "name": "writing-relayflows", - "version": "1.0.0", - "description": "Use when authoring a Relayflows flow (@relayflows/surface / @relayflows/sdk, the v2 journal-based engine, CLI `flows`) in TypeScript or YAML/JSON. Covers the run/llm/agent ladder, human/dispatch/done, verification gates, cli/model resolution, flows.json, and flows check/run/resume refusal shapes. Not for the older @relayflows/core WorkflowBuilder (chained .pattern('dag')/.agent()/.step() calls) \u2014 see writing-agent-relay-workflows / migrating-persona-to-relayflow instead.", + "version": "1.4.0", + "description": "Use when authoring a Relayflows flow (@relayflows/surface / @relayflows/sdk, the v2 journal-based engine, CLI `flows`) in TypeScript or YAML/JSON. Covers the run/llm/agent ladder, human/dispatch/done, verification gates, cli/model resolution, flows.json, and flows check/run/resume refusal shapes. Not for the older @relayflows/core WorkflowBuilder (chained .pattern('dag')/.agent()/.step() calls) — see writing-agent-relay-workflows / migrating-persona-to-relayflow instead.", "format": "claude", "subtype": "skill", "languages": [ @@ -136,8 +136,8 @@ }, { "name": "orchestrating-agent-relay", - "version": "2.3.1", - "description": "The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, and team coordination.", + "version": "2.4.0", + "description": "The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, GitHub PR-owner integration subscriptions, and team coordination.", "format": "claude", "subtype": "skill", "tags": [ @@ -154,11 +154,12 @@ }, { "name": "relay-80-100-workflow", - "version": "1.0.9", - "description": "Use when writing agent-relay workflows that must fully validate features end-to-end before merging - covers the 80-to-100 pattern with review-depth fresh-eyes review/fix loops with test hardening, PGlite in-memory Postgres testing, mock sandbox patterns, test-fix-rerun loops, verify gates, and full lifecycle from implementation through passing tests to commit", + "version": "2.1.0", + "description": "Use when writing a Relayflows v2 flow (@relayflows/surface / @relayflows/sdk, CLI `flows`) that must fully validate a feature end-to-end before it is sealed or merged - the 80-to-100 gap. Covers the evidence-recorder pattern that replaces v1's failOnError/captureOutput, repairable gates on the critical path, edit and hazard gates, gate cross-checking, fresh-eyes review rounds, and what a wall of green gates still misses.", "format": "claude", "subtype": "skill", "tags": [ + "relayflows", "workflow", "testing", "e2e-validation", @@ -478,7 +479,7 @@ "id": "agent-relay-starter", "name": "Agent Relay Starter", "description": "Essential skills for building multi-agent systems with Agent Relay - swarm pattern selection, workflow authoring, and trail debugging", - "version": "1.1.0", + "version": "1.2.0", "category": "development", "tags": [ "multi-agent", @@ -495,10 +496,16 @@ "reason": "Decision framework for picking the right multi-agent orchestration pattern" }, { - "packageId": "@agent-relay/writing-agent-relay-workflows", + "packageId": "@agent-relay/writing-relayflows", "version": "latest", "required": true, - "reason": "WorkflowBuilder API guide for authoring DAG-based multi-agent workflows with review-depth review/fix loops and test hardening" + "reason": "Authoring guide for the current Relayflows v2 engine (@relayflows/surface / @relayflows/sdk, CLI `flows`)" + }, + { + "packageId": "@agent-relay/writing-agent-relay-workflows", + "version": "latest", + "required": false, + "reason": "DEPRECATED v1 WorkflowBuilder reference - install only to maintain an existing v1 workflow" }, { "packageId": "@agent-workforce/trail-snippet", diff --git a/skills/migrating-persona-to-relayflow/SKILL.md b/skills/migrating-persona-to-relayflow/SKILL.md index 7a78e740..74240f29 100644 --- a/skills/migrating-persona-to-relayflow/SKILL.md +++ b/skills/migrating-persona-to-relayflow/SKILL.md @@ -5,6 +5,12 @@ description: Use when migrating an existing `defineAgent` persona (in AgentWorkf # Migrating a persona to Relayflow v1 +> **v1 engine.** This skill targets the superseded `@relayflows/core` engine via +> `ctx.workflow.run()`. The current engine is **v2** — `@relayflows/surface` / +> `@relayflows/sdk`, CLI `flows` — documented in +> [`writing-relayflows`](../writing-relayflows/SKILL.md). Use this skill only to move an +> existing persona onto v1's durable execution today. + ## When to use this skill You are working on a persona in `AgentWorkforce/agents` or `AgentWorkforce/internal-agents` that today does its work inline in a `defineAgent({ handler })` — LLM calls via `ctx.llm(...)`, ranking, curation, review — and you want to move the LLM-heavy portion behind a durable Relayflow v1 workflow via `ctx.workflow.run()` so that the work is exactly-once resumable, journaled, and step-level retriable. @@ -16,7 +22,7 @@ Read the RFC first: [`docs/RFC-0001-everything-is-a-relayflow.md`](https://githu ## Do not use this skill for - **Authoring a v1 workflow from scratch** — use [`writing-agent-relay-workflows`](../writing-agent-relay-workflows/SKILL.md). -- **Cloud v2 execution** — v2 is still under construction (see the RFC gate 3–4 work). Migrate to v1 first; v2 becomes an opt-in `relayflowVersion: "v2"` flip on the same `ctx.workflow.run()` call once v2 is generally available. +- **Authoring for the v2 engine** — v2 (`@relayflows/surface`/`@relayflows/sdk`, CLI `flows`, packages `2.0.x`) is a different product that ships today; see [`writing-relayflows`](../writing-relayflows/SKILL.md). This skill and `writing-agent-relay-workflows` cover the superseded **v1** engine. Personas still migrate to v1 first: v2 becomes an opt-in `relayflowVersion: "v2"` flip on the same `ctx.workflow.run()` call once the persona runtime supports it. - **Repointing a workflow between packages** (`@agent-relay/sdk/workflows` → `@relayflows/core`) — that is covered in `writing-agent-relay-workflows`. ## The seven-step recipe diff --git a/skills/orchestrating-agent-relay/SKILL.md b/skills/orchestrating-agent-relay/SKILL.md index 23b8d937..51f76eab 100644 --- a/skills/orchestrating-agent-relay/SKILL.md +++ b/skills/orchestrating-agent-relay/SKILL.md @@ -1,6 +1,6 @@ --- name: orchestrating-agent-relay -description: The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, and team coordination. +description: The canonical way to run agent-relay - self-bootstrap the local broker and autonomously spawn, monitor, and coordinate a team of worker agents without human intervention. Covers infrastructure startup, agent spawning, lifecycle monitoring, message-based reading via the relay MCP, GitHub PR-owner integration subscriptions, and team coordination. --- # Orchestrating Agent Relay @@ -66,7 +66,12 @@ reference is the **`using-agent-relay`** skill. | Read worker replies (MCP) | `check_inbox(limit: 20)` / `list_messages(channel: "general")` | | Give a human a follow-along link | `agent-relay observer` | | Inspect a worker's TTY | `agent-relay node agent attach Worker1 --mode view` | -| Release worker | `agent-relay node agent release Worker1` | +| Fleet-wide agent inventory | `agent-relay fleet agent list` | +| Subscribe PR-owner to GitHub PR | `agent-relay integration subscribe github --resource '/github/repos///pulls//**' --to @Owner` (also `pulls//reviews/**`, `pulls//status/**`, `issues//comments/**`) | +| List integration bindings | `agent-relay integration subscribe --list` | +| Unsubscribe a provider resource | `agent-relay integration unsubscribe github --resource '...'` | +| Failed deliveries | `agent-relay node deadletters` | +| Release worker | `agent-relay node agent release Worker1` / `agent-relay fleet release Worker1` | | Stop broker | `agent-relay node down` | ## Bootstrap Flow @@ -146,9 +151,12 @@ add_agent( ``` `node agent spawn` takes the provider as a positional argument -(`claude`, `codex`, `gemini`, `droid`, …) and `--name` / `--task` / `--channels` -/ `--model` / `--cwd` flags. By default the agent joins the `general` channel and -runs in `interactive` spawn mode; pass `--exit-after-task` for a one-shot worker. +(`claude`, `codex`, `gemini`, `grok`, `opencode`, `muse`, `devin`, `droid`, …) +and `--name` / `--task` / `--channels` / `--model` / `--cwd` flags. By default +the agent joins the `general` channel and runs in `interactive` spawn mode; pass +`--exit-after-task` for a one-shot worker. Fleet placement uses +`agent-relay fleet spawn ` (or MCP `spawn` / `add_agent`); `droid` remains +high-risk for delegation. > **Expect a 30–60s gap between spawn and the first ACK.** A worker shows in > `node agent list` within ~5s (the process is up), but the underlying CLI @@ -226,13 +234,19 @@ agent-relay node metrics ### Step 4: Release Workers +Unsubscribe any provider resources that worker owned (see +[Subscribe the live PR owner](#subscribe-the-live-pr-owner)) before release. + ```text remove_agent(name: "Worker1", reason: "Work accepted") ``` -CLI equivalent: +CLI equivalent — prefer the control-plane form when the worker may not be on +this broker's default `connection.json`: ```bash +agent-relay fleet release Worker1 +# local-only fallback: agent-relay node agent release Worker1 ``` @@ -425,11 +439,19 @@ Quick Reference. Then enforce this protocol: over the relay MCP — never `node tail` (that streams broker debug events, not worker messages). See the "Channel vs DM" section for the full reading model -- Poll `agent-relay node agent list` for worker liveness; set a wall-clock - fallback so a silently-dead worker can't hang the loop +- Poll `agent-relay node agent list` (local) or `agent-relay fleet agent list` + (cross-node) for worker liveness; set a wall-clock fallback so a + silently-dead worker can't hang the loop +- The moment a worker opens or owns a GitHub PR, subscribe **that live + PR-owner identity** to the PR glob **and** `pulls//reviews/**`, + `pulls//status/**`, and `issues//comments/**` (same `--to @`). + Do not subscribe helpers to those globs. After respawn under a new + name, rebind. Verify a real GitHub event arrives in `check_inbox`, not + just that `--list` shows a binding. Unsubscribe owned resources before + release - If a human is watching, give them a follow-along link with - `agent-relay observer` and print the URL it returns. Never print the - workspace key or put it in a URL + `agent-relay observer` (or MCP `get_observer_url`) and print the URL it + returns. Never print the workspace key or put it in a URL ``` ## Multi-Round Review Loops (DONE → NO-GO → fix → re-review) @@ -476,7 +498,8 @@ Do NOT self-remove — stay alive for re-review." Always pass branch + commit SHA + the **complete** reviewer verdict. A fresh worker has none of the loop's history; a summarized verdict loses the -specifics it needs to fix. +specifics it needs to fix. Rebind any GitHub PR subscription to the new +live identity (see below) — a new name does not inherit the old route. ### Detecting a silently-dead worker @@ -492,6 +515,152 @@ worker still thinking. Defenses: - Always set a wall-clock fallback (e.g. a ScheduleWakeup ~30 min out) so a silently-dead worker can't hang the loop forever waiting on a message that will never arrive. +- Across nodes, `agent-relay fleet agent list` joins live broker agents + against the workspace roster. `idle` means the harness is at its prompt, + not that the assigned work is complete. Roster-only presence is not a live + worker. + +## Subscribe the live PR owner + +Whenever a worker **opens or owns a PR**, subscribe **that live PR-owner +agent** to the PR's GitHub provider resources immediately — the moment the +PR number exists, not after the first review comment. Conversation comments, +reviews, review comments, pushes, and check changes can then wake the owner. +Binding creation alone does not establish end-to-end delivery; keep a bounded +GitHub polling fallback until the event types and harness wake are verified. + +The default recipe is four owner bindings. Each `--resource` is a distinct +binding key, so these do not replace each other. Paths match the GitHub +adapter layout used by `writing-agent-relay-workflows` (`pulls//reviews/**`, +`pulls//status/**`) plus GitHub's issue-comment path for PR conversation +comments. Replace every angle-bracket placeholder in these shell examples +with the actual owner handle, repository, and PR number before execution: + +```bash +OWNER='@' +REPO='/github/repos//' +N='' +for resource in \ + "$REPO/pulls/$N/**" \ + "$REPO/pulls/$N/reviews/**" \ + "$REPO/pulls/$N/status/**" \ + "$REPO/issues/$N/comments/**" +do + agent-relay integration subscribe github --resource "$resource" --to "$OWNER" +done +``` + +`--to` is `@agent` or `#channel`. Point it at the owner handle. `--spawn ` +can launch and confirm a recipient before binding; do not use it to replace an +already-live owner. `--events` defaults to Relay `message.created,thread.reply`. + +`subscribe` is not a chat listener. It provisions three artifacts: + +1. A Relay inbound webhook (`webhookId`) delivering into the recipient's + identity-bound channel +2. A Relayfile webhook subscription (`webhookSubscriptionId`) filtered by + `path_glob` +3. A Relayfile binding keyed by `(provider, resolved path glob)` + +`--resource` is a Relayfile VFS glob. `owner/repo` resolves to repository +scope; provider URLs are not accepted. Matching also uses the event's +`resource_ref` (for example `/github/repos/__/pulls/by-id/.json`), +not only the inventory file path — so a review whose file is +`/github/repos///reviews/.json` can still wake a `/pulls//**` +binding when `resource_ref` names that PR. Do **not** rely on that alone for +conversation comments or checks: GitHub conversation comments on a PR are +inventoried under `/issues//comments/**`, and status/check records under +`/pulls//status/**`. Subscribe those globs in the default recipe above. +Do not give any of them to a helper; they are still writable owner routes. +One `(github, glob)` binding per resource. + +List bindings with `agent-relay integration subscribe --list`. This is +distinct from `agent-relay integration subscription list`, which lists Relay +workspace event subscriptions (`message.created`, …), and from +`agent-relay integration webhook create `, which mints an inbound +webhook URL that posts into a channel (it does **not** take a URL or `--event`; +that older shape is stale). + +### Ownership transfer / rebinding after respawn + +Bindings follow the **recipient identity**, not the OS process. + +- **Same name, identity kept** (`fleet release` / `node agent release` without + `--delete-agent`): the binding still points at `@Owner`. Respawn that name, + confirm it is live, and re-subscribe only if `--list` no longer shows + `@Owner` on that resource. +- **New name or deleted identity** (`--delete-agent`, or a fresh + `Implementer2`): the new agent does **not** inherit the old subscription. + Subscribe the new live identity to the same `--resource` immediately. The + binding key is `(provider, resolved path)` — a re-subscribe **replaces** the + previous route (create-first). Record the prior `webhookId` / + `webhookSubscriptionId` so you can confirm they retired. +- Do not repair a failed spawn by inviting a helper onto the owner's channel + or by pointing `--to` at a disposable worker. + +### Helper agents: no second writable subscribe + +Only the PR owner gets `--to @Owner` on those owner globs. A second +`subscribe` to the same `(github, path glob)` **replaces** that route. +Reviewers, shadows, and one-shot helpers read the PR through git/`gh` or +channel traffic; they do not get their own writable subscription to the +owner's resources. + +### Unsubscribe / release hygiene + +Unsubscribe **owned** resources before releasing the owner, then verify the +IDs are gone: + +```bash +agent-relay integration subscribe --list +agent-relay integration webhook list +agent-relay integration webhook list-inbound +for resource in \ + "/github/repos///pulls//**" \ + "/github/repos///pulls//reviews/**" \ + "/github/repos///pulls//status/**" \ + "/github/repos///issues//comments/**" +do + agent-relay integration unsubscribe github --resource "$resource" +done +# confirm each binding, webhookId, and webhookSubscriptionId disappeared +agent-relay fleet release '' +``` + +Unsubscribe does not delete agents or channels. Do not unsubscribe by guessed +name, and do not delete a webhook another binding still references. A failed +re-subscribe must leave the prior working binding in place — do not tear it +down first. + +### Verify delivery, not just binding creation + +A successful `subscribe` (or a 2xx webhook create) is **not** proof that +GitHub events will wake the agent. Verify in this order: + +1. **Binding** — `agent-relay integration subscribe --list` shows + `(github, resolved PR glob) → @Owner`. +2. **Webhook inventory** — `integration webhook list` / `list-inbound` shows + the new `webhookId` (and the prior id retired after a replacement). +3. **Live recipient** — `fleet agent list` or `node agent list` shows `@Owner` + live on a node. Roster-only is not enough. +4. **Actual event** — after a real GitHub comment, review, check, or push, the + owner receives a Relay message from `github` whose body names the event and + a Relayfile path (or `resource_ref`) for this PR. Read it with `check_inbox` + / `list_messages`. Capture the GitHub event ID, Relay message ID, and + timestamps; test discussion, review, push, and check events separately. +5. **Idle harness wake** — with the owner idle before the event, confirm a new + agent turn consumes that message and rereads the authoritative PR/check + state. A channel message, inbox read, or broker delivery-injected event + alone does not prove an idle harness resumed. Inspect `node deadletters` + for failed delivery. + +If steps 1–3 pass but step 4 does not, delivery remains unverified. If step 4 +passes but step 5 does not, the remaining gap is harness wake/consumption. +In particular, a delivered comment does not establish `check_run.completed` +delivery or idle wake across other harnesses. Report those as separate +limitations and keep the polling fallback for unverified paths. Do not +treat Cloud/Nango ingress success, an HTTP 200, or a channel post you wrote +yourself as delivery proof. ## Lifecycle Events @@ -730,9 +899,16 @@ this whole workaround once `relay#1446` lands rather than letting it outlive the | Targeted `fleet spawn` fails with `Targeted Fleet spawn requires an agent token` | Pass `--token` or set `RELAY_AGENT_TOKEN`; mint one with `agent-relay agent register --type system` (capture it without echoing). `--task` is also mandatory and the error only surfaces one problem at a time | | Node shows `online` but never receives a spawn | `online` ≠ available. Check `capabilities` contains `spawn:*` — a record can be live with no spawn capacity. Confirm with a throwaway targeted spawn, verified by `pgrep` **on the target host**, then release | | Harness blocks `sleep 25; check_inbox ...` | Bare foreground `sleep` wait loops are disallowed in harnessed environments. Run the poll loop with `run_in_background` (or Monitor + until-loop); the inline `sleep` snippets show logic only | -| Worker self-removed; can't send review fixes | Instruct workers not to self-remove until told. If already gone, spawn a fresh worker and re-inject branch + commit SHA + full verdict (see Multi-Round Review Loops) | +| Worker self-removed; can't send review fixes | Instruct workers not to self-remove until told. If already gone, spawn a fresh worker and re-inject branch + commit SHA + full verdict; rebind any GitHub PR subscription to the new live identity (see [Subscribe the live PR owner](#subscribe-the-live-pr-owner)) | | Told the user to open an observer URL built from the workspace key | That is an admin credential in a query string, and the realtime endpoint rejects it. Run `agent-relay observer` (or `get_observer_url`) and share the `ot_live_` URL it returns | -| Worker died silently; loop hangs | Inbox polling fires on messages only. Poll `agent-relay node agent list` for liveness and set a wall-clock fallback (~30 min ScheduleWakeup) | +| Worker died silently; loop hangs | Inbox polling fires on messages only. Poll `agent-relay node agent list` / `fleet agent list` for liveness and set a wall-clock fallback (~30 min ScheduleWakeup) | +| Opened a PR but did not subscribe the live owner | Review comments, pushes, and checks will not wake anyone. Immediately subscribe `@Owner` to `/pulls//**`, `/pulls//reviews/**`, `/pulls//status/**`, and `/issues//comments/**` | +| Subscribed a helper/reviewer to the same PR glob | Binding key is `(provider, resolved path)` — the second subscribe **replaces** the owner's writable route. Only the PR-owner identity gets `--to @Owner` | +| Treated `--list` / webhook create as delivery proof | Binding creation is not a wake. Confirm a real GitHub event arrives as a Relay message from `github` with a Relayfile path under that glob (`check_inbox` / `list_messages`); check `node deadletters` if it does not | +| Respawned the owner under a new name and left the old binding | New identities do not inherit subscriptions. Re-subscribe `--to @NewOwner` on the same `--resource`; confirm the prior webhook/subscription IDs retired | +| Released the owner without unsubscribing | Unsubscribe owned resources, verify IDs disappeared, then `fleet release`. Unsubscribe does not delete agents | +| Used `integration webhook create --event …` | Current CLI takes `` (inbound webhook into a channel). Provider wakes use `integration subscribe`, not that older URL/event shape | +| Used `integration subscription list` to inspect PR bindings | That lists Relay workspace event subscriptions. Provider bindings are `integration subscribe --list` | ## Prerequisites diff --git a/skills/relay-80-100-workflow/SKILL.md b/skills/relay-80-100-workflow/SKILL.md index f115941a..1315b219 100644 --- a/skills/relay-80-100-workflow/SKILL.md +++ b/skills/relay-80-100-workflow/SKILL.md @@ -1,553 +1,339 @@ --- name: relay-80-100-workflow -description: Use when writing agent-relay workflows that must fully validate features end-to-end before merging. Covers the 80-to-100 pattern - going beyond "code compiles" to "feature works, tested E2E locally." Includes repair-before-failure validation gates, review-depth fresh-eyes review/fix loops with test hardening, PGlite for in-memory Postgres testing, mock sandbox patterns, test-fix-rerun loops, verify gates after every edit, and the full lifecycle from implementation through passing tests to commit. +description: Use when writing a Relayflows v2 flow (@relayflows/surface / @relayflows/sdk, CLI `flows`) that must fully validate a feature end-to-end before it is sealed, committed, or merged - the 80-to-100 gap. Covers the evidence-recorder pattern that makes a red check work for an agent instead of the end of the run, repairable gates on the critical path, edit and hazard gates, fresh-eyes review rounds, and what a wall of green gates still misses. --- -# Writing 80-to-100 Validated Workflows +# Writing 80-to-100 Validated Flows ## Overview -Most agent workflows get features to ~80%: code written, types check, maybe a build passes. This skill covers the **80-to-100 gap** — making workflows that fully validate features end-to-end before committing. The goal: every feature merged via these workflows is **tested, verified, and known-working**, not just "it compiles." +Most agent workflows get a feature to ~80%: code written, types check, maybe a build passes. This skill covers the **80-to-100 gap** — making a flow that fully validates a feature before it commits anything. The goal is that everything these flows produce is **tested, verified and known-working**, not just "an agent said it was done." -## When to Use +Everything here is written for **Relayflows v2** — `@relayflows/surface` / `@relayflows/sdk`, CLI `flows`. For the authoring surface itself (the ladder, gates, `cli`/`model` resolution, `flows.json`, refusal shapes) see `writing-relayflows`. This skill is only about the validation shape you build on top of it. -- Writing workflows where the deliverable must be **production-ready**, not just code-complete -- Features that touch databases, APIs, or infrastructure that can be tested locally -- Any workflow where "it compiles" is not sufficient proof of correctness -- When you want confidence that the commit actually works before deploying +> Migrating from the old `@relayflows/core` `WorkflowBuilder`? v2 has no `failOnError`, no `captureOutput`, no `{{steps.X.output}}` templating and no `.onError()`. Every deterministic step is gated on exit code with no opt-out. **The evidence recorder** below is what replaces all four. See `writing-agent-relay-workflows` for the old engine. -## Core Principle: Test In The Workflow +## When to use -The key insight: **run tests as deterministic steps inside the workflow itself**. Don't just write test files — execute them, verify they pass, fix failures, and re-run. The workflow doesn't commit until tests are green. +- The deliverable must be **production-ready**, not just code-complete. +- The feature touches databases, APIs or infrastructure that can be exercised locally. +- "It compiles" is not sufficient proof of correctness. +- You want the commit at the end of the flow to represent code that is proven working. +## Green gates are necessary and nowhere near sufficient + +Read this before the mechanics; it reframes all of them. + +A 67-step flow of exactly this shape produced a tree on which **every deterministic gate was green** — scope, feature-manifest routing, targeted-verification selection, invariant tests with a mutation transcript, `cargo fmt`, `clippy -D warnings`, a release build, typecheck, 3,395 unit tests and all five end-to-end parity suites. A fresh adversarial reviewer then read that tree and returned **15 findings, 2 of them disqualifying**, with the verdict *"do not seal"*: + +- a mutation probe left shipping on a live delivery path, able to panic the process; +- a seam whose coordinator was rebuilt per call, so three of its four contract invariants had no production effect; +- an adapter that had stopped observing whether its write landed, so a dead writer read as a successful delivery. + +None of those defects was caught by the gates on that run. Gates check *properties you thought to name*. Review catches *the thing you did not think to name* — and on that run, that was most of the risk. + +So: build the gates, and do not treat them as the finish line. The review rounds below are not a formality bolted onto a working pipeline. They are where the defects were actually found. + +## The evidence recorder + +v2 fails a deterministic step the moment its command exits non-zero, and there is no opt-out. But an 80-to-100 flow does not want a red test run to end the run — a red test run is **work for the repair agent**. `{ … } || true` discards the exit code, which leaves a later gate nothing to read and lets a forgotten gate ship red work silently. + +The shape that works is a small script of your own that journals verdicts to files: + +```js +// gates.mjs — the entire contract, in two subcommands. +// +// node gates.mjs record --name unit --command-base64 [--retries 1] +// Decodes and runs the command, writes evidence/.json: +// { name, command, exitCode, verdict: 'green' | 'red', +// attempts, passedOnAttempt, tail, runId } +// and ALWAYS exits 0. +// +// node gates.mjs require-green --names unit,clippy,e2e +// Reads those files back. If any verdict is not 'green' or any runId is +// not this run's, it FIRST writes evidence/BLOCKED_NO_COMMIT.md naming +// every invalid record (including foreign-run green records), with name, +// verdict, expected runId, actual runId, and tail, THEN exits non-zero. +// Missing or unreadable records are invalid too; report unavailable fields. ``` -implement → write tests → run tests → fix failures → re-run → build check → regression check → commit + +Four details are load-bearing: + +- **Base64-encode the command.** It will contain `&&`, pipes and quotes, and the calling shell must not reinterpret them. +- **Stamp each record with the run that produced it** (`runId`, from the flow's input or an env var). `flows run --reuse-from ` keys on `step_spec_hash`, and a recorder step *always exits 0*, so it is always eligible for reuse — an unstamped recording resurrects a previous run's verdict, and `require-green` happily reads it as today's evidence. +- **Run tools from the project's own lockfile** — `npx --no vitest run`, or a package script. A bare `npx` silently fetches a package the repo never pinned, so a gate can pass against a version nobody reviewed. +- **`require-green` is the only thing that says "green."** Final acceptance recomputes the verdict from recordings; it never trusts an agent's report of its own work. It is also the step that writes `BLOCKED_NO_COMMIT` — write the artifact *before* the non-zero exit, because v2 stops the run on that exit and nothing downstream will get the chance. + +Compared to the old `failOnError: false` this is more machinery, and it buys something the old engine did not have: the repair agent reads a **file**, not an interpolated string, so it sees the whole tail rather than a truncated template. + +### The four-step ladder + +Every meaningful gate is four steps: record, repair, re-record, assert. + +```ts +import { flow } from '@relayflows/surface'; + +const b64 = (cmd: string) => Buffer.from(cmd, 'utf8').toString('base64'); +const record = (name: string, cmd: string) => + `node gates.mjs record --name ${name} --command-base64 ${b64(cmd)}`; +const requireGreen = (...names: string[]) => + `node gates.mjs require-green --names ${names.join(',')}`; + +export default flow('ship-feature', { budget: { wallclock: '110m' } }, async (f) => { + await f.run(record('unit', 'npx --no vitest run'), { timeout: '15m' }); // exits 0 either way + + await f.agent('repair-unit', { + cli: 'claude', + task: [ + 'Read evidence/unit.json.', + 'If verdict is "green", do nothing and say so. Do not invent work.', + 'If verdict is "red", read the tail, fix the source or the test,', + 'and rerun `npx --no vitest run` until it passes.', + ].join('\n'), + }); + + await f.run(record('unit', 'npx --no vitest run'), { timeout: '15m' }); + await f.run(requireGreen('unit')); // the only step here allowed to be red + + f.done('success'); +}); ``` -This means the commit at the end of the workflow represents code that is **proven working**, not just code that an agent wrote and claimed works. +The same thing in YAML, where `dependsOn` is the only ordering primitive: + +```yaml +version: '0.1.0' +name: ship-feature +steps: + - id: unit + type: deterministic + command: 'node gates.mjs record --name unit --command-base64 ' + - id: repair-unit + type: agent + dependsOn: [unit] + cli: claude + instruction: | + Read evidence/unit.json. Green means do nothing. + Red means fix it and rerun until the recorder writes green. + - id: unit-final + type: deterministic + dependsOn: [repair-unit] + command: 'node gates.mjs record --name unit --command-base64 ' + - id: unit-assert + type: deterministic + dependsOn: [unit-final] + command: 'node gates.mjs require-green --names unit' +``` -## Repair Before Failure +In TypeScript, ordering is the await sequence and `Promise.all` is first-class for fan-out. In YAML/JSON it is `dependsOn` and nothing else — there is no `pattern`, no `maxConcurrency`. An agent's retry budget is `maxIterations`, which counts the first attempt: the old `retries: 2` is `maxIterations: 3`. -An 80-to-100 workflow should not stop merely because a test, typecheck, lint, schema, or E2E gate turns red. That red output is work for the agent team. Capture it, hand it to a repair owner, fix it, and rerun. Workflow-owned validation gates should never terminate the run with `FAILED`. If the team exhausts its repair budget or hits an external blocker such as missing credentials, wrong repository, or unsafe dirty worktree, write a `BLOCKED_NO_COMMIT` artifact and end without committing or opening a PR instead of crashing the workflow. +### Retry a flaky gate once, and record which attempt passed -Use this shape for every meaningful gate: +Contention between suites is not a regression. A run 35 steps deep died to a `Verified: 2/3, Failed: 0` from a suite that passed 3/3 standalone three times in a row. Give `record` a `--retries 1`: the command must still pass, so the gate is not weakened, but the recorded tail has to say `passed on attempt 2`. A suite that only ever passes on retry then stays visible instead of being quietly laundered green. -1. `run-*`: deterministic command with `captureOutput: true` and `failOnError: false`. -2. `fix-*`: agent step that reads `{{steps.run-*.output}}`, fixes source/tests/config, and reruns the command locally until green. -3. `verify-*`: deterministic rerun, usually still `failOnError: false`, followed by a final repair step if red. -4. `commit-if-green`: deterministic step that reruns the full acceptance command and commits only when every exit code is zero. If anything is still red, it writes `BLOCKED_NO_COMMIT` with the failing evidence and exits successfully so the workflow reports a handled blocked state, not a runtime failure. +## Enforce after the agent, not on it -AgentWorkforce/relay#827 added repair-aware reliability to the SDK (`.reliable()` / `.repairable()` and repair-aware retry-mode workflows). Prefer those presets when available, but still model explicit repair owners when gate output needs domain-specific fixing. +Do not hang the enforcement of an agent's work on a gate attached to that agent step. Put it in the deterministic recorded step that follows. -## Keep Repairable Gates On The Critical Path +A gate on an agent step makes a dropped transport look like a crashed run. The same work, checked by the next deterministic step, reads as *"nothing was written"* — a repairable fact, with a journaled tail explaining it. (`writing-relayflows` has the current per-gate caveats; the shipped gate set moves between releases, and at least one gate has silently swallowed its own output.) Independently of any of that, this is the better shape, and it is the same rule as keeping repairable gates on the critical path. -Repair-before-failure only works after the workflow reaches a deterministic gate. If a long-running interactive agent step is a hard dependency for the first gate, then a dropped PTY, agent spawn error, or transport failure can stop the workflow before the repair loop ever sees evidence. +### Do not add a repair step to a gate that has nothing to repair -For large rollouts, treat implementation agents as advisory producers and put a deterministic reconciliation step on the critical path: +The ladder applies where the gate has a real, agent-fixable failure surface. A +deterministic gate sitting downstream of deterministic work — a transcript +line-count check after a series of deterministic recorder steps — is correct by +construction; there is no failure mode a repair agent could act on. Wiring one in +anyway spawns a fresh agent for a guaranteed no-op, and pays the spawn cost plus, +often, a follow-on idle wait. Keep the hard gate, skip the repair. -1. Start implementation/review agents in parallel if useful, but require them to write durable artifacts such as `.workflow-artifacts//runtime.md`, self-review notes, changed-file lists, and command evidence. -2. Add `implementation-reconcile`: a deterministic step that inspects `git status --short -- `, required files, artifact files, and diff stats. It should use `captureOutput: true` and `failOnError: false`. -3. Add `repair-implementation-reconcile`: a focused repair owner that reads the reconcile output and finishes missing artifacts or code before validation gates run. -4. Make discovery, typecheck, E2E, and final acceptance depend on the reconcile/repair path, not directly on every long-lived implementation agent. -5. Keep the final commit deterministic and green-only; red final evidence becomes a repair/blocking artifact, not a failed workflow. +### Grade computable things in a deterministic step, never with an LLM -This shape prevents "agent transport failed" from masquerading as "the product failed." The product still has to pass the same gates; the difference is that the workflow can reach the gates and repair them. +If the comparison is computable — a string match, a test's pass/fail, a regex, schema +validity — grade it in a `run` step and record the verdict. An `llm` or `agent` step +asked to adjudicate a fixed rule will disagree with itself across runs, and its verdict +is not evidence. Agents *produce*; deterministic steps *judge*. -## Squad Review Before Final Acceptance +### Keep repairable gates on the critical path -For high-stakes implementation workflows, validation should include human-like review structure, not only command gates. Use small implementation squads and make review state durable: +Repair-before-failure only helps once the flow *reaches* a deterministic gate. If a long-running agent step is a hard dependency of the first gate, a dropped worker stops the flow before the repair loop ever sees evidence. -1. Split independent scopes into 2-3 agent squads. Each squad has an implementer, a shadow reviewer, and optionally a validation/test owner. -2. The shadow reviewer follows the implementer while work is happening and flags spec drift early. -3. Before external review, the implementer writes a self-reflection artifact under `.workflow-artifacts//` covering spec coverage, changed files, tests/proofs, repo-rule alignment, and known risks. -4. A fresh self-review agent reads the actual files, AGENTS.md / CLAUDE.md, recent related work, and local conventions. It writes findings to disk. -5. The implementer repairs valid findings, then deterministic gates rerun from captured output. -6. After all squads converge, run the selected review-depth fresh-eyes review/fix path. Light requires `review-claude` -> `fix-loop` and gates final review pass on `post-fix-validation`. Standard adds `final-review-claude` -> `final-fix-claude` and gates final review pass on `final-fix-claude`. Deep requires the standard Claude path plus `review-codex` -> `fix-loop-codex` -> `final-review-codex` -> `final-fix-codex` and gates final review pass on `final-fix-codex`. -7. If the selected review path still finds issues, run another explicit fix pass or write `BLOCKED_NO_COMMIT` with exact evidence. -8. Commit or PR creation is allowed only after the selected review-depth path, final-review-pass gate, final deterministic acceptance, and scoped diff/regression gates are green. Otherwise write a `BLOCKED_NO_COMMIT` artifact with exact evidence. +1. Treat implementation agents as advisory producers. Require them to write durable artifacts: a changed-file list, command evidence, self-review notes. +2. Add a deterministic `reconcile` recorder that inspects `git status --short -- `, required files and diff stats. +3. Add a repair owner that reads the reconcile evidence and finishes whatever is missing. +4. Make typecheck, E2E and final acceptance depend on the reconcile/repair path — not directly on every long-lived agent. +5. Keep the final commit deterministic and green-only. Red final evidence becomes a `BLOCKED_NO_COMMIT` artifact, not a crashed run. -This keeps "100%" tied to both executable evidence and independent review over the final state. +This stops "the agent transport failed" from masquerading as "the product failed." The product still has to pass the same gates; the difference is that the flow can reach them. -## The Test-Fix-Rerun Pattern +### Partition the agents' lanes yourself -Every testable feature in a workflow should follow this four-step pattern: +`permissions` on an agent step is journaled and **not** enforced — a `flows run` is not a sandbox, and every agent shares one checkout. Declare `permissions` for reviewability, then partition lanes by path *and* sequence the agents that touch adjacent ones, and make an edit gate that rejects out-of-scope changes. And pass `--local-agent`: without it a flow containing any agent step parks at the first one, with a message that reads as a missing worker rather than a missing flag. -```typescript -// Step 1: Run tests (allow failure — we expect issues on first run) -.step('run-tests', { - type: 'deterministic', - dependsOn: ['create-tests'], - command: 'npx tsx --test tests/my-feature.test.ts 2>&1 | tail -60', - captureOutput: true, - failOnError: false, // <-- Don't fail the workflow, let the agent fix it -}) +## Verify every edit -// Step 2: Agent reads output, fixes issues, re-runs until green -.step('fix-tests', { - agent: 'tester', - dependsOn: ['run-tests'], - task: `Check the test output and fix any failures. +Never trust that an agent edited a file. Establish a clean pre-edit baseline for the target paths, then record a deterministic check after every agent edit and route its evidence to a repair owner. If existing work must be preserved, snapshot it and compare against that snapshot instead; do not clean it away. -Test output: -{{steps.run-tests.output}} +```ts +const editSchemaCheck = ` + state=$(git status --short -- lib/db/schema.ts) || exit 1 + if [ -z "$state" ]; then echo NOT_MODIFIED; exit 1; fi + grep -q my_new_table lib/db/schema.ts || { echo MISSING_TABLE; exit 1; } + echo EDIT_OK +`; -If all tests passed, do nothing. -If there are failures: -1. Read the failing test file and source files -2. Fix the issues (could be in test or source) -3. Re-run: npx tsx --test tests/my-feature.test.ts -4. Keep fixing until ALL tests pass.`, - verification: { type: 'exit_code' }, -}) +// Stop before editing if this path already has staged, unstaged, or untracked work. +await f.run(` + state=$(git status --short -- lib/db/schema.ts) || exit 1 + if [ -n "$state" ]; then echo DIRTY_BASELINE; exit 1; fi +`); -// Step 3: Deterministic rerun — capture result for a final repair pass -.step('run-tests-final', { - type: 'deterministic', - dependsOn: ['fix-tests'], - command: 'npx tsx --test tests/my-feature.test.ts 2>&1', - captureOutput: true, - failOnError: false, -}) +await f.agent('edit-schema', { cli: 'claude', task: 'Edit lib/db/schema.ts …' }); +await f.run(record('edit-schema', editSchemaCheck)); -// Step 4: Repair again if the rerun is still red -.step('fix-tests-final', { - agent: 'tester', - dependsOn: ['run-tests-final'], - task: `If the final test rerun passed, record the green evidence. -If it failed, fix the remaining issue and rerun until green: -{{steps.run-tests-final.output}}`, - verification: { type: 'exit_code' }, -}) +await f.agent('repair-edit-schema', { + cli: 'claude', + task: 'Read evidence/edit-schema.json. Green means do nothing. Red means make the edit land.', +}); + +await f.run(record('edit-schema', editSchemaCheck)); +await f.run(requireGreen('edit-schema')); ``` -**Why four steps instead of one?** -- The first run captures output for the agent to diagnose -- The agent step can iterate (read errors, fix, re-run) multiple times -- The final deterministic run is still evidence-based, but a repair agent sees it before the workflow stops -- The last repair step keeps the workflow aligned with the agent-team model instead of ending on a fixable failure +**Use `git status --short -- `, not `git diff --quiet`.** `git diff` only sees tracked changes, so a valid new package, test directory or generated artifact is misclassified as "no changes." `git diff --quiet` is fine only for tracked-only edits to files that already exist. -## PGlite: In-Memory Postgres for Database Testing +**Verify** that the file changed from the pre-edit baseline (reuse that same baseline for repair and final acceptance), and that key content exists (a table name, an exported symbol, an import). **Do not verify** exact content, formatting, line counts or byte sizes — agents format differently and those checks are pure brittleness. -When your feature touches the database, use **PGlite** — a WASM-based Postgres that runs in-process. No Docker, no external services, no flaky network dependencies. +### Gate the hazard, not a tool's mode -### Setup +A gate that asserts *"the selector reported `targeted`"* rather than *"no runtime path is unmapped"* can be unsatisfiable by construction. In one campaign, registering a feature required editing the manifest, and editing the manifest forced the selector into `full-smoke` unconditionally. The gate failed correct work — and it read as the agent's fault. Assert the condition you actually care about, read out of the tool's structured output, not its summary verdict. -Install as a dev dependency in the workflow: +### Verify at the granularity the risk lives at -```typescript -.step('install-pglite', { - type: 'deterministic', - command: 'npm install --save-dev @electric-sql/pglite 2>&1 | tail -5', - captureOutput: true, -}) -``` +"256 test functions before, 256 after, so nothing was deleted" is wrong: roughly 90 assertions had been removed from *inside* those functions. Count what can actually be weakened, not the container it sits in. -### Test Helper Pattern +## A requirement that induces a hazard must gate that hazard -Create a reusable helper that boots an in-memory Postgres with your schema: +This one cost a near-miss, and it generalises much further than it looks. -```typescript -// tests/helpers/pglite-db.ts -import { PGlite } from '@electric-sql/pglite'; -import { drizzle } from 'drizzle-orm/pglite'; -import * as schema from '../../packages/web/lib/db/schema.js'; - -// Raw DDL matching your Drizzle schema — PGlite doesn't run Drizzle migrations -const MY_TABLE_DDL = ` -CREATE TABLE IF NOT EXISTS my_table ( - id UUID PRIMARY KEY DEFAULT gen_random_uuid(), - name TEXT NOT NULL, - created_at TIMESTAMPTZ NOT NULL DEFAULT NOW() -); -`; +The flow required proof that tests bite: *mutate the guarded code, capture the failing transcript, restore it.* An implementer did the first two and skipped the third, leaving this in shipping product code on a live delivery path: -export async function createTestDb() { - const pg = new PGlite(); - await pg.exec(MY_TABLE_DDL); - const db = drizzle(pg, { schema }); - return { db, pg, schema, cleanup: () => pg.close() }; +```rust +// MUTATION: drop the addressee and truncate the body. +if std::env::var("RELAY_MUTATION_LOSSY_FORMAT").is_ok() { + return format!("Relay message from {}:\n\n{}", delivery.from, &delivery.body[..1]); } ``` -### PGlite Gotchas - -| Issue | Fix | -|-------|-----| -| `pgcrypto` extension not available | Use `gen_random_uuid()` (built-in since PG 13) or generate UUIDs in app code | -| UUID columns | PGlite supports UUID natively — no special handling needed | -| `drizzle-orm/pglite` import | Exists since drizzle-orm 0.30+. If not found, check version. | -| Index creation | PGlite supports standard CREATE INDEX — no limitations | -| Concurrent writes | PGlite is single-connection. Test concurrent logic with sequential assertions. | - -### Test Structure - -```typescript -// tests/my-feature.test.ts -import { describe, it } from 'node:test'; -import assert from 'node:assert/strict'; -import { randomUUID } from 'node:crypto'; -import { createTestDb } from './helpers/pglite-db.js'; - -describe('my feature', () => { - it('does the thing correctly', async () => { - const { db, schema, cleanup } = await createTestDb(); - try { - // Arrange - const testId = randomUUID(); - // Act — use your module against the real (in-memory) Postgres - // Assert - assert.equal(result.name, 'expected'); - } finally { - await cleanup(); - } - }); -}); -``` +`&body[..1]` panics on a multi-byte first character. -## Verify Gates After Every Edit - -Never trust that an agent edited a file correctly. Add a deterministic verify gate after every agent edit step: - -```typescript -// Agent edits a file -.step('edit-schema', { - agent: 'impl', - dependsOn: ['read-schema'], - task: `Edit packages/web/lib/db/schema.ts...`, - verification: { type: 'exit_code' }, -}) - -// Deterministic verification — did the edit actually land? -.step('verify-schema', { - type: 'deterministic', - dependsOn: ['edit-schema'], - command: `if git diff --quiet packages/web/lib/db/schema.ts; then echo "NOT MODIFIED"; exit 1; fi -grep "my_new_table" packages/web/lib/db/schema.ts >/dev/null && echo "OK" || (echo "MISSING"; exit 1)`, - failOnError: false, - captureOutput: true, -}) -.step('fix-schema-verification', { - agent: 'impl', - dependsOn: ['verify-schema'], - task: `Fix the schema edit if verification failed. Output:\n{{steps.verify-schema.output}}`, - verification: { type: 'exit_code' }, -}) -``` +**Every deterministic gate passed that tree.** The edit gate saw a changed file in scope; the invariant gate saw four correctly-named tests and a mutation transcript; typecheck, clippy, the release build and all five parity suites were green. Only the adversarial reviewer caught it. -**What to verify:** -- File was actually modified (`git diff --quiet` returns non-zero) -- Key content exists (grep for table names, function names, imports) -- For new files: `file_exists` verification type -- For new directories, package trees, generated files, or mixed tracked/untracked - edits: use `git status --short -- `, because `git diff --quiet` - ignores untracked files - -**What NOT to verify:** -- Exact content (too brittle — agents format differently) -- Line counts or byte sizes (meaningless) - -### Edit Gates That Include New Files - -When an agent may create new files or package directories, do not use -`git diff --quiet -- ` as the only edit gate. It only sees tracked -changes, so a valid new package can be misclassified as "no changes." - -Use `git status --short -- ` and keep the first gate repairable: - -```typescript -.step('edit-gate-capture', { - type: 'deterministic', - dependsOn: ['implement'], - command: `if [ -z "$(git status --short -- packages/new-adapter tests docs)" ]; then - echo "NO_CHANGES" - exit 1 -fi -echo "EDIT_GATE_OK"`, - captureOutput: true, - failOnError: false, -}) -.step('fix-edit-gate', { - agent: 'impl', - dependsOn: ['edit-gate-capture'], - task: `If the edit gate reported NO_CHANGES, inspect the acceptance contract -and current git status, then add the missing source/test/artifacts. - -Gate output: -{{steps.edit-gate-capture.output}} - -If it already passed, do nothing.`, - verification: { type: 'exit_code' }, -}) -.step('edit-gate-final', { - type: 'deterministic', - dependsOn: ['fix-edit-gate'], - command: `if [ -z "$(git status --short -- packages/new-adapter tests docs)" ]; then - echo "NO_CHANGES" - exit 1 -fi -echo "EDIT_GATE_FINAL_OK"`, - captureOutput: true, - failOnError: true, -}) -``` +Read literally by an agent, "prove the test fails" is an instruction to damage production code. If you ask for that proof, you must gate the residue: -Rule of thumb: `git diff --quiet` is fine for tracked-only edits to known -files. Use `git status --short -- ` for materialization gates that may -include new tests, docs, generated artifacts, or package directories. - -## Mock Sandbox Pattern - -When testing code that interacts with Daytona sandboxes, use inline mock objects matching the existing test conventions: - -```typescript -const daytona = { - create: async () => ({ - id: 'sandbox-id', - process: { - executeCommand: async (cmd, cwd, env) => ({ - result: 'output', - exitCode: 0, - }), - }, - fs: { - uploadFile: async () => undefined, - }, - getUserHomeDir: async () => '/home/daytona', - }), - remove: async () => undefined, -}; +```js +// refuse mutation scaffolding anywhere in product source +for (const file of walk('src').filter((f) => f.endsWith('.rs'))) { + if (/MUTATION|_MUTATION_/.test(readFileSync(file, 'utf8'))) problems.push(`residue in ${file}`); +} ``` -For testing that your code calls the right methods, record calls in an array: +Then prove *that* gate bites, by reinstating a one-line probe and watching it go red. -```typescript -const emitted: EmitEventOptions[] = []; -const mockClient: SessionEventClient = { - emit: async (opts) => { emitted.push(opts); }, - getEvents: async () => [], - getLatestSequence: async () => 0, -}; +Generalise it: **any instruction that tells an agent to temporarily break something needs a matching check that it was put back.** Temporarily lowering a timeout, stubbing a provider, disabling a guard — same shape, same hazard. -// ... run the code ... +### A test that guards a helper does not guard the call site -assert.equal(emitted.length, 4); -assert.equal(emitted[0].eventType, 'sandbox_created'); -``` +A mutation proof must mutate **where the bug would be written**, not where the abstraction lives. Mutating the shared helper and watching the suite go red proves only that the helper is covered; the call site that forgot to use the helper is exactly the defect you were trying to exclude, and it survives untouched. Point the probe at the production call path. + +## Cross-check your gates against each other + +Nothing validates that the assertions you write are mutually satisfiable, and the agents pay for it when they are not. In one campaign a `seam-rules` gate **required** a file to be edited while an `edit-gate` **rejected** that same file as out-of-scope. No implementation could pass both. The agent began reverting correct work to appease the contradiction — and it read as the agent failing. + +If one gate names a path, assert at authoring time that every other gate permits it. That check is a few lines, and it found a real contradiction immediately. + +## Give every repair owner a "do nothing" clause -## Regression Testing +A repair agent handed a green gate will invent work rather than conclude there is none. One burned 21 minutes on a gate reporting `not-required`, against a 2–7 minute norm, because its prompt said "make the scenario real" with no branch for the passing case. -After your new tests pass, always run the **existing test suite** to catch regressions: +Every repair prompt needs an explicit *"if the evidence is green, do nothing and say so."* Audit them: 4 of 7 in an otherwise mature flow were missing it. -```typescript -.step('run-existing-tests', { - type: 'deterministic', - dependsOn: ['fix-build'], - command: 'npm run orchestrator:test 2>&1 | tail -40', - captureOutput: true, - failOnError: false, -}) +## Seal the product tree, not just the evidence -.step('fix-regressions', { - agent: 'impl', - dependsOn: ['run-existing-tests'], - task: `Check the full test suite for regressions caused by our changes. +If a signoff step binds a reviewer to an artifact digest, digest the **changed source files** too. Otherwise the reviewer is bound to a hash of *evidence files* while the code those files describe changes underneath them. Prove it bites: a one-line source edit must change the digest. -Test output: -{{steps.run-existing-tests.output}} +## Review before acceptance -If all tests passed, do nothing. -If EXISTING tests broke, read the failing test, find what we broke, fix it. -Most likely cause: constructor signatures changed, new required fields added -without defaults, or import paths shifted. +Command gates are not review. For high-stakes work, make review state durable and put it on the critical path: -Run: npm run orchestrator:test -Fix until all tests pass.`, - verification: { type: 'exit_code' }, -}) +1. Split independent scopes into small squads: an implementer, a shadow reviewer who flags spec drift while the work happens, and optionally a test owner. +2. Before external review, the implementer writes a self-reflection artifact — spec coverage, changed files, proofs, repo-rule alignment, known risks. +3. A **fresh-context** reviewer reads the actual files, `AGENTS.md` / `CLAUDE.md`, recent related work and local conventions, and writes findings to disk. Fresh context is the point: a reviewer that watched the work being built inherits its blind spots. +4. The implementer repairs valid findings; the deterministic gates re-record from scratch. +5. Scale the depth to the stakes — one Claude review/fix round, then a second round, then an independent Codex review/fix round for the deepest tier. Gate final acceptance on the *last* fix step of whichever tier you chose. +6. Commit or PR creation is allowed only after the review path, final deterministic acceptance and scoped diff/regression gates are all green. Otherwise write `BLOCKED_NO_COMMIT` with the exact evidence and end without committing. + +## Local test substrates + +Tests that need Docker or a network do not belong in a gate you intend to rerun four times. + +**PGlite** (`@electric-sql/pglite`) is a WASM Postgres running in-process — no Docker, no external service, no flaky network. Boot it with raw DDL matching your schema (it does not run Drizzle migrations), and hand the client to the code under test: + +```ts +import { PGlite } from '@electric-sql/pglite'; +import { drizzle } from 'drizzle-orm/pglite'; +import * as schema from '../../lib/db/schema.js'; + +export async function createTestDb() { + const pg = new PGlite(); + await pg.exec(MY_TABLE_DDL); // raw DDL, kept in step with schema.ts + return { db: drizzle(pg, { schema }), pg, schema, cleanup: () => pg.close() }; +} ``` -## Full Workflow Template - -Here's the complete pattern for a feature that touches the database: - -```typescript -import { workflow } from '@relayflows/core'; - -const result = await workflow('my-feature') - .description('Add feature X with full E2E validation') - .pattern('dag') - .channel('wf-my-feature') - .maxConcurrency(3) - .timeout(3_600_000) - .repairable() - - .agent('impl', { cli: 'claude', preset: 'worker', retries: 2 }) - .agent('tester', { cli: 'claude', preset: 'worker', retries: 2 }) - - // ── Phase 1: Read ──────────────────────────────────────────────── - .step('read-target', { - type: 'deterministic', - command: 'cat path/to/file.ts', - captureOutput: true, - }) - - // ── Phase 2: Implement ─────────────────────────────────────────── - .step('edit-target', { - agent: 'impl', - dependsOn: ['read-target'], - task: `Edit path/to/file.ts. Current contents: -{{steps.read-target.output}} - -Only edit this one file.`, - verification: { type: 'exit_code' }, - }) - .step('verify-target', { - type: 'deterministic', - dependsOn: ['edit-target'], - command: 'git diff --quiet path/to/file.ts && (echo "NOT MODIFIED"; exit 1) || echo "OK"', - failOnError: false, - captureOutput: true, - }) - .step('fix-target-verification', { - agent: 'impl', - dependsOn: ['verify-target'], - task: `Fix the target edit if verification failed. Output:\n{{steps.verify-target.output}}`, - verification: { type: 'exit_code' }, - }) - - // ── Phase 3: Test infrastructure ───────────────────────────────── - .step('install-pglite', { - type: 'deterministic', - command: 'npm install --save-dev @electric-sql/pglite 2>&1 | tail -5', - captureOutput: true, - }) - .step('create-test-helpers', { - agent: 'tester', - dependsOn: ['install-pglite'], - task: 'Create tests/helpers/pglite-db.ts with ...', - verification: { type: 'file_exists', value: 'tests/helpers/pglite-db.ts' }, - }) - .step('create-tests', { - agent: 'tester', - dependsOn: ['create-test-helpers', 'fix-target-verification'], - task: 'Create tests/my-feature.test.ts with ...', - verification: { type: 'file_exists', value: 'tests/my-feature.test.ts' }, - }) - - // ── Phase 4: Test-fix-rerun loop ───────────────────────────────── - .step('run-tests', { - type: 'deterministic', - dependsOn: ['create-tests'], - command: 'npx tsx --test tests/my-feature.test.ts 2>&1 | tail -60', - captureOutput: true, - failOnError: false, - }) - .step('fix-tests', { - agent: 'tester', - dependsOn: ['run-tests'], - task: `Fix any test failures. Output:\n{{steps.run-tests.output}}`, - verification: { type: 'exit_code' }, - }) - .step('run-tests-final', { - type: 'deterministic', - dependsOn: ['fix-tests'], - command: 'npx tsx --test tests/my-feature.test.ts 2>&1', - captureOutput: true, - failOnError: false, - }) - .step('fix-tests-final', { - agent: 'tester', - dependsOn: ['run-tests-final'], - task: `If the final test rerun is red, fix and rerun until green. Output:\n{{steps.run-tests-final.output}}`, - verification: { type: 'exit_code' }, - }) - - // ── Phase 5: Build + regression ────────────────────────────────── - .step('build-check', { - type: 'deterministic', - dependsOn: ['fix-tests-final'], - command: 'npx tsc --noEmit 2>&1 | tail -20; echo "EXIT: $?"', - captureOutput: true, - failOnError: false, - }) - .step('fix-build', { - agent: 'impl', - dependsOn: ['build-check'], - task: `Fix type errors if any. Output:\n{{steps.build-check.output}}`, - verification: { type: 'exit_code' }, - }) - .step('run-existing-tests', { - type: 'deterministic', - dependsOn: ['fix-build'], - command: 'npm test 2>&1 | tail -40', - captureOutput: true, - failOnError: false, - }) - .step('fix-regressions', { - agent: 'impl', - dependsOn: ['run-existing-tests'], - task: `Fix regressions if any. Output:\n{{steps.run-existing-tests.output}}`, - verification: { type: 'exit_code' }, - }) - - // ── Phase 6: Commit ────────────────────────────────────────────── - .step('commit', { - type: 'deterministic', - dependsOn: ['fix-regressions'], - command: [ - 'npx tsx --test tests/my-feature.test.ts', - 'npm test', - 'git add ', - 'git commit -m "feat: ..."', - ].join(' && '), - captureOutput: true, - failOnError: false, - }) - .step('repair-commit', { - agent: 'impl', - dependsOn: ['commit'], - task: `If commit failed, fix the blocker, rerun the feature and regression tests, and create the commit. -If commit passed, confirm the commit subject. -Output: -{{steps.commit.output}}`, - verification: { type: 'exit_code' }, - }) - .step('verify-commit-created', { - type: 'deterministic', - dependsOn: ['repair-commit'], - command: 'git log -1 --pretty=%s | grep -q "^feat: " && echo "COMMIT_OK" || (echo "COMMIT_MISSING"; exit 1)', - captureOutput: true, - failOnError: true, - }) - - .onError('retry', { maxRetries: 2, retryDelayMs: 10_000 }) - .run({ cwd: process.cwd() }); +Gotchas: no `pgcrypto` (use the built-in `gen_random_uuid()`); single connection, so exercise concurrency with sequential assertions; `drizzle-orm/pglite` needs drizzle-orm 0.30+. The real hazard is DDL drift — if the test DDL and `schema.ts` disagree, the tests pass against a schema that does not exist. Derive one from the other, or gate them against each other. + +For external services (sandboxes, event clients), prefer an inline mock that **records calls into an array**, then assert on the recording. Asserting "it did not throw" proves nothing about what it called: + +```ts +const emitted: EmitEventOptions[] = []; +const mockClient: SessionEventClient = { emit: async (o) => { emitted.push(o); }, /* … */ }; +// … +assert.equal(emitted.length, 4); +assert.equal(emitted[0].eventType, 'sandbox_created'); ``` -## Checklist: Is Your Workflow 80-to-100? +## Checklist: is your flow 80-to-100? | Check | How | -|-------|-----| -| Tests exist | `file_exists` verification on test file | -| Tests actually run | Deterministic step executes them | -| Test failures get fixed | Agent step reads output, fixes, re-runs | -| Final test run is repairable | Deterministic rerun captures output, then a repair owner gets one more pass | -| Build passes | `npx tsc --noEmit` deterministic step | -| No regressions | Existing test suite runs after changes | -| Every edit is verified and repairable | `git diff --quiet` + grep for tracked-only edits; `git status --short -- ` when new files/packages may appear; then a fix step | -| Commit only happens after green evidence | Final commit step reruns acceptance checks and commits only on zero exit codes | - -## Common Anti-Patterns +|---|---| +| Tests exist | Deterministic check for the file, recorded | +| Tests actually run | A recorder step runs them; nobody's summary is trusted | +| Failures become work, not a dead run | Recorder always exits 0; a repair owner reads `evidence/.json` | +| Green is recomputed, not reported | `require-green` reads the recordings at the end | +| Evidence belongs to this run | Every record stamped with `runId`; `--reuse-from` can't resurrect a stale one | +| Flaky gates stay visible | `--retries 1`, and the tail names the attempt that passed | +| Every edit is verified and repairable | `git status --short -- ` + a content grep, then a repair step | +| Temporary breakage is put back | A residue gate for every "prove it fails" instruction, proven to bite | +| Gates are mutually satisfiable | Authoring-time cross-check of every path a gate names | +| Fresh eyes read the final tree | Review rounds on the critical path, findings written to disk | +| Commit only after green evidence | Final step recomputes acceptance; red writes `BLOCKED_NO_COMMIT` | + +## Common anti-patterns | Anti-pattern | Why it fails | Fix | -|-------------|-------------|-----| -| Tests written but never executed | Agent claims they pass, they don't | Add deterministic `run-tests` step | -| Single `failOnError: true` test run | First failure kills workflow, no chance to fix | Use repairable run-fix-rerun-final-fix loops | -| No regression test | New feature works, old features break | Run `npm test` after build check | -| Agent asked to "write and run tests" in one step | Agent writes tests, runs them, they fail, it edits, output is garbled | Separate write/run/fix into distinct steps | -| PGlite DDL doesn't match Drizzle schema | Tests pass on wrong schema | Derive DDL from schema.ts or test with real migration | -| Final test output not handed to an agent | Broken tests can stop the run or get ignored | Add a final repair owner before commit | -| Testing only happy path | Edge cases break in prod | Specify edge case tests in the task prompt | -| No verify gate after agent edits | Agent exits 0 without writing anything | Add `git diff --quiet` check after every edit, then route failures to a repair step | -| `git diff --quiet` for new package/test directories | Untracked files are invisible, so valid new artifacts can look like "no changes" | Use `git status --short -- ` and a repairable capture → fix → final gate pattern | -| Committing after `failOnError: false` without checking exits | Broken work can be committed because the shell step returned successfully | In `commit-if-green`, record each exit code and skip commit unless all are zero | +|---|---|---| +| Tests written but never executed | The agent claims they pass; they don't | A recorder step runs them | +| A bare check as the gate | The first red kills the run, with no chance to repair | Record → repair → re-record → assert | +| `\|\| true` instead of a recorder | Throws the exit code away; a forgotten gate then ships red work silently | Journal `{exitCode, verdict, tail}` to a file | +| Unstamped evidence + `--reuse-from` | Recorder steps always exit 0, so they are always reusable — stale verdicts read as fresh | Stamp `runId`; `require-green` rejects foreign stamps | +| Gating enforcement on the agent step | A dropped transport reads as a crashed run instead of "nothing was written" | Enforce in the deterministic step after | +| Repair prompt with no green branch | The agent invents work against a passing gate | "If the evidence is green, do nothing and say so" | +| `git diff --quiet` for new files | Untracked files are invisible, so valid new packages look like "no changes" | `git status --short -- ` | +| Asserting a tool's summary verdict | Can be unsatisfiable by construction; fails correct work | Assert the hazard, from structured output | +| Counting containers, not contents | 256 tests before and after hid ~90 deleted assertions | Verify at the granularity the risk lives at | +| Mutating the helper to prove coverage | Proves the helper is covered, not the call site that forgot it | Mutate where the bug would be written | +| Sealing only the evidence directory | The reviewer is bound to a hash of the wrong thing | Digest the changed source too, and prove it bites | +| Treating all-green as done | 15 findings, 2 disqualifying, on an all-green tree | Fresh-eyes review rounds before acceptance | +| A repair step on a gate with no agent-fixable failure | Burns a spawn cycle on a guaranteed no-op, plus the idle wait after it | Ladder only where an agent action could turn red green | +| An `llm`/`agent` step used as a grading oracle | A computable rule adjudicated by a model disagrees with itself across runs | Grade it in a `run` step and record the verdict | +| Copying a v1 `.step({ failOnError, captureOutput })` | Those fields do not exist in v2; the flow will not compile | The evidence recorder | diff --git a/skills/writing-agent-relay-workflows/SKILL.md b/skills/writing-agent-relay-workflows/SKILL.md index 35445e8b..f2181580 100644 --- a/skills/writing-agent-relay-workflows/SKILL.md +++ b/skills/writing-agent-relay-workflows/SKILL.md @@ -1,10 +1,28 @@ --- name: writing-agent-relay-workflows -description: Use when building multi-agent workflows with relay broker-sdk. Covers conversation vs pipeline coordination, WorkflowBuilder/DAG steps, agents, {{steps.X.output}} chaining, Relayfile-backed human assistance, Slack answer injection, integration subscriptions, waitFor gates, repairable verification, review-depth fresh-eyes review/fix loops with test hardening, channels, chat-native recipes, error handling, event listeners, step sizing, lead+workers teams, and parallel waves. +description: DEPRECATED - documents the superseded v1 `@relayflows/core` WorkflowBuilder engine (`workflow().pattern().agent().step().run()`, `failOnError`, `captureOutput`, `{{steps.X.output}}`, `.onError()`). Use `writing-relayflows` for the current v2 engine (`@relayflows/surface`/`@relayflows/sdk`, CLI `flows`) on all new work. Keep this open only to read or maintain an existing v1 workflow: conversation vs pipeline coordination, DAG steps, idle detection, Relayfile-backed human assistance, waitFor gates, review-depth review/fix loops, channels, chat-native recipes, and parallel waves. --- # Writing Agent Relay Workflows +> ## ⚠️ DEPRECATED — this is the v1 engine +> +> This skill documents **Relayflows v1**: `@relayflows/core`'s `WorkflowBuilder`, the +> chained builder (`workflow('name').pattern('dag').agent(...).step(...).run()`) with +> `failOnError`, `captureOutput`, `{{steps.X.output}}` templating and `.onError()`. +> **v1 is superseded.** +> +> **New work goes to the v2 engine** — `@relayflows/surface`'s `flow()` and the +> YAML/JSON dialect compiled by `@relayflows/sdk`, CLI binary `flows`, packages `2.0.x`. +> Read [`writing-relayflows`](../writing-relayflows/SKILL.md) for the authoring surface +> and [`relay-80-100-workflow`](../relay-80-100-workflow/SKILL.md) for the v2 validation +> shape. None of `failOnError`, `captureOutput`, `{{steps.X.output}}`, `.onError()` or +> `idleThresholdSecs` exists in v2, so following this skill for a v2 flow produces +> something that will not compile. +> +> Everything below is kept intact for people reading or maintaining an **existing v1 +> workflow**. + ## Overview The relay broker-sdk workflow system orchestrates multiple AI agents (Claude, Codex, Gemini, Aider, Goose) through typed DAG-based workflows. Workflows can be written in **TypeScript** (preferred), **Python**, or **YAML**. @@ -64,6 +82,8 @@ The point of an agent team workflow is not to discover a red gate and stop. The Avoid hard-stop gates (`failOnError: true` with no repair step) in workflows that are supposed to be self-healing. Even cheap preconditions such as missing credentials, wrong repository, or an unsafe dirty worktree should normally write a clear `BLOCKED_*` artifact and exit cleanly. For implementation, build, test, lint, schema, artifact, and review failures, model the fix path in the workflow. +**Counterweight — don't reflexively add a repair step.** This rule applies when the gate has a real, agent-fixable failure surface. A deterministic gate that sits downstream of deterministic work (e.g. a transcript line-count check after a series of deterministic grader steps) is correct by construction — there is no failure mode for a repair agent to address. Wiring a repair step there just spawns a fresh agent for a guaranteed no-op, paying the spawn cost (and often a follow-on idle-timeout wait) for nothing. Keep the hard gate, skip the repair. Reserve repair branches for gates where a real agent action could plausibly turn red into green. + ## Human Assistance and Relayfile Events Relayflows can block on humans and provider events without shelling out to Slack, GitHub, Linear, or the Relayfile CLI. Declare the integration intent in the workflow and let the SDK/runtime use the existing local Relayfile/Pear connection. @@ -266,6 +286,7 @@ Before writing the workflow, decide *how the agents will coordinate*. The relay | Shape | What it is | Use when | |---|---|---| | **Conversation** (chat-native) | Interactive agents share a channel; messages, `@-mentions`, and ambient awareness drive coordination. Lead and workers spawn in parallel and self-organize. The relay is the coordination layer, not just transport. | Multi-file work, peer review loops, cross-agent feedback, dynamic re-planning, multi-PR coordination, anything with a human-in-the-loop escape, swarms where workers pick up each other's output. | +| **Per-turn interactive spawn** (bounded interactive) | Interactive PTY agents (no `preset`), but each step is bounded to *one* turn that exits as soon as its file/message is produced. Agents still join the channel — they just don't live across turns. | Turn-taking flows where each turn is short, well-defined, finishes **inside the idle window**, and reads its inputs from a deterministic source (file on disk, prior grade step). Cuts the long-running wait-for-event race; a silent turn that itself exceeds ~30s still needs `idleThresholdSecs`. | | **Pipeline** (one-shot DAG) | Each step runs as a one-shot subprocess (`claude -p`, `codex exec`); steps hand off via `{{steps.X.output}}` text injection. No agents are alive at the same time; no chat happens. | Linear, well-specified transformations; deterministic data passing; no live agent-to-agent coordination during implementation. The selected review-depth path and deterministic final gates still apply. | **Default to Conversation for any non-trivial work.** Pipeline DAGs are simpler to reason about but they do not exercise the relay primitive — they are a Unix pipe with extra steps. If you would happily write the same task as a single shell pipeline, pipeline-shape is fine. Otherwise, you almost certainly want a Conversation shape. @@ -1723,6 +1744,22 @@ Edit files as assigned. Report completion. Fix issues from feedback.`, Once you're in the Interactive Team shape, the channel is your coordination medium. These are recipes for using it well — they are *prompt-authoring patterns*, not new SDK surface. All of them assume interactive agents (no `preset`) sharing a `.channel('wf-...')`. +> **Idle detection beats "wait for X" prompts.** The runtime treats a silent interactive PTY as complete after a short idle window (~30 seconds). A task prompt that tells an agent to *wait for message Y* / *exit only on `GAME_OVER`* / *stay until @reviewer posts a verdict* will lose that race if the awaited event is more than ~30s away — the runtime will mark the agent complete and tear it down before the message arrives. This affects every recipe below: **Q/A, Broadcast/Ack, Peer Review, and Hand-Off all encode a wait.** +> +> **Two ways to handle it:** +> +> 1. **Per-agent escape hatch — set `idleThresholdSecs` on the agent.** The builder option `idleThresholdSecs` overrides the per-agent idle window (default 30s; `0` disables idle detection entirely). Use this when a specific agent reliably has long quiet stretches by design: +> ```typescript +> .agent('long-running-reviewer', { cli: 'claude', idleThresholdSecs: 0 }) +> ``` +> Pick the lowest value that comfortably exceeds your worst-case wait — `0` is safest but means a genuinely-hung agent will sit forever. +> +> 2. **Restructure to Per-turn interactive spawn.** For multi-turn flows, use the **Per-turn interactive spawn** shape from the [coordination table](#choose-your-coordination-style--conversation-vs-pipeline): one bounded step per turn, with a deterministic step in between carrying state on disk. You still get interactive PTY + channel. This only beats idle detection when **each turn finishes well inside the window** — a slow reasoning turn still needs `idleThresholdSecs` (option 1). Do not treat per-turn spawn as a substitute for raising the threshold on a turn that is itself quiet for >30s. +> +> The chat-native recipes below work as written when the next expected event reliably arrives well inside the idle window, or when you've extended the window per (1). +> +> Symptom to watch for in logs: a player exits with `Completion inferred from clean process exit (code 0) — no coordination signal was required` *before* the event it was supposed to wait for arrived; or a lead step that drives waits via shell `sleep` calls. + #### 1. Question / Answer (blocking ask) When agent A needs information only agent B has, instruct A to **post a direct question and wait for a reply** rather than guessing or proceeding. @@ -2230,6 +2267,9 @@ When you set `.pattern('supervisor')` (or `hub-spoke`, `fan-out`), the runner au |---------|-----| | Treating relay as transport, not as a coordination layer (every step is `preset: 'worker'`, every handoff is `{{steps.X.output}}`) | Default to **Conversation shape** for non-trivial work — interactive agents on a shared channel. Pipeline-shape is only correct when the work could be expressed as a `bash \| bash \| bash` pipe. | | Interactive agents on a channel whose task strings don't tell them to talk to each other | Pick a [Chat-Native Coordination Recipe](#chat-native-coordination-recipes) (Q/A, Broadcast/Ack, Peer Review, Standup, Hand-Off) and bake it into the task prompt — otherwise you're paying for a chat substrate you're not using | +| Task prompt that tells an interactive agent to wait for a later channel event | Idle detection (~30s) will complete the PTY before the event arrives. Set `idleThresholdSecs` (0 disables) or restructure to [Per-turn interactive spawn](#choose-your-coordination-style--conversation-vs-pipeline) | +| Using an LLM as a wait/grade oracle (`sleep 30` loops, "guess whether the secret matches") | If the comparison is computable (string match, test pass/fail, regex, schema validity), grade it in a **deterministic step**. LLMs should *produce*, not adjudicate fixed rules | +| Adding a repair step downstream of a gate that has no failure mode an agent can fix | "Workflows repair before they fail" applies when the gate has a real, agent-fixable failure surface. A deterministic gate downstream of deterministic work is correct by construction — a repair step there just burns a spawn cycle on a guaranteed no-op. Skip the repair, keep the hard gate | | All workflows run sequentially | Group independent workflows into parallel waves (4-7x speedup) | | Every step depends on the previous one | Only add `dependsOn` when there's a real data dependency | | Self-review step with no timeout | Set `timeout: 300_000` (5 min) — Codex hangs in non-interactive review | diff --git a/skills/writing-relayflows/SKILL.md b/skills/writing-relayflows/SKILL.md index 3df7474d..77671e6f 100644 --- a/skills/writing-relayflows/SKILL.md +++ b/skills/writing-relayflows/SKILL.md @@ -73,11 +73,11 @@ steps: model: claude-sonnet-4-6 ``` -**Version note.** This skill was refreshed against `@relayflows/surface@2.0.16` and `@relayflows/sdk@2.0.16` — the current published npm packages, not a source build. Both examples above pass `flows check` for real against those versions (see **Verified against**). `2.0.17` shipped since, fixing the `flows run --cloud` bug documented below (**Running it**) — that one claim is re-verified against `2.0.17`; everything else in this document was checked against `2.0.16` and hasn't been independently re-run on `2.0.17`, though nothing in its release notes suggests the other claims changed. +**Version note.** Every type in this skill is read from the published `@relayflows/surface@2.0.22` / `@relayflows/sdk@2.0.22` `.d.ts` files — the current npm `latest`, not a source build (see **Verified against**). Three things changed between `2.0.16` and `2.0.22`, and each is stated below at its shipped shape rather than at both: `AgentOptions.permissions` landed in `2.0.17`; predicate `.gate()` and the `artifact_exists` gate landed with [flows#449](https://github.com/AgentWorkforce/flows/pull/449); and `f.human` became executable. ## The real `Ctx` contract (TypeScript) -From the shipped `@relayflows/surface@2.0.16` `.d.ts` files (`dist/context.d.ts`, `dist/step.d.ts`, `dist/flow.d.ts`, `dist/completion.d.ts`): +From the shipped `@relayflows/surface@2.0.22` `.d.ts` files (`dist/context.d.ts`, `dist/step.d.ts`, `dist/flow.d.ts`, `dist/completion.d.ts`): ```ts export interface AgentResult { @@ -85,9 +85,16 @@ export interface AgentResult { artifacts: string[]; } +export interface PermissionsSpec { // exported from the package root + fileGlobs?: string[]; + networkAllowlist?: string[]; + accessPreset?: 'readonly' | 'readwrite'; +} + export interface AgentOptions { task: string; workspace?: string; + permissions?: PermissionsSpec; // since 2.0.17 — declared and journaled, NOT enforced cli?: string; model?: string; cwd?: string; // working dir for the CLI subprocess; defaults to the flow-runner's cwd @@ -106,7 +113,7 @@ export interface Ctx extends Helpers { // Helpers = f.slack, f.github, f.linea llm(strings: TemplateStringsArray, ...values: unknown[]): Step; llm(prompt: string, options: LlmOptions): Step; agent(name: string, options: AgentOptions): Step; - human(question: string, options: { to: string }): Promise; + human(question: string, options: { to: string }): Step; dispatch(flow: string, input: unknown): Promise; done(reason: FlowCompletionReason): void; cloud: CloudHelper; @@ -114,7 +121,9 @@ export interface Ctx extends Helpers { // Helpers = f.slack, f.github, f.linea } ``` -Do not add fields to `AgentOptions`/`Ctx` that aren't in this list — they don't exist in the shipped SDK. In particular: **no flow-level named-agent selector on `f.agent`, no `recoveryMode`/`permissions`/`surfaces`** on the TypeScript call site. Those are YAML/JSON step fields today. `FlowHeader` (`dist/flow.d.ts`) allows `use`, `identity`, `memory`, `budget`, `tools`, `workspace` — no bare `agents` map key (TypeScript composes reusable flows via `use: string[]` instead — see **Named agents and flow composition**). Passing an unknown header field or an unknown `AgentOptions` field throws a `TypeError` at authoring time, before anything is journaled. +Do not add fields to `AgentOptions`/`Ctx` that aren't in this list — they don't exist in the shipped SDK. In particular: **no flow-level named-agent selector on `f.agent`, no `recoveryMode`/`surfaces`/`output`** on the TypeScript call site. Those stay YAML/JSON step fields. `FlowHeader` (`dist/flow.d.ts`) allows `use`, `identity`, `memory`, `budget`, `tools`, `workspace` — no bare `agents` map key (TypeScript composes reusable flows via `use: string[]` instead — see **Named agents and flow composition**). An unknown *header* field throws a `TypeError` at authoring time, before anything is journaled; an unknown `AgentOptions` field is caught by `tsc`, not at runtime. + +**`permissions` is a real TypeScript option and it does not sandbox anything.** Since `2.0.17` it is on `AgentOptions`, validated by the SDK, lowered to `file_globs`/`network_allowlist`/`access_preset` and journaled with the step — and nothing reads it to gate a file or a network call. `flows check` warns `permissions_unenforced` on any declaration, including `{}`. Enforcement is [flows#442](https://github.com/AgentWorkforce/flows/issues/442). Declare it to record intent; do not rely on it to contain an agent. `run()`'s second argument is a real, current option: `{ timeout?: string | number }` — a duration string (`"15m"`) or milliseconds. Every real cookbook example that runs something slower than a few seconds sets this explicitly. @@ -124,9 +133,9 @@ Do not add fields to `AgentOptions`/`Ctx` that aren't in this list — they don' export const FLOW_COMPLETION_REASONS = ['success', 'step_failed', 'canceled', 'budget_exceeded', 'needs_human', 'declined'] as const; ``` -An authored body can meaningfully call `f.done('success')`, `f.done('step_failed')`, `f.done('needs_human')` (parks the run — it's not a kernel cancellation), or `f.done('declined')` (a deliberate decision not to act — also not a cancellation). `'canceled'` and `'budget_exceeded'` exist in the same closed set but are the **kernel's** to record; a body calling them itself is calling the wrong verdict for what actually happened. The verdict for a human declining a durable approval is `f.done('declined')`, not `f.done('canceled')` — a human declining is a decision, not the kernel calling off the run. (Correct verdict *if* the approval call reaches that point — `f.human` itself doesn't execute yet; see **Human approval and dispatch**.) +An authored body can meaningfully call `f.done('success')`, `f.done('step_failed')`, `f.done('needs_human')` (parks the run — it's not a kernel cancellation), or `f.done('declined')` (a deliberate decision not to act — also not a cancellation). `'canceled'` and `'budget_exceeded'` exist in the same closed set but are the **kernel's** to record; a body calling them itself is calling the wrong verdict for what actually happened. The verdict for a human declining a durable approval is `f.done('declined')`, not `f.done('canceled')` — a human declining is a decision, not the kernel calling off the run. -`Step` (`dist/step.d.ts`) is a `PromiseLike` with one extra method, `.gate(...)` — see **Verification gates**, which has a critical caveat for the predicate form. Every awaited step must actually be awaited — an unawaited or manually-`.then()`-chained step is refused (`unawaited_step` / a manual `.then()` is not an await), not silently dropped. +`Step` (`dist/step.d.ts`) is a `PromiseLike` with one extra method, `.gate(...)` — see **Verification gates**. Every awaited step must actually be awaited — an unawaited or manually-`.then()`-chained step is refused (`unawaited_step` / a manual `.then()` is not an await), not silently dropped. ## Verification gates @@ -137,6 +146,7 @@ Verification is control flow, not decoration — a gate decides whether a step a ```ts export type VerificationSpec = ExitCodeGate | OutputContainsGate | JsonSchemaGate | NamedDataGate; // NamedDataGate = ReferencesInputGate | SubprocessGate | WordCountBoundsGate | RegexMatchGate +// | ArtifactExistsGate ``` - `exit_code` — implicit default for `deterministic` steps. Not configurable; writing it explicitly is allowed and compiles to the same thing as omitting it. @@ -144,6 +154,7 @@ export type VerificationSpec = ExitCodeGate | OutputContainsGate | JsonSchemaGat - `json_schema` — step output validates against a JSON Schema (`boolean | Record`). Used for structured LLM/agent output; `output?: JsonOutputSchema` on an `llm`/`agent` step is sugar that compiles to this. - `subprocess_gate` — runs `command` under `/bin/sh`, judged on its exit code. The gate that actually works for "check a file the agent wrote": `{ type: 'subprocess_gate', command: 'test -s review.md' }`. If you also need to inspect the step's own output (via `from_output`), the SDK's generated wrapper reads its internal `FLOWS_INPUT` env var, extracts the value, and passes it to *your* command as `$INPUT` — `FLOWS_INPUT` itself is never visible to the author's command, only `$INPUT` is: `{ type: 'subprocess_gate', command: 'echo "$INPUT" | grep -q PASSED', from_output: ['summary'] }`. - `regex_match` — the step's output (or a value at `in_output_at`) matches `pattern`, evaluated by a non-backtracking RE2 engine (only `i`, `m`, `s` flags). +- `artifact_exists` — passes when the step's journaled `artifacts` list contains `path` (working-directory-relative POSIX). It reads the journal, not the disk, so it sees exactly what the worker measured at `step.completed`; a path the worker never journaled reads as missing even when the file is there. - `word_count_bounds`, `references_input` — narrower named gates; see `packages/sdk/src/named-gates.ts` in the flows repo for the exact contract. ```yaml @@ -159,32 +170,26 @@ export type VerificationSpec = ExitCodeGate | OutputContainsGate | JsonSchemaGat ### TypeScript (`Step.gate(...)`) — read this before you write one -`Step` has **two** overloads, and only one of them works today: +`Step` has **two** overloads, and **one `.gate()` per step** — a second throws `unsupported_gate`: ```ts gate(config: NamedGate): Step; -// NamedGate = ReferencesInputNamedGate | SubprocessNamedGate | WordCountBoundsNamedGate | RegexMatchNamedGate -// — the surface-visible subset of NamedDataGate above. This one is journal-honest and runs. +// NamedGate = ReferencesInputNamedGate | SubprocessNamedGate | WordCountBoundsNamedGate +// | RegexMatchNamedGate | ArtifactExistsNamedGate +// — the surface-visible subset of NamedDataGate above. Data, not code: `flows check` can prove it. gate(predicate: (value: T) => boolean, because?: string): Step; -// The type itself documents why this is dangerous — straight from the shipped .d.ts: -// "Predicate gates cannot be journaled — the JavaScript closure would not survive -// replay — so the executor refuses this branch with unsupported_gate." -``` - -**Verified for real, this session, against `flows@main` (post-v2.0.16, current as of 2026-09-17):** any flow using the predicate form — `.gate((out) => out.trim().length > 0, "some reason")` — fails at `flows run` time with: - -``` -FAILED [protocol_error] relayflowd could not complete the run request: unsupported_gate: -postfix .gate(predicate) closures cannot be journaled; use .gate({type: "…", …}) with a -slice-P named gate instead. +// Runs since flows#449. The executor calls the closure once, on the journaled value, and +// journals the VERDICT as a lowered `.gate` step — so resume and replay read the +// recorded verdict and never re-run the function. A false verdict fails the run as +// `gate_failed`, carrying `because`. ``` -This reproduced identically on two different real example flows (`examples/pr-review-pipeline`, `examples/dependency-upgrade-bot` in `AgentWorkforce/flows`) that both used the predicate form — neither got past their first gate. **Always use the config-object form**: +The predicate form was refused with `unsupported_gate` through `2.0.16`; on `2.0.22` it runs. **Prefer the config-object form anyway** — `flows check` cannot prove a predicate (docs/SURFACE.md §6), so a named gate is the only kind a preflight can tell you about before you spend a run: ```ts const outdated = await f.run('npm outdated --json 2>/dev/null || true') - .gate({ type: 'subprocess_gate', command: 'test -n "$INPUT"' }); // not .gate((out) => out.length > 0, '…') + .gate({ type: 'subprocess_gate', command: 'test -n "$INPUT"' }); // provable by `flows check` const review = await f.agent('review', { task: '…', cli: 'claude' }) .gate({ type: 'subprocess_gate', command: 'test -s review.md' }); // checks a file the agent wrote, ignores $INPUT @@ -192,9 +197,9 @@ const review = await f.agent('review', { task: '…', cli: 'claude' }) (Verified for real: `.gate({ type: 'subprocess_gate', command: 'test -n "$INPUT" && echo "$INPUT" | grep -q pkg' })` against a step outputting `{"pkg":"left-pad"}` passes — `$INPUT`, not `$FLOWS_INPUT`, is what the author's command sees. `FLOWS_INPUT` is an env var the SDK's own generated wrapper reads internally to extract the value before invoking your command; it is never visible to `command` itself.) -[flows#449](https://github.com/AgentWorkforce/flows/pull/449) (open at time of writing) adds journaled `artifact_exists` gates and real predicate-gate support (the executor runs the closure once, on the journaled value, and journals the verdict) — check whether it's merged before assuming predicate gates still don't work. Until it lands, `subprocess_gate`/`regex_match`/`word_count_bounds`/`references_input` are the only gates that actually run in TypeScript. +An agent step rarely fails by crashing; it fails by returning something plausible and wrong, which a plain retry-on-error never catches. Validate every `agent`/`llm` result, either directly with a supported gate or in a following deterministic verifier. -An agent step rarely fails by crashing; it fails by returning something plausible and wrong, which a plain retry-on-error never catches. Always give an `agent`/`llm` step a real, config-object gate — never the default, never a predicate. +**Where to put that check depends on what you want a red result to do.** A config-object or predicate gate directly on the agent/LLM step ends the run when it fails; use it when that failure should be terminal. For repairable failures, the following deterministic verifier must explicitly read the preceding output or workspace, produce its own verification result, and record it for repair. Output-dependent gates inspect the step they guard: moving the same gate unchanged would inspect the verifier's output, not the agent's. Use the record/repair/re-record/assert pattern in `relay-80-100-workflow`, gating the final verification result after repair. ## Helpers: journaled effects, not just Slack @@ -287,7 +292,7 @@ const planner = { cli: 'claude', model: 'claude-opus-5' }; await f.agent('plan', { ...planner, task: '…' }); ``` -What TypeScript *does* have, which the map doesn't give you, is `FlowHeader.use?: string[]` — "relative paths to reusable authored flows composed by this body" ([flows#300](https://github.com/AgentWorkforce/flows/issues/300), closed). This is a real field in the shipped `2.0.16` type; this skill hasn't independently run a flow that exercises it, so treat its exact runtime semantics as a pointer to verify against `docs/SURFACE.md` and the flows repo, not as tested guidance the way the rest of this document is. +What TypeScript *does* have, which the map doesn't give you, is `FlowHeader.use?: string[]` — "relative paths to reusable authored flows composed by this body" ([flows#300](https://github.com/AgentWorkforce/flows/issues/300), closed). This is a real field in the shipped `2.0.22` type; this skill hasn't independently run a flow that exercises it, so treat its exact runtime semantics as a pointer to verify against `docs/SURFACE.md` and the flows repo, not as tested guidance the way the rest of this document is. ## Parallel agents @@ -322,30 +327,25 @@ What this actually is, per `docs/AGENT-RELAY-TRANSPORT.md`: the step calls `POST Don't describe this as "agents talking to each other" — there's no documented pattern in this skill's scope for two flow steps to hold a live back-and-forth over Relay channels while both are running. What's real: the step's agent becomes a genuine Agent Relay workspace participant while it runs (so a human, or another agent with the right access, can DM it and potentially steer it — per the type's own wording), and the flow still only sees a request-and-durable-receipt shape, not an open channel. Multi-agent *coordination* inside a flow today means sequential handoff (one step's return value feeds the next) or parallel fan-out (**Parallel agents**, above) — not live messaging. -## Human approval and dispatch (declared, not yet runnable) - -`Ctx` declares `human(question, {to})` and `dispatch(flow, input)`, and both typecheck and pass `flows check`. **Neither actually runs today.** Verified for real against `@relayflows/surface@2.0.16`: +## Human approval and dispatch -``` -$ flows run human.flow.ts --local-agent --input '{}' -FAILED [protocol_error] relayflowd could not complete the run request: -unsupported_verb: the initial authored executor does not lower f.human - -$ flows run dispatch.flow.ts --local-agent --input '{}' -FAILED [protocol_error] relayflowd could not complete the run request: -unsupported_verb: the initial authored executor does not lower f.dispatch -``` +`Ctx` declares `human(question, {to})` and `dispatch(flow, input)`. **`f.human` runs on `2.0.22`; `f.dispatch` still does not** — the executor throws `unsupported_verb: the initial authored executor does not lower f.dispatch`. Both typecheck and pass `flows check`, so `flows check` green tells you nothing about which is which. -Type availability does not establish executable support for either verb — the shipped executor throws `unsupported_verb` for both, identically, regardless of what the rest of the flow does. [flows#400](https://github.com/AgentWorkforce/flows/issues/400) tracks wiring up durable human approval; there's no dedicated tracking issue found yet for `f.dispatch`. Until one of these lands, treat both as authoring-surface-only: +`f.human` parks the run durably on a kernel `wait.human`, keyed `human-` in the order the body asked. A person answers, and `flows resume` continues the body from that line: ```ts -// This typechecks and passes `flows check` — it will FAIL at `flows run` time. -// Do not ship a flow whose only path to done() runs through f.human or f.dispatch. const ok = await f.human(`Ship this?\n${plan.summary}`, { to: 'khaliq' }); -if (!ok) return f.done('declined'); // correct verdict, if this call ever executes +if (!ok) return f.done('declined'); // a person's "no" is declined, never canceled +``` + +``` +$ flows answer human-1 yes --by khaliq --note 'reviewed the diff' +$ flows resume ``` -If you need durable approval today, do it in YAML with `permissions`/`recoveryMode: manual` (parks the run as `needs_human` with a diff for a person to resolve), or split the decision out to a separate, human-triggered flow run rather than a mid-flow `f.human` call. +`to` names who is asked and is recorded with the question — **it is not a delivery address**; nothing notifies that person for you. The kernel closes a wait once, so the first answer wins, and it records `attribution: "client_asserted"` because the daemon socket, not the kernel, authenticated whoever `--by` names. The answer step takes a `.gate(config)` like any other step. One caveat: `f.human` needs a durable root run to park in — under a runner that has none it refuses with `unsupported_verb`, so run it with `flows run`. + +For `f.dispatch`, split the work into a separate, independently triggered flow run. Do not ship a flow whose only path to `done()` goes through it. **The same "types accept it, the executor doesn't" trap applies to `workspace` on an agent step under `--local-agent`.** Neither an annotated (`'acme/api: readonly'`) nor a bare (`'acme/api'`) value works — both fail identically: @@ -354,7 +354,7 @@ REFUSED [invalid_spec] unsupported_workspace_permission: The local agent worker stream-only steps. Remove workspace or attach a worker that holds its revision pins. ``` -It isn't the `: readonly`/`: readwrite` annotation syntax that's rejected — it's that the local-agent worker doesn't hold workspace revision pins at all. Omit `workspace` entirely for a step you intend to run with `--local-agent`; it's a real, working field only against a worker that supports it (Cloud's). +It isn't the `: readonly`/`: readwrite` annotation syntax that's rejected — it's that the local-agent worker doesn't hold workspace revision pins at all. Omit `workspace` entirely for a step you intend to run with `--local-agent`; it's a real, working field only against a worker that supports it (Cloud's). Against a worker that does hold pins, the suffix is separately refused, and the remedy the SDK's own message names is `f.agent`'s `permissions` option — which records the scope and does not enforce it. ## Running it: `flows check` / `run` / `deploy` / `schedule` @@ -404,7 +404,7 @@ Both are real, both are exit 2 — the same underlying problem can print a diffe ## Common mistakes - **Forgetting `version` in a YAML/JSON `FlowSpec`.** Required, not optional. -- **Using a predicate `.gate((v) => …, reason)` in TypeScript.** Refused at runtime (`unsupported_gate`) until [flows#449](https://github.com/AgentWorkforce/flows/pull/449) lands. Use a config-object gate — see **Verification gates**. +- **Attaching two `.gate()` calls to one step.** `unsupported_gate`: a step takes one. Also note a predicate gate is invisible to `flows check` — prefer a config object when preflight should be able to prove it. - **Putting `tools`, `budget`, or any other `FlowHeader` field into `flows.json`.** `flows.json` only accepts `cli`, `executors`, `models`, `mcp`, `deploy` — anything else is `config_invalid`. - **Assuming `flows.json`'s `models` sets a default model.** It only validates models already declared elsewhere; it never selects one. - **Declaring `.on(...)` trigger handlers without registering the provider in `flows.json`'s `executors`.** Even a flow you only ever run directly (never via a real webhook) needs this, or it's refused `no_executor` before your input is even read. @@ -419,7 +419,7 @@ Both are real, both are exit 2 — the same underlying problem can print a diffe - **Triggers/webhooks** (`.on(github.pull_request(...), ...)`), **memory retrieval**, and the full **helper** catalog's per-provider quirks (`f.mcp`, provider-specific settings) — each is its own surface; see the [Relayflows product docs](https://agentrelay.com/docs/relayflows) and the [flows-cookbook](https://github.com/AgentWorkforce/flows-cookbook) for real, run-verified examples of each. - **Named-agent map equivalent in TypeScript beyond `use:`** — `FlowHeader.use` exists in the shipped type but this skill hasn't independently run a flow that exercises it; verify current semantics against `docs/SURFACE.md` before relying on this section alone. -- **YAML-only agent step fields with no TypeScript equivalent**: `recoveryMode`, `permissions`, `surfaces`, and enforced `workspace` (TypeScript's `workspace` string is accepted by the type but rejected by the local-agent worker — see **Human approval and dispatch**). Author that step in YAML if you need them; `f.dispatch` is not a working escape hatch for this today (see below). +- **YAML-only agent step fields with no TypeScript equivalent**: `recoveryMode`, `surfaces`, `output`, the `agent:` named-agent selector, and enforced `workspace` (TypeScript's `workspace` string is accepted by the type but rejected by the local-agent worker — see **Human approval and dispatch**). Author that step in YAML if you need them; `f.dispatch` is not a working escape hatch for this today (see below). `permissions` is **not** on this list — it is a TypeScript option too (see **The real `Ctx` contract**). - The **older `@relayflows/core` `WorkflowBuilder`** engine — see `writing-agent-relay-workflows` and `migrating-persona-to-relayflow` in this repo. ## Quick reference @@ -429,10 +429,10 @@ Both are real, both are exit 2 — the same underlying problem can print a diffe | `f.run(command, {timeout?})` / `type: deterministic` | both | shell command, implicit `exit_code` gate | | `f.llm(...)` / `type: llm` | both | bare model call, no workspace | | `f.agent(name, opts)` / `type: agent` | both | harnessed coding agent, returns `{summary, artifacts}` | -| `f.human(question, {to})` | TS only | **not executable yet** — `unsupported_verb`; typechecks, fails at `flows run`. [flows#400](https://github.com/AgentWorkforce/flows/issues/400) | -| `f.dispatch(flow, input)` | TS only | **not executable yet** — `unsupported_verb`; typechecks, fails at `flows run` | +| `f.human(question, {to})` | TS only | parks the run on a durable `wait.human`; `flows answer human- yes\|no` then `flows resume` | +| `f.dispatch(flow, input)` | TS only | **not executable** — `unsupported_verb`; typechecks and passes `flows check`, fails at `flows run` | | `f.done(reason)` | TS / kernel | one of `success \| step_failed \| needs_human \| declined` from an authored body; `canceled \| budget_exceeded` are kernel-only | -| `.gate({type: '...', ...})` | TS | the only `.gate()` form that runs today — config object, never a predicate | +| `.gate(config)` / `.gate(predicate, because)` | TS | one per step; both run since flows#449, but only a config object is provable by `flows check` | | `options.transport: 'relay'` | TS/YAML `agent` step | dispatch to Agent Relay's task infra; request+receipt, not live chat | | `Promise.all([...steps])` | TS | first-class supported concurrent fan-out | | `flows.json`: `cli, executors, models, mcp, deploy` | project config | exact accepted key set — nothing else | @@ -444,6 +444,8 @@ Both are real, both are exit 2 — the same underlying problem can print a diffe ## Verified against -`@relayflows/surface@2.0.16` and `@relayflows/sdk@2.0.16` (the published npm packages) — types read directly from the shipped `.d.ts` files, not from documentation or memory. Beyond static type-checking, the claims in this refresh that carry a **real run**, not just `flows check`, are backed by the recipes in [`AgentWorkforce/flows-cookbook`](https://github.com/AgentWorkforce/flows-cookbook) — each recipe's own README states exactly what was run, when, and what the result was (a real local run, a real Cloud deploy, or both), rather than duplicating that evidence here where it will go stale. If a claim in this skill and a cookbook recipe's README disagree, trust whichever was verified more recently — check the README's date. +**`@relayflows/surface@2.0.22` and `@relayflows/sdk@2.0.22`** — the current npm `latest` (there is no `2.0.23`). Every type, union and option in this skill was read directly from those packages' shipped `.d.ts` and `dist/*.js`, not from documentation or memory. + +This revision resolved a set of contradictions left by an earlier refresh that was verified at `2.0.16` and then merged with notes taken at `2.0.22`. Each was settled against the published package, and the losing side was deleted rather than hedged: `AgentOptions.permissions` exists on the TypeScript call site (added in `2.0.17`, validated and journaled, not enforced — preflight warns `permissions_unenforced`); `run(command, options?)` takes a second `{timeout}` argument; `f.done` takes the six `FLOW_COMPLETION_REASONS`; there is no `budget.maxWallclockMs` (`FlowHeader.budget` is `string | {tokens?, dollars?, wallclock?}`); predicate `.gate()` and the `artifact_exists` gate both ship; and `f.human` executes while `f.dispatch` still does not. Beyond static type-checking, the claims in this refresh that carry a **real run**, not just `flows check`, are backed by the recipes in [`AgentWorkforce/flows-cookbook`](https://github.com/AgentWorkforce/flows-cookbook) — each recipe's own README states exactly what was run, when, and what the result was (a real local run, a real Cloud deploy, or both), rather than duplicating that evidence here where it will go stale. If a claim in this skill and a cookbook recipe's README disagree, trust whichever was verified more recently — check the README's date. -The predicate-gate failure, the `flows.json` schema refusal, the `--data-dir` git-hygiene interaction, and the `flows run --cloud` bug were all reproduced directly while building that cookbook, on `AgentWorkforce/flows@main` past the `v2.0.16` tag (2026-09-17). A subsequent review pass (Devin, on this PR) flagged three more claims worth checking rather than trusting on sight — `f.human`/`f.dispatch` executability, the `workspace: readonly` annotation, and the `subprocess_gate` env var name — all three were independently re-run against the real CLI, confirmed as real breaks (one of the review's own suggested fixes, for `workspace`, turned out not to work either — the bare form fails identically to the annotated one), and corrected in the sections above rather than taken on the reviewer's word alone. +The `flows.json` schema refusal, the `--data-dir` git-hygiene interaction, and the `flows run --cloud` bug were all reproduced directly while building that cookbook, on `AgentWorkforce/flows@main` past the `v2.0.16` tag (2026-09-17); they are unchanged in `2.0.22`'s shipped code. The predicate-gate refusal reproduced there too — and it is the one claim `2.0.22` overturned, which is why the section above now documents the working form instead. An earlier review pass (Devin, on this PR) flagged three more claims worth checking rather than trusting on sight — `f.human`/`f.dispatch` executability, the `workspace: readonly` annotation, and the `subprocess_gate` env var name. The last two were re-run against the real CLI and confirmed as real breaks, still present in `2.0.22` (one of the review's own suggested fixes, for `workspace`, turned out not to work either — the bare form fails identically to the annotated one). The first has since split: `f.human` executes on `2.0.22`, `f.dispatch` does not.