Skip to content

skill(orchestrating-agent-relay): subscribe live PR owners on Agent Relay 12.4 - #108

Closed
khaliqgant wants to merge 3 commits into
mainfrom
docs/orchestrating-agent-relay-12.4-pr-owner-subscribe
Closed

khaliqgant wants to merge 3 commits into
mainfrom
docs/orchestrating-agent-relay-12.4-pr-owner-subscribe

Conversation

@khaliqgant

@khaliqgant khaliqgant commented Sep 21, 2026 •

Copy link
Copy Markdown
Member

Summary

Audit skills/orchestrating-agent-relay/SKILL.md against the Agent Relay 12.4 CLI (agent-relay --version → 12.4.0) and make this the durable orchestrator default:

The moment a worker opens or owns a PR, subscribe that live PR-owner agent to the PR's GitHub provider resource so review comments, pushes, check changes, and discussion wake it:

agent-relay integration subscribe github \
  --resource '/github/repos/<owner>/<repo>/pulls/<n>/**' \
  --to @<PR-owner-agent>

Why this belongs in the orchestrator skill

Provider wakes are not a workflow-YAML concern here. Headless teams that ship PRs with node/fleet workers currently poll inbox/gh and miss review traffic while idle. 12.4 integration subscribe provisions a Relay inbound webhook, a Relayfile path_glob subscription, and a binding keyed by (provider, resolved path glob). That is the orchestrator's job the instant a PR number exists.

What the skill now requires

  • Subscribe immediately on PR open, targeting the live owner identity (--to @Owner), not a helper and not a shared channel.
  • Rebind after respawn. Bindings follow recipient identity. A new name (Implementer2, --delete-agent) does not inherit the old route; re-subscribe. Same-name release without delete keeps the binding — confirm --list still points at @Owner.
  • No duplicate writable subscriptions on helpers. A second subscribe to the same (github, glob) replaces the owner's route (create-first). Reviewers read via git/gh or channel traffic.
  • Unsubscribe/release hygiene. Unsubscribe owned resources, verify webhookId / webhookSubscriptionId / binding disappeared, then fleet release. Do not tear down a working binding before a replacement lands.
  • Verify delivery, not binding creation. --list + webhook inventory + live fleet agent list are necessary but not sufficient. A real GitHub event must arrive as a Relay message from github with a Relayfile path under that glob (check_inbox / list_messages). node deadletters must not hold it. Cloud/Nango 2xx is not proof.

12.4 CLI corrections (validated against --help)

  • integration subscribe / unsubscribe / subscribe --list are the provider-binding surface. integration subscription list is a different Relay workspace-event API.
  • integration webhook create takes <channel> (inbound). The older <url> --event shape in public docs is stale.
  • fleet spawn / fleet release / fleet agent list are the control-plane lifecycle commands; node agent * remains local-broker.
  • Spawn providers on 12.4 include muse, grok, opencode, devin (plus the existing CLIs). droid stays high-risk for delegation.
  • node deadletters is the failed-delivery inspection path.

Relationship to PR #56

Inspected #56 (skill(writing-agent-relay-workflows): lessons from multi-CLI game test).

That PR edits writing-agent-relay-workflows, not this skill. After the 12.4 audit:

#56 change Verdict
Per-turn interactive spawn row Useful, wrong home for orchestrating. Salvaged into writing-agent-relay-workflows (still absent on main 1.6.24).
Idle-detection (~30s) vs wait-for-event, idleThresholdSecs Useful, wrong home. Salvaged there. Confirmed idleThresholdSecs exists on @relayflows/core WorkflowBuilder.
LLM-as-oracle anti-pattern Useful, wrong home. Salvaged as a Common Mistakes row.
Repair-step counterweight Useful, wrong home. Salvaged next to "Workflows repair before they fail."
Everything else in #56 Stale/duplicative of later 1.6.x work already on main (conversation vs pipeline, chat-native recipes, repairable gates).

This replacement PR therefore supersedes #56: unique surviving lessons are preserved in writing-agent-relay-workflows 1.6.25; the orchestrating skill is the 12.4 operational surface #56 never covered.

Test plan

  • Read canonical orchestrating-agent-relay skill end-to-end.
  • Audit agent-relay 12.4 --help for node, fleet (spawn/list/release/agent list), integration (subscribe/unsubscribe/webhook/subscription), observer, message, agent.
  • Cross-check changelog 12.0–12.4, provider-subscriptions.mdx, and tests/e2e/github-subscriptions/README.md for binding/replacement/delivery semantics.
  • Validate every newly documented command group with --help (all OK).
  • prpm.json parses; package files exist; descriptions ≤500 chars; no duplicate package names.
  • Inspected PR skill(writing-agent-relay-workflows): lessons from multi-CLI game test #56 diff/comments; salvaged only still-valid workflow lessons.
  • After this PR exists: subscribe @skills-orchestration-audit-grok to /github/repos/AgentWorkforce/skills/pulls/<this>/** and verify webhook binding + delivery evidence.
  • Close skill(writing-agent-relay-workflows): lessons from multi-CLI game test #56 as superseded with a link to this PR.

Note

Low Risk
Documentation and skill version bumps only; no runtime or application code changes.

Overview
Bumps orchestrating-agent-relay to 2.4.0 (and package metadata/README) to document Agent Relay 12.4 headless orchestration: when a worker owns a PR, subscribe that live agent via integration subscribe github to four Relayfile globs (PR body, reviews, status, issue comments), with rebinding after respawn, helper/no-duplicate-binding rules, unsubscribe-before-release hygiene, and delivery verification (real GitHub events in inbox, not just --list).

The skill also expands quick reference and protocol for fleet agent list / fleet release, node deadletters, corrected integration CLI surfaces (subscribe --list vs subscription list, inbound webhook create), and extra spawn CLIs; release flow now requires unsubscribing owned resources first.

writing-agent-relay-workflows goes to 1.6.25 with workflow-authoring fixes salvaged from superseded work: when not to add repair steps, per-turn interactive spawn vs long waits, idleThresholdSecs / ~30s idle detection vs “wait for channel event”, and new common-mistake rows (LLM-as-oracle, pointless repair gates).

Reviewed by Cursor Bugbot for commit 1967690. Bugbot is set up for automated code reviews on this repo. Configure here.

Review in cubic


Agent Relay sessions

  • claude session a32d0191 · contributor · ran gh pr commands · last active 2026-09-21

Audit the orchestrating skill against Agent Relay 12.4 and make
subscribing the live PR-owner agent to the PR's GitHub provider
resource the durable default. Cover rebinding after respawn,
unsubscribe/release hygiene, no duplicate writable helper
subscriptions, and verifying actual event delivery.

Salvage still-valid #56 workflow lessons (idle detection,
per-turn interactive spawn, LLM-as-oracle, repair-step
counterweight) into writing-agent-relay-workflows.
@coderabbitai

coderabbitai Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Advanced

Run ID: aac672a3-7291-4e65-a11d-2da5746ca783


Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 21, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-21T00:49:43.432389Z f0ecd77 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@devin-ai-integration devin-ai-integration Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Devin Review found 2 potential issues.

Devin Review

Comment thread skills/orchestrating-agent-relay/SKILL.md Outdated
Comment thread skills/writing-agent-relay-workflows/SKILL.md Outdated
@khaliqgant

Copy link
Copy Markdown
Member Author

[relay-subscribe-probe] Wake check for @skills-orchestration-audit-grok on /github/repos/AgentWorkforce/skills/pulls/108/** — please ignore; verifying provider event delivery.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f0ecd77a76

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread skills/orchestrating-agent-relay/SKILL.md Outdated
…12.4 delivery

The PR glob wakes events whose resource_ref is that PR even when the
inventory file lives under /reviews or /comments. Per-turn interactive
spawn only beats idle detection when each turn itself finishes inside
the window.
…s, checks, comments

Codex P1: the single /pulls/<n>/** command did not match the promised
wake set. Subscribe the owner to the adapter nested reviews/status
globs plus GitHub issue-comment paths as well.
khaliqgant added a commit that referenced this pull request Sep 21, 2026
Folds the #108 work into this branch so the two land as one PR.
No file overlap between the two SKILL.md sets; README/prpm resolved
to keep this branch's relay-80-100-workflow 2.0.0 rewrite alongside
#108's orchestrating-agent-relay 2.4.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
khaliqgant added a commit that referenced this pull request Sep 21, 2026
…ant lessons

Retarget pass over the folded-in #108 content plus the open review
findings on this PR.

Deprecation (v1 -> v2):
- writing-agent-relay-workflows 1.6.25 -> 2.0.0. DEPRECATED banner at the
  top of SKILL.md and a rewritten frontmatter description, so a skill
  listing shows the deprecation and the pointer to writing-relayflows
  without opening the file. All v1 reference material kept intact below.
- migrating-persona-to-relayflow 1.0.0 -> 1.1.0. Same short pointer, and
  corrects the stale "v2 is still under construction" bullet — v2 ships
  today as @relayflows/surface / @relayflows/sdk, CLI `flows`.
- agent-relay-starter collection 1.1.0 -> 1.2.0: writing-relayflows is now
  the required workflow-authoring skill; the v1 skill drops to optional.

Retargeted out of the v1 skill into relay-80-100-workflow 2.0.0 -> 2.1.0
(the v2 validation skill), because both lessons are about the ladder, not
about either engine's syntax:
- do not add a repair step to a gate with no agent-fixable failure surface
- grade computable things in a deterministic step, never with an LLM
Both also get an anti-pattern row. The idle-detection / per-turn spawn
material stayed in the v1 skill: idleThresholdSecs, preset and interactive
channel agents have no v2 equivalent.

Review findings resolved in relay-80-100-workflow:
- "None of that is reachable by a gate" was too categorical; the residue
  gate later in the skill shows the opposite
- the re-record example used `/* same command */`, which does not compile;
  the command is now a named const reused by both record calls
- require-green now writes BLOCKED_NO_COMMIT before its non-zero exit, so
  the artifact the skill requires actually exists when acceptance is red
- `npx vitest run` -> `npx --no vitest run`, with a note to run tools from
  the project's own lockfile (CWE-494)

writing-relayflows 1.2.0 -> 1.3.0: reconciles "always gate an agent step"
with relay-80-100-workflow's "enforce after the agent, not on it" — the
two skills contradicted each other. Where the check goes now depends on
whether a red result should stop the run or become repairable work.

Root agent-workforce-skills 1.5.0 -> 1.6.0.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@khaliqgant

Copy link
Copy Markdown
Member Author

Folded into #107 — closing this one

Combining the two into a single PR, per the user. This branch is merged into feat/relayflows-v2-authoring-friction at f41e050, so nothing here is lost: all three commits (f0ecd77, 8f0f45f, 1967690), the orchestrating-agent-relay 12.4 work in full, and the review threads with their live binding evidence are all in #107's history.

Why #107 was the base rather than this one. It is the larger piece (relay-80-100-workflow rewritten v2-native, 675 → 309 lines) and had already merged main at 0ac4206 to take main's writing-relayflows refresh. The two touch no SKILL.md in common — this PR has orchestrating-agent-relay + writing-agent-relay-workflows, #107 has relay-80-100-workflow — so the only conflict was two adjacent rows in the README table.

What carried over unchanged. Everything in skills/orchestrating-agent-relay/SKILL.md: the four-glob PR-owner subscribe recipe, rebinding after respawn, the no-duplicate-writable-helper rule, unsubscribe/release hygiene, verify-delivery-not-binding-creation, and the 12.4 CLI corrections. That is relay operational surface, not engine-specific, and it was in the right skill already.

What changed about the writing-agent-relay-workflows additions. This PR's own framing said those were #56 lessons salvaged into the workflow skill, and the retarget review agreed with most of that:

  • The idle-detection block, idleThresholdSecs, and the per-turn interactive spawn row are v1-only and stayed exactly where you put them. v2 has no interactive PTY agents sharing a channel — there is no idleThresholdSecs anywhere in writing-relayflows, and v2 coordination is sequential handoff or parallel fan-out, never live messaging. Moving them would have made them wrong.
  • The repair-step counterweight and the LLM-as-oracle anti-pattern are about the record → repair → re-record ladder rather than about either engine's syntax, and were missing on the v2 side. Both moved into relay-80-100-workflow (the v2 validation skill, not writing-relayflows — they are validation-shape rules, not authoring surface), each with an anti-pattern row.

writing-agent-relay-workflows is now marked DEPRECATED at the top of its SKILL.md, with the notice repeated in its frontmatter description, its prpm.json description, and its README row, since that description is what an agent selects a skill on. All the v1 reference material — including your additions — stays intact below the notice for anyone maintaining an existing v1 workflow.

Review threads on this PR. All three (4058630190, 4058630211, 4058635595) were already answered and fixed here before the fold, with live webhook/binding ids as evidence. Those fixes are in the commits merged into #107; nothing is outstanding.

Follow #107 from here. Neither PR is being merged — that call is the user's.

🤖 Generated with Claude Code

@khaliqgant khaliqgant closed this Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant