diff --git a/docs/plans/321-the-loop-recovers-from-ci.md b/docs/plans/321-the-loop-recovers-from-ci.md index 37c49379..1779f783 100644 --- a/docs/plans/321-the-loop-recovers-from-ci.md +++ b/docs/plans/321-the-loop-recovers-from-ci.md @@ -1,6 +1,6 @@ # Plan 321 โ€” The loop recovers from CI -**Status:** ๐Ÿ“‹ planned โ€” nothing built; start at P0 +**Status:** ๐Ÿ“‹ P0 measured (ยง5); P1 is next โ€” nothing built **Builds on:** Plan 315 (the loop: commission โ†’ review โ†’ publish; `mergeFrom`; the push posture), the catalog's `pr-shepherd` skill (the hosted code lane's shape of the same job), the scheduler (`agent.schedule.create`), and the `github` connector. @@ -14,16 +14,16 @@ What the hosted `pr-shepherd` already settles: one invocation is describe โ†’ re ## 2. Decisions -**D1 โ€” The shepherd is a skill, run by the Runner, and every decision it makes is read from data.** `local-pr-shepherd` is a catalog skill on the host lane: it finds the pull requests the loop opened, reads each one's live state, decides `fix | wait | abandon | leave`, and for `fix` commissions the coding agent on the pull request's branch with `base: ` and `mergeFrom: origin/` and publishes the result appended, through Local Publish, under the folder's posture. No step of it is a Helmsman conversation. _Rejected:_ a Helmsman prompt that says "check the pull requests" โ€” a cold session with prose instructions is the shape that commissions duplicate work; what a cold run needs is the state as inputs to its tasks (D3), not a paragraph about being careful. +**D1 โ€” The shepherd is a skill, run by the Runner, started by an agent that decides nothing, and every decision it makes is read from data.** A schedule does not start a skill: it starts an agent run, and the skill runs because that agent calls `workflow.run.start` (ยง5 P0.1). A schedule carries only a persistent target, `platform-role` or `custom-agent`, never an inline agent (`apps/aflow-orchestrator/src/services/SessionOrchestrator/handlers/inlineOps/scheduleCrud.ts:388`), and the platform agent that holds `workflow.run.start`, the Helmsman (`cybernetic-helmsman`), is a model acting on a template โ€” the prompt rejected below. The agent that decides nothing is therefore a custom agent (`custom-agent`, `packages/schemas/src/runtime/agentTarget.ts`): a step-graph agent, of the kind `agent.manage.create` writes, whose definition is one operation step โ€” `workflow.run.start` of `local-pr-shepherd`, with the skill's slug and the folder binding, `bindingId`, as literal values โ€” and no model turn at all: no `ai.agent.turn`, no prompt, nothing to drift. The step's inputs are fixed when the agent is installed โ€” no `$memoryRef`, which fails the firing, and nothing derived at fire time, which a template cannot carry. The Local PR Shepherd bundle installs the agent beside the skill, under its slug, so the operator creates nothing by hand, and the schedule targets that agent by its slug: the target it stores is `{ kind: 'custom-agent', agentId }`, the id of the agent the slug names. `local-pr-shepherd` is a catalog skill on the host lane: it finds the pull requests the loop opened, reads each one's live state, decides `fix | wait | abandon | leave`, and for `fix` commissions the coding agent on the pull request's branch with `base: ` and `mergeFrom: origin/` and publishes the result appended, through Local Publish, under the folder's posture. No step of the skill is a Helmsman conversation, and the agent's run that starts it reads nothing about pull requests. _Rejected:_ a Helmsman prompt that says "check the pull requests" โ€” a cold session with prose instructions is the shape that commissions duplicate work; what a cold run needs is the state as inputs to its tasks (D3), not a paragraph about being careful. -**D2 โ€” The trigger is a schedule on the local edition, and a webhook later where there is a public address.** A recurring schedule (`agent.schedule.create`, `cron`) fires the skill every few minutes while the executor is paired; the interval is an operator knob with a default measured in P0 against GitHub's rate budget, not a constant chosen here. The firing carries nothing but the connected folder: the skill discovers its pull requests itself. The appliance's own webhook ingest is the hosted edition's trigger, where `api.aflow.ai` can receive GitHub's `check_suite` events; a laptop cannot, so it polls. A recurring schedule's `maxFirings` is finite by design; the skill's last task renews the schedule when it is near its cap, so the shepherd does not expire silently. +**D2 โ€” The trigger is a schedule on the local edition, and a webhook later where there is a public address.** A recurring schedule (`agent.schedule.create`, `action: 'start_run'`, `cron`) starts a run of the shepherd's custom agent every five minutes while the executor is paired, and that agent's one operation step starts the skill (D1); the interval is an operator knob whose default, `*/5 * * * *`, is derived from GitHub's rate budget in ยง5 P0.2. The schedule's target is the custom agent the Local PR Shepherd bundle installs, named by its slug; its template is static JSON, checked at creation only for non-emptiness, and the step reads none of it โ€” the skill's slug and `bindingId` are literals in the agent's definition. The firing carries nothing else: the skill discovers its pull requests itself, because a template carries no live value and no cursor across firings. The appliance's own webhook ingest is the hosted edition's trigger, where `api.aflow.ai` can receive GitHub's `check_suite` events; a laptop cannot, so it polls. A recurring schedule's `maxFirings` is finite by design; the skill's last task renews the schedule when it is near its cap, with the same target and template, so the shepherd does not expire silently. **D3 โ€” A cold run reads four things before it acts, and acts on none it did not read.** 1. **The pull requests that are its to tend**: open pull requests whose head branch is under the folder's publish prefix and whose body carries Local Publish's marker. A pull request a person opened, or one labelled `hands-off`, is left alone. 2. **The head's checks**: `listCheckRuns` on the head sha. The decision reads the conclusions, not the names. 3. **The handoff**: one comment per pull request, ``, carrying `attemptCount`, the `checkRunId` of the last failure acted on, the head sha it acted on, and the run id of the fix it commissioned. Written by the shepherd alone. -4. **The space's in-flight work on that branch**: a lease in the memory store, `shepherd///`, taken for the duration of a fix and released at its end, holding the run id and the time; and the space's attention list (`workflow.run.list_attention`) for a paused publication or commission naming that branch. A branch with a live lease or a paused run on it is `wait`, whoever started that run โ€” a person's conversation included. +4. **The space's in-flight work on that branch**: a lease in the memory store, `shepherd///`, taken for the duration of a fix and released at its end, holding the run id and the time; and a bounded read of the space's runs by workflow and input field โ€” `workflow.run.list`, space-wide and one page at most, for the non-terminal runs, paused included, of `publish-local-changes` whose `branch` input is the branch and of `commission-change` whose `base` input is the branch. The attention list cannot answer this: its items name a run, never a branch (ยง5 P0.4). A branch with a live lease or a non-terminal run on it is `wait`, whoever started that run โ€” a person's conversation included. The four reads are tasks with typed outputs, and the deciding step receives them as its inputs. A read that fails ends the run as `wait` with the reason, never as a guess. @@ -39,22 +39,22 @@ The four reads are tasks with typed outputs, and the deciding step receives them ### 3.1 Tasks -| Task | Operation | Reads / does | -| ---------------- | ------------------------------------------------------------------------ | ------------------------------------------------------------------------------------------- | -| `read-folder` | `host.binding.inspect` | the branch prefix, the posture, `origin` | -| `list-prs` | `api.http.call` `listPullRequests` | open pull requests, filtered to the prefix and the marker, minus `hands-off` | -| `read-pr` | per pull request: `getPullRequest`, `listCheckRuns`, `listIssueComments` | head sha, checks, the handoff | -| `read-in-flight` | `memory.store.get`, `workflow.run.list_attention` | the lease and any paused run on the branch | -| `decide` | `ai.generate_json` or a pure rule | `fix \| wait \| abandon \| leave`, with the reason; a rule where the inputs decide it alone | -| `read-failure` | `listWorkflowRunJobs`, `getJobLogs` | failing jobs and bounded log excerpts, as a payload | -| `lease` | `memory.store.put` | the lease, written before the commission | -| `fix` | `host.harness.run` | `base: `, `mergeFrom: origin/`, the assembled task | -| `publish` | `workflow.run.start` Local Publish | appended onto the branch, `wait: 'until_complete'` | -| `handoff` | `createIssueComment` / edit | the updated record, every pass | -| `release` | `memory.store.delete` | the lease, on every terminal path | -| `renew` | `agent.schedule.create` | the next schedule when this one nears `maxFirings` | - -Whether `decide` needs a model at all is a P0 question: the inputs above decide the four outcomes without one, and a model is reached for only to read a log excerpt a rule cannot. +| Task | Operation | Reads / does | +| ---------------- | ------------------------------------------------------------------------ | -------------------------------------------------------------------------------------- | +| `read-folder` | `host.binding.inspect` | the branch prefix, the posture, `origin` | +| `list-prs` | `api.http.call` `listPullRequests` | open pull requests, filtered to the prefix and the marker, minus `hands-off` | +| `read-pr` | per pull request: `getPullRequest`, `listCheckRuns`, `listIssueComments` | head sha, checks, the handoff | +| `read-in-flight` | `memory.store.get`, `workflow.run.list` | the lease and the non-terminal runs, paused included, whose branch input is the branch | +| `decide` | a pure rule; `ai.generate_json` for a log excerpt only | `fix \| wait \| abandon \| leave`, with the reason | +| `read-failure` | `listWorkflowRunJobs`, `getJobLogs` | failing jobs and bounded log excerpts, as a payload | +| `lease` | `memory.store.put` | the lease, written before the commission | +| `fix` | `host.harness.run` | `base: `, `mergeFrom: origin/`, the assembled task | +| `publish` | `workflow.run.start` Local Publish | appended onto the branch, `wait: 'until_complete'` | +| `handoff` | `createIssueComment` / edit | the updated record, every pass | +| `release` | `memory.store.delete` | the lease, on every terminal path | +| `renew` | `agent.schedule.create` | the next schedule when this one nears `maxFirings` | + +`decide` is a rule. The four reads of D3 decide `fix`, `wait`, `abandon` and `leave` without a model; a model, `ai.generate_json`, is reached for only to read a log excerpt a rule cannot. ### 3.2 What is recorded @@ -63,13 +63,18 @@ Every pass leaves the handoff current and one run in the ledger. A `fix` also le ## 4. What it affects - `packages/platform-artifacts` โ€” the `local-pr-shepherd` skill and its bundle; `listWorkflowRunJobs` and `getJobLogs` on the `github` connector; Local Publish's body marker. -- `packages/schemas` โ€” nothing new unless P0 finds the attention list cannot name a branch, in which case a bounded read of a space's runs by workflow and input field. +- `packages/schemas` โ€” `workflow.run.list`, the bounded read D3.4 needs, because the attention list cannot name a branch (ยง5 P0.4): its input is a workflow slug, one input field and the value it must equal, a status filter defaulting to the non-terminal statuses with `paused` among them, and a `limit` (default 25, max 100); its output is each matching run's id, status, start time and `own`, space-wide. It is registered in the `workflow.run` group beside `list_attention` (`packages/schemas/src/operations/workflow/registrations.ts`), a group the capability profiles already carry. It lands before P2, whose exit โ€” a branch with a paused publication is `wait` โ€” is proven through it. - `packages/web-product` โ€” the shepherd's recent decisions in the space context (D7). - `docs/dev/driving-work-through-the-loop.md` โ€” a stream that wants a branch left alone labels its pull request `hands-off`. ## 5. Phases -**P0 โ€” Measure.** How a fired schedule starts a skill run and what input reaches it (`scheduleCrud`); GitHub's rate budget against the polling interval; the size of a failing job's log and the excerpt that carries the failure; whether `workflow.run.list_attention` can name a branch. _Exit:_ each answer in this plan with the command that produced it. +**P0 โ€” Measure.** How a fired schedule starts a skill run and what input reaches it (`scheduleCrud`); GitHub's rate budget against the polling interval; the size of a failing job's log and the excerpt that carries the failure; whether `workflow.run.list_attention` can name a branch. _Exit:_ each answer in this plan with the command that produced it โ€” met. + +1. **The schedule-to-skill-input path.** _Command:_ a code read of the scheduler path (`apps/aflow-orchestrator/src/services/ScheduleEvaluator.ts`, `packages/schemas/src/schedules/inputResolution.ts`, `apps/aflow-orchestrator/src/services/SessionOrchestrator/lifecycle/startRun.ts` and their callers) by the coding agent at base `a7939a74`. _Answer:_ a schedule never calls `workflow.run.start`; it starts an agent run, and a skill runs only if that agent then calls `workflow.run.start` with inputs it picks itself. The template is stored verbatim, checked at creation only for non-emptiness (`apps/aflow-orchestrator/src/services/SessionOrchestrator/handlers/inlineOps/scheduleCrud.ts:152-160`, `:499`; `packages/schemas/src/schedules/operations.ts:145-153` โ€” no skill-slug field exists). Firing resolves it with ctx `{now}` (`apps/aflow-orchestrator/src/services/ScheduleEvaluator.ts:513`); `on_completion` adds `{sourceRun, now}` where `sourceRun.output` is a `PayloadRef` string, not the dereferenced output, and `input` is never set (`:889-892`, `:937-940`). Substitutions are `$now`, `$sourceRef`, `$memoryRef` (`packages/schemas/src/schedules/inputResolution.ts:68-129`); no caller supplies a memory resolver, so any `$memoryRef` fails the firing (`apps/aflow-orchestrator/src/services/ScheduleEvaluator.ts:514-522`). Dispatch is inline base64 JSON on a `start_run` control message (`:737-757`, `:1090-1093`), trigger `'schedule'`, unattended turn (`packages/schemas/src/runtime/agentPolicies.ts:23-25`). The agent's input contract is checked only at run start (`apps/aflow-orchestrator/src/services/SessionOrchestrator/lifecycle/startRun.ts:117-139`). A scheduled run can carry: static JSON (a branch name, an owner/repo, a SHA range as literal text), `$now` timestamps, and for `on_completion` the source `runId`/`status`/`outputRef`. It cannot carry: live or derived values, any cross-firing cursor (only `firing_count` advances), a skill slug or typed skill inputs as first-class fields, or a repo coordinate โ€” a coding skill's repo is campaign config (`apps/aflow-orchestrator/src/services/SessionOrchestrator/handlers/inlineOps/workflowCrud/run/start.ts:446`) and `pr-shepherd`'s only run input is `prNumber` (`packages/platform-artifacts/src/skillCatalog/prShepherd.ts:136-141`, `:158-165`). +2. **GitHub's rate budget.** _Command:_ `gh api rate_limit` (live probe of the binding's token; `x-ratelimit-*` headers read off a bound `GET /repos/aflowai/aflow`). _Answer:_ core limit 5,000 requests/hour; 5 used at probe, window reset ~38 minutes out. `x-ratelimit-*` and `Retry-After` reach the calling agent unparsed (`apps/aflow-executor-api/src/lib/header-redaction.ts:9-20`). In code there is no GitHub-specific handling: only the generic retry โ€” 2 retries, exponential backoff 1sโ†’30s, on 429/502/503/504, idempotent methods only (`apps/aflow-executor-api/src/handlers/api/execution.ts:74-163`, `apps/aflow-executor-api/src/handlers/api/config.ts:26-40`); it never reads `Retry-After` or `x-ratelimit-*`; the exhausted-limit 403 is not retried; POST/PATCH calls are never retried on 429; `API_RATE_LIMITED` is declared but never raised (`apps/aflow-executor-api/src/lib/api-errors.ts:92`, `:157`). _The interval it gives:_ one pass costs one `listPullRequests` read, then for each pull request the requests of the ยง3.1 tasks: `read-pr`'s three reads (`getPullRequest`, `listCheckRuns`, `listIssueComments`), `read-failure`'s jobs read for the failing run and log read for the failing job, and `handoff`'s one write โ€” five reads and one write, 6 requests, for a pull request red on one failing job; each further failing job adds one log read. A pass over N such pull requests is 1 + 6N requests. With N = 10: 1 + 60 = 61 requests a pass. One pass every five minutes is 12 an hour: 12 ร— 61 = 732 requests an hour, 732 / 5,000 โ‰ˆ 15% of the budget. With every GET retried twice, the generic retry's ceiling, and the write not retried: 3 ร— (1 + 50) + 10 = 163 a pass, 12 ร— 163 = 1,956 an hour, โ‰ˆ 39%. Five minutes spends the whole budget at 5,000 / 12 โ‰ˆ 416 requests a pass: 1 + 6N โ‰ค 416 gives N = 69 (12 ร— 415 = 4,980), and at the retry ceiling 3 ร— (1 + 5N) + N = 3 + 16N โ‰ค 416 gives N = 25 (12 ร— 403 = 4,836). The default interval is five minutes, `*/5 * * * *`, and the rest of the hour's budget stays with the commission, Local Publish and the operator's own use of the same token. +3. **The size of a failing job's log.** _Commands:_ `gh api repos/aflowai/aflow/actions/runs/37460214524/attempts/1/jobs` to name the failing jobs, then `gh api repos/aflowai/aflow/actions/jobs//logs` โ€” GitHub's `GET /repos/{owner}/{repo}/actions/jobs/{job_id}/logs`, which answers with a redirect to a signed URL โ€” for CI run 37460214524 on `aflowai/aflow` (PR #94's first run). _Answer:_ the Test job (112257622723) is 5,955 lines, 923,330 bytes; Vitest's "Failed Tests" block begins at line 5,676, so the last 280 lines carry the failure and its assertion. The Dependency audit job (112257622741) is 421 lines, 32,310 bytes; the advisories first appear at line 232 and the final attempt's block begins at line 387, so the last 35 lines carry every advisory once. An excerpt bounded at the last 300 lines, under 60 KB, carried the failure for both job kinds observed; a bound needs no endpoint binding to hold, and the D5 endpoints are P1's. +4. **Whether the attention list can name a branch.** _Command:_ `workflow.run.list_attention` (pending and consumed items). _Answer:_ no. Items carry `id`, `kind`, `relatedRunId`, `relatedResource` (`workflow_run:`), `payload.terminalStatus`, `priority`, `createdAt`, `consumedAt`, `own` โ€” no branch, no PR, no repo field; learning what a run produced costs a follow-up run-detail read per item. **P1 โ€” One pass, by hand.** The skill with `read-*`, `decide`, `handoff` and `release`, fired manually against a pull request with a planted failing test; `fix` and `publish` land it. _Exit:_ the planted failure is fixed and pushed within one pass without a hand; a second pass on the same check run is `wait`.