From 20f3b7a128f25c4e4d25422ab271123dac6dca20 Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 22:50:21 +0800 Subject: [PATCH 1/9] chore(loop): record the write-credential scope decision (does PR write include merge?) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Answers a question the host had no answer for, and pins the L2 scope before anyone configures LOOP_GH_TOKEN. No behaviour change. The finding that drives the decision: **merging a pull request is not covered by "Pull requests: write"**. GitHub's endpoint-to-permission table puts `PUT /pulls/{n}/merge` under **Contents**, because merging means writing commits to the target branch. `Pull requests: write` covers create/update/review and review comments, and nothing more. Two consequences worth stating plainly: - "Can open a PR" and "can merge a PR" are the same permission. A PR needs a head branch, creating a branch needs `Contents: write`, and that is the merge permission. They cannot be separated. - `Contents: write` also carries `POST /releases` (the release gate) and `POST /dispatches`, so granting it for PR creation hands over more than the obvious. Hence two sizes of L2, with different risk: - L2a (recommended): `Issues: RW` + `Pull requests: RW` + `Contents: read`. Covers every write action the host actually implements (`planActions`: label / comment / close), and leaves merging, code and branches physically out of reach. `no self-merge` moves from a prompt constraint to a platform impossibility, matching the guarantee `git push` already has. - L2b: adds `Contents: RW` for branch push and PR creation — and with it merge and release. Server-side rulesets do not rescue L2b in this repository: it is a solo repo (one collaborator with admin), so a required-approval rule would block the owner's own PRs forever, and an admin bypass actor would let the loop's PAT bypass it too. The existing `Main branch` ruleset is `enforcement: disabled` and only covers `deletion` + `non_fast_forward` — it never restricted merging. Also corrects three statements that had drifted out of date: the architecture diagram now shows the `terminal` decision (D33) and the D34-conditional push, the `auto` row notes that permission is not ability while the PAT is unset, and the L2 checklist item points at the scope choice. --- scripts/loop/README.md | 73 +++++++++++++++++++++++++++++++++++++++--- 1 file changed, 68 insertions(+), 5 deletions(-) diff --git a/scripts/loop/README.md b/scripts/loop/README.md index 5278dc2..f25ae36 100644 --- a/scripts/loop/README.md +++ b/scripts/loop/README.md @@ -47,20 +47,79 @@ pnpm loop summary | view # summary / task list | Level | Effect | | --- | --- | | `report` | State layer + report only. GitHub write actions are **listed, never executed** (A1 semantics). | -| `auto` | Executes labels/comments/close (the A0 closed-loop boundary; L2 default, and the rehearsal mode). | +| `auto` | Executes labels/comments/close (the A0 closed-loop boundary; L2 default, and the rehearsal mode). Note this is *permission* to write, not *ability*: with `LOOP_GH_TOKEN` unset, `auto` still cannot write — see the credential section below. | In `report` mode the pending actions are appended to `state/reports/.md` as a checklist and surfaced in the Actions Step Summary, so a human can execute them by hand. +### GitHub write credential (`LOOP_GH_TOKEN`, unset today) + +`GH_TOKEN: ${{ secrets.LOOP_GH_TOKEN || github.token }}`. The secret is +**deliberately not configured**: `github.token` is capped by the workflow's +`permissions: {contents: read, issues: read, pull-requests: read}`, so today a +run that ignores its prompt and calls `gh pr merge` gets a 403. Two independent +locks, and the platform one does not depend on the model behaving. + +Configuring the secret lifts only the platform lock — `writeLevel` still gates +`applyActions` in `state.mjs`, and its default is `report`, so ordinary runs +keep reporting. What *does* change: the agent subprocess inherits +`env: process.env` (`shared/agent.mjs`), so a PAT reaches it and the only thing +left between the agent and a write is the prompt. That is the trade, and it is +why the scope below matters. + +**Do not grant `Contents: write`.** Merging a PR is not under "Pull requests" — +it is under "Contents" (GitHub's endpoint→permission table): + +| Action | gh command | Permission required | +| --- | --- | --- | +| label / comment an **issue** | `gh issue edit` / `comment` | `Issues: write` | +| label / comment a **PR** | `gh pr edit` / `comment` | `Issues: write` **or** `Pull requests: write` — both sections list `POST /issues/{n}/labels` and `/issues/{n}/comments` | +| close an **issue** | `gh issue close` | `Issues: write` | +| close a **PR** | `gh pr close` | `Pull requests: write` (GraphQL `closePullRequest`, not an `/issues/` call) | +| open a PR | `gh pr create` | `Pull requests: write` **plus** `Contents: write` — the head branch must exist first | +| **merge a PR** | `gh pr merge` | **`Contents: write`** — `PUT /pulls/{n}/merge` | +| push / edit a file / delete a branch | `git push`, `PUT /contents/{path}`, `DELETE /git/refs/{ref}` | `Contents: write` | + +Two consequences that are easy to get wrong: + +1. **`Pull requests: write` cannot merge.** It covers `POST /pulls`, + `PATCH /pulls/{n}`, reviews and review comments — the merge endpoint sits in + the `Contents` section, because merging means writing commits to the target + branch. +2. **"Can open a PR" and "can merge a PR" are the same permission.** Opening a + PR needs a head branch, creating one needs `Contents: write`, and that is the + merge permission. They cannot be separated in this model. + +So the L2 step has two distinct sizes, and only the first is safe today: + +| | Scopes | Gains | Cost | +| --- | --- | --- | --- | +| **L2a** (recommended) | `Issues: RW`, `Pull requests: RW`, `Contents: read`, Metadata: read | labels, comments, closes, review comments | cannot open PRs — but **merging is impossible**, and no code or branch can be touched | +| **L2b** | L2a + `Contents: RW` | everything above plus branch push and PR creation | **also grants merge**, `POST /releases` (the release gate) and `POST /dispatches` | + +Prefer a **fine-grained** PAT scoped to `Alex1990/tiny-oss`; a classic `repo` +token grants all of the above at once, defeating the point. Set an expiry and +record who renews it, or L2 fails silently later. + +Server-side protection does **not** rescue L2b here: this is a solo repository +(one collaborator, `admin: true`), so a ruleset requiring approvals would block +the owner's own PRs forever (GitHub forbids self-approval), and adding an +admin `bypass_actor` would let the loop's PAT bypass it too. The existing +`Main branch` ruleset is `enforcement: disabled` and contains only `deletion` +and `non_fast_forward` — it never restricted merging. + +Net effect of choosing L2a: `no self-merge` stops being a prompt constraint and +becomes a platform impossibility — the same guarantee `git push` has today. + ## A1 architecture (Actions + R2) ``` GitHub event ─▶ loop.yml (concurrency group `loop` = platform-level single writer) ├─ setup node/pnpm ▸ install pi ▸ print `pi --list-models deepseek` ├─ r2-sync pull (state layer → job-local state/) - ├─ entry.mjs (route → inbox | metrics | run) - ├─ r2-sync push (if: always()) + ├─ entry.mjs (route → inbox | metrics | terminal | run) + ├─ r2-sync push (if: always() && pull did not skip — D34) └─ upload pi sessions (artifact, 90d) ``` @@ -351,7 +410,11 @@ was *correct*, and whether new events produce proposals that match reality. `Install engine (pi)`, before `entry.mjs`, so they write no run row at all — see the failure-classification item above. (Distinction worth keeping: 21 workflow executions, 18 run records.) -- [ ] No drift **and** no incorrect proposal after a week → human decides on L2 +- [ ] No drift **and** no incorrect proposal after a week → human decides on L2. + When it does, choose the scope deliberately: **L2a** (`Issues: RW` + + `Pull requests: RW`, no `Contents: RW`) keeps merging impossible for the + loop, while **L2b** adds PR creation and merge in one indivisible step. + See "GitHub write credential" above for the endpoint-level evidence. ### Not reachable under A1 (structural — decide before the L2 switchover) @@ -365,7 +428,7 @@ validated at the report boundary however long it runs: | `pr-review` route | also needs a loop PR; A1 never produces one | untested here — exercised in A0 | | External-PR read-only analysis | D28 — a fork/Dependabot `pull_request` run carries no secrets, so not even a read-only analysis can reach the LLM | accepted for A1; needs its own credentials to enable | | Label/comment effects on GitHub | the report boundary never writes | proposals must be judged on *correctness*, not on effect | -| `execute_writes` rehearsal | `LOOP_GH_TOKEN` is unset, so `auto` cannot write even when requested | correct as defence in depth — but it means **no write path has ever executed** | +| `execute_writes` rehearsal | `LOOP_GH_TOKEN` is unset, so `auto` cannot write even when requested | correct as defence in depth — but it means **no write path has ever executed**. Configuring the PAT is a prerequisite for testing any of it; see "GitHub write credential" | | R2 bucket versioning | R2 offers no object versioning | accepted deviation from 05 §8 (see Environment facts); `push`/`seed` never use `--delete` | Consequence for the L2 decision: a clean observation week proves the loop is From 1037acce6584c6c71c8ad885947b33443470fb4c Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:04:16 +0800 Subject: [PATCH 2/9] =?UTF-8?q?chore(loop):=20record=20L2c=20=E2=80=94=20t?= =?UTF-8?q?he=20loop=20opens=20PRs=20and=20still=20cannot=20merge?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Owner's requirement: the loop should open pull requests but must never be able to merge one. The previous revision of this section stopped at "a single credential cannot express that", which is true but not the whole answer — it cannot express it *within one credential*, and the split can be made across roles. - `GITHUB_TOKEN` (the workflow's own steps): `contents: write`, used only to create and push the `loop/-*` branch. - `LOOP_GH_TOKEN` (the agent subprocess): `Issues: RW` + `Pull requests: RW`, no `Contents` — enough for labels, comments and opening the PR, and 403 for `PUT /pulls/{n}/merge`. Two required details, both easy to miss: - `actions/checkout` must set `persist-credentials: false`. Its default is `true` and writes the job token into the repository git config, which would hand the agent the very `Contents: write` this design removes. - Only the push has to move to a host step. Committing is local and needs no network credential, so the agent keeps staging and committing in the workspace; the host pushes and opens the PR from the agent's proposed metadata — the existing "agent proposes, host executes" division. Side effect worth stating: a PR created with `GITHUB_TOKEN` yields a `pull_request` event whose runs start in approval-required state, so the loop will not triage or review its own PR and CI will not run until a human approves the runs. For this loop that is desirable — it removes recursive self-review on the loop's own output, which is the same direction D26 points. Also scopes two earlier claims to "within a single credential" so they no longer contradict L2c, and points the L2 decision item at this target. --- scripts/loop/README.md | 85 +++++++++++++++++++++++++++++++----------- 1 file changed, 64 insertions(+), 21 deletions(-) diff --git a/scripts/loop/README.md b/scripts/loop/README.md index f25ae36..e7f37aa 100644 --- a/scripts/loop/README.md +++ b/scripts/loop/README.md @@ -68,8 +68,9 @@ keep reporting. What *does* change: the agent subprocess inherits left between the agent and a write is the prompt. That is the trade, and it is why the scope below matters. -**Do not grant `Contents: write`.** Merging a PR is not under "Pull requests" — -it is under "Contents" (GitHub's endpoint→permission table): +**Do not grant the agent's credential `Contents: write`.** Merging a PR is not +under "Pull requests" — it is under "Contents" (GitHub's endpoint→permission +table): | Action | gh command | Permission required | | --- | --- | --- | @@ -87,30 +88,71 @@ Two consequences that are easy to get wrong: `PATCH /pulls/{n}`, reviews and review comments — the merge endpoint sits in the `Contents` section, because merging means writing commits to the target branch. -2. **"Can open a PR" and "can merge a PR" are the same permission.** Opening a - PR needs a head branch, creating one needs `Contents: write`, and that is the - merge permission. They cannot be separated in this model. +2. **"Can open a PR" and "can merge a PR" are the same permission** — within a + single credential. Opening a PR needs a head branch; creating one needs + `Contents: write`; and that is the merge permission. Separating them means + splitting the *roles*, which is what L2c below does. -So the L2 step has two distinct sizes, and only the first is safe today: +So the L2 step has three sizes rather than two. Two are pure scope choices; the +third is the one that satisfies "loop may open PRs but must never merge them", +and it needs a small host change. -| | Scopes | Gains | Cost | +| | Scopes / changes | Loop can | Loop cannot | | --- | --- | --- | --- | -| **L2a** (recommended) | `Issues: RW`, `Pull requests: RW`, `Contents: read`, Metadata: read | labels, comments, closes, review comments | cannot open PRs — but **merging is impossible**, and no code or branch can be touched | -| **L2b** | L2a + `Contents: RW` | everything above plus branch push and PR creation | **also grants merge**, `POST /releases` (the release gate) and `POST /dispatches` | +| **L2a** | `Issues: RW` + `Pull requests: RW` + `Contents: read` | label, comment, close, review | open PRs, merge, touch code or branches | +| **L2c** | L2a, **plus** the host pushes `loop/*` branches with its own token | label, comment, close, review, **open PRs** | **merge anything** | +| **L2b** | L2a + `Contents: RW` on the loop's own credential | everything, PR creation included | — (nothing; this is the unchecked size) | Prefer a **fine-grained** PAT scoped to `Alex1990/tiny-oss`; a classic `repo` token grants all of the above at once, defeating the point. Set an expiry and record who renews it, or L2 fails silently later. -Server-side protection does **not** rescue L2b here: this is a solo repository -(one collaborator, `admin: true`), so a ruleset requiring approvals would block -the owner's own PRs forever (GitHub forbids self-approval), and adding an -admin `bypass_actor` would let the loop's PAT bypass it too. The existing -`Main branch` ruleset is `enforcement: disabled` and contains only `deletion` -and `non_fast_forward` — it never restricted merging. +#### L2c — the loop opens PRs and still cannot merge -Net effect of choosing L2a: `no self-merge` stops being a prompt constraint and -becomes a platform impossibility — the same guarantee `git push` has today. +Worth stating up front: **a single credential cannot express this.** Opening a +PR needs a head branch, creating one needs `Contents: write`, and that is the +same permission `PUT /pulls/{n}/merge` requires. The split has to happen across +*roles*, not inside one scope list: + +| Credential | Held by | Scopes | Used for | +| --- | --- | --- | --- | +| `GITHUB_TOKEN` (job) | the workflow's own steps | `contents: write` | creating and pushing the `loop/-*` branch | +| `LOOP_GH_TOKEN` (PAT) | the agent subprocess | `Issues: RW`, `Pull requests: RW` — **no `Contents`** | labels, comments, opening the PR | + +Two details are required and both are easy to miss: + +- `actions/checkout` must set **`persist-credentials: false`**. Its default is + `true` and writes the job token into the repository's git config so scripts + can run authenticated git commands — which would hand the agent exactly the + `Contents: write` this design exists to remove. +- Pushing is the *only* thing that has to move to a host step. Committing is a + local operation with no network credential, so the agent can still stage and + commit its work in the workspace; the host pushes the branch and opens the PR + from the agent's proposed metadata. That also fits the existing division of + labour — the agent proposes, the host executes. + +Result: the agent has no route to `PUT /pulls/{n}/merge` (403 — no `Contents`), +and no route to push anything anywhere. The loop still produces PRs. `no +self-merge` becomes a platform guarantee instead of a prompt instruction, which +is the whole point. + +One side effect to plan for: a PR created with `GITHUB_TOKEN` produces a +`pull_request` event whose workflow runs start in an **approval-required** state +(except `closed`/`labeled`/`edited`, which do not create runs at all). So the +loop will not triage or review its own PR, and CI will not run on it, until a +human clicks "Approve workflows to run". For this loop that is a feature — it +removes the recursive self-review the `pr-review` route would otherwise perform +on the loop's own output, and it matches D26's direction. A human merging the PR +still fires `pull_request_target.closed` normally, which is what drives the +`metrics` gate. + +Server-side protection does **not** substitute for any of this: this is a solo +repository (one collaborator, `admin: true`), so a ruleset requiring approvals +would block the owner's own PRs forever (GitHub forbids self-approval), and +adding an admin `bypass_actor` would let the loop's PAT bypass it too. The +existing `Main branch` ruleset is `enforcement: disabled` and contains only +`deletion` and `non_fast_forward` — it never restricted merging. The guarantee +has to come from the credential split. ## A1 architecture (Actions + R2) @@ -411,10 +453,11 @@ was *correct*, and whether new events produce proposals that match reality. — see the failure-classification item above. (Distinction worth keeping: 21 workflow executions, 18 run records.) - [ ] No drift **and** no incorrect proposal after a week → human decides on L2. - When it does, choose the scope deliberately: **L2a** (`Issues: RW` + - `Pull requests: RW`, no `Contents: RW`) keeps merging impossible for the - loop, while **L2b** adds PR creation and merge in one indivisible step. - See "GitHub write credential" above for the endpoint-level evidence. + Owner's target is **L2c**: the loop opens PRs but can never merge. That + needs the credential split (host pushes branches with `GITHUB_TOKEN`; the + agent's PAT carries no `Contents`) plus `persist-credentials: false` — see + "GitHub write credential" above for the endpoint-level evidence, and note + that a single credential cannot express it. ### Not reachable under A1 (structural — decide before the L2 switchover) From d02b8cef5c703e82f34d0a68d3aaa40a85cb4641 Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:19:28 +0800 Subject: [PATCH 3/9] chore(loop): measure the GitHub credential instead of assuming it MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `GH_TOKEN` is `secrets.LOOP_GH_TOKEN || github.token`, so configuring a PAT silently changes every `gh` call in the job — it can narrow the read-only ones sweep depends on, or widen what the agent can write. Neither is visible from the workflow file, so it has to be measured. Adds `gh-check.mjs` and runs it before the agent can reach the credential. Permission probing without side effects: each write endpoint is called against an impossible id, so `403` means the scope is absent and `404`/`422` means the request passed the permission check and only failed on the missing resource. Nothing is mutated — every probe targets 999999999, far beyond this repository's range. Checks both directions, because they fail differently: - reads the host depends on (repo metadata, issues, PRs, `gh release list`) — a PAT narrower than the job token breaks sweep silently; - writes it should and should not hold — `Issues`, `Pull requests`, and `Contents`, where the last one is reported as a warning because it carries merging, branch pushes and `POST /releases` together. The token value is never printed, only the login it authenticates as. --- .github/workflows/loop.yml | 10 +++ scripts/loop/gh-check.mjs | 126 +++++++++++++++++++++++++++++++++++++ 2 files changed, 136 insertions(+) create mode 100644 scripts/loop/gh-check.mjs diff --git a/.github/workflows/loop.yml b/.github/workflows/loop.yml index 0710ebd..343a327 100644 --- a/.github/workflows/loop.yml +++ b/.github/workflows/loop.yml @@ -168,6 +168,16 @@ jobs: fi echo "model $want is available" + # Measure what the GitHub credential can actually do, before the agent can + # reach it. `GH_TOKEN` is `secrets.LOOP_GH_TOKEN || github.token`, so + # configuring a PAT silently changes every `gh` call in this job — it can + # narrow the read-only ones sweep depends on, or widen what the agent can + # write. Neither is visible from this file. Probes are side-effect free: + # they call write endpoints against an impossible id, so `403` means the + # scope is absent and `404`/`422` means it is present. + - name: Check GitHub credential + run: node scripts/loop/gh-check.mjs + - name: Pull state from R2 id: pull run: node scripts/loop/r2-sync.mjs pull diff --git a/scripts/loop/gh-check.mjs b/scripts/loop/gh-check.mjs new file mode 100644 index 0000000..7602fba --- /dev/null +++ b/scripts/loop/gh-check.mjs @@ -0,0 +1,126 @@ +#!/usr/bin/env node +/** + * GitHub credential self-check — invoked as a standalone workflow step. + * + * Usage: node scripts/loop/gh-check.mjs [owner/repo] + * + * Why this exists: `GH_TOKEN` is `secrets.LOOP_GH_TOKEN || github.token`. The + * moment a real PAT is configured it replaces the job token for **every** `gh` + * call, including the read-only ones sweep depends on. A PAT that is narrower + * than the job token therefore breaks reads silently, and one that is broader + * silently hands the agent write access nobody chose. Neither is visible from + * the workflow file, so it has to be measured. + * + * Permission probing without side effects: a write endpoint is called against a + * resource number that cannot exist. `403` means the token lacks the scope; + * `404`/`422` means the request passed the permission check and only failed on + * the missing resource — i.e. the scope is present. Nothing is ever mutated, + * because every probe targets an id far beyond this repository's range. + * + * The token itself is never printed. Only the login it authenticates as. + */ + +import { spawnSync } from 'node:child_process'; + +const IMPOSSIBLE = '999999999'; // far beyond any real issue/PR number here + +/** Run `gh`, returning the HTTP status it saw and whether it succeeded. */ +function gh(args, { json = false } = {}) { + const r = spawnSync('gh', json ? [...args, '--jq', '.'] : args, { + encoding: 'utf8', + env: { ...process.env, GH_PAGER: '', NO_COLOR: '1' }, + }); + const all = `${r.stdout || ''}\n${r.stderr || ''}`; + const m = /HTTP (\d{3})/.exec(all); + return { + ok: r.status === 0, + status: m ? Number(m[1]) : (r.status === 0 ? 200 : null), + out: (r.stdout || '').trim(), + err: (r.stderr || '').trim(), + }; +} + +/** + * `403`/`401` = scope absent. Anything else means the permission check passed + * and the request failed later (missing resource, bad body) — scope present. + */ +const hasScope = (status) => status !== null && status !== 403 && status !== 401; + +function probe(label, args) { + const r = gh(args); + const present = hasScope(r.status); + console.log(` ${present ? 'present' : 'ABSENT '} ${label}` + + ` (HTTP ${r.status ?? 'n/a'})`); + return { label, present, status: r.status, err: r.err }; +} + +const repo = process.argv[2] || process.env.GITHUB_REPOSITORY || ''; +if (!repo) { + console.error('[loop:gh] error: pass owner/repo or set GITHUB_REPOSITORY'); + process.exit(1); +} + +console.log(`[loop:gh] repo: ${repo}`); + +const who = gh(['api', 'user', '--jq', '.login']); +if (!who.ok) { + // A token that cannot even identify itself is the one hard failure: every + // later `gh` call in the job would fail too, so fail loudly and early. + console.error(`[loop:gh] error: token cannot authenticate — ${who.err || who.status}`); + process.exit(1); +} +console.log(`[loop:gh] token authenticates as: ${who.out}`); +const isJobToken = who.out === 'github-actions[bot]'; +// Only claim which credential this is inside Actions: run locally, `gh` uses +// whatever the developer is logged in as, and naming LOOP_GH_TOKEN there would +// be plainly wrong. +if (process.env.GITHUB_ACTIONS) { + console.log(`[loop:gh] credential: ${isJobToken + ? 'github.token (job token — LOOP_GH_TOKEN is unset or empty)' + : 'LOOP_GH_TOKEN (a PAT, replacing the job token for every gh call)'}`); +} + +console.log('\n[loop:gh] reads — the host depends on these; a miss breaks sweep:'); +// sweep lists open issues/PRs and calls `gh release list`; repo metadata is the +// cheapest liveness check. +const reads = [ + ['repo metadata (GET /repos)', ['api', `repos/${repo}`, '--jq', '.full_name']], + ['issues list (sweep)', ['issue', 'list', '-R', repo, '--state', 'open', '--limit', '1']], + ['pull requests list (sweep)', ['pr', 'list', '-R', repo, '--state', 'open', '--limit', '1']], + ['releases list (sweep)', ['release', 'list', '-R', repo, '--limit', '1']], +]; +const readResults = reads.map(([label, args]) => { + const r = gh(args); + console.log(` ${r.ok ? 'OK ' : 'FAILED '} ${label}` + + `${r.ok ? '' : ` — ${(r.err || r.status).toString().slice(0, 90)}`}`); + return { label, ok: r.ok }; +}); + +console.log('\n[loop:gh] writes — probes target an impossible number, nothing is changed:'); +const writes = [ + ['Issues: write (label/comment/close)', ['api', '--method', 'PATCH', + `repos/${repo}/issues/${IMPOSSIBLE}`, '-f', 'state=open']], + ['Pull requests: write (open/edit PR)', ['api', '--method', 'PATCH', + `repos/${repo}/pulls/${IMPOSSIBLE}`, '-f', 'state=open']], + ['Contents: write (MERGE + push)', ['api', '--method', 'PUT', + `repos/${repo}/pulls/${IMPOSSIBLE}/merge`]], +]; +const results = writes.map(([label, args]) => probe(label, args)); +const canMerge = results.find((r) => r.label.startsWith('Contents'))?.present ?? false; +const brokenReads = readResults.filter((r) => !r.ok); +const canLabel = results.find((r) => r.label.startsWith('Issues'))?.present ?? false; + +console.log(''); +if (canMerge) { + console.log('[loop:gh] WARNING: the token holds Contents:write. That single scope grants' + + ' merging (PUT /pulls/{n}/merge), branch pushes, and POST /releases — none of which' + + ' the loop should have. See "GitHub write credential" in scripts/loop/README.md.'); + console.log('[loop:gh] Recreate the PAT with: Issues RW, Pull requests RW, Contents read.'); +} else { + console.log('[loop:gh] Contents:write absent — merging is impossible for this credential,' + + ' which is the intended L2c property.'); +} +console.log(`[loop:gh] label/comment/close: ${canLabel ? 'available' : 'NOT available — the loop cannot do its job'}`); +console.log(`[loop:gh] reads: ${brokenReads.length + ? `${brokenReads.length} FAILED — the host's sweep will break: ${brokenReads.map((r) => r.label.trim().split(/\s+/)[0]).join(', ')}` + : 'all OK'}`); From a9a97e1fc8ec949a29baec30c26ae6dea8aa5e5c Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:21:57 +0800 Subject: [PATCH 4/9] chore(loop): widen the credential probe to the dangerous scopes too MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Contents is the one that matters most, but it is not the only way to reach code or CI without a pull request. Splits the probes into scopes the loop needs and scopes it must not hold, and adds the two other high-value ones: - Actions: write — dispatching a workflow is indirect code execution - Secrets — listing secret names requires the permission, and it can overwrite the CI credentials. The listing is never printed, only whether it is present. Local check against an admin token reports all three held, confirming the probes discriminate rather than always returning 'present'. --- scripts/loop/gh-check.mjs | 42 ++++++++++++++++++++++++++------------- 1 file changed, 28 insertions(+), 14 deletions(-) diff --git a/scripts/loop/gh-check.mjs b/scripts/loop/gh-check.mjs index 7602fba..a71b538 100644 --- a/scripts/loop/gh-check.mjs +++ b/scripts/loop/gh-check.mjs @@ -96,29 +96,43 @@ const readResults = reads.map(([label, args]) => { return { label, ok: r.ok }; }); -console.log('\n[loop:gh] writes — probes target an impossible number, nothing is changed:'); -const writes = [ +console.log('\n[loop:gh] writes the loop needs (probes target an impossible id — nothing changes):'); +const needed = [ ['Issues: write (label/comment/close)', ['api', '--method', 'PATCH', `repos/${repo}/issues/${IMPOSSIBLE}`, '-f', 'state=open']], - ['Pull requests: write (open/edit PR)', ['api', '--method', 'PATCH', + ['Pull requests: write (open/edit PR, reviews)', ['api', '--method', 'PATCH', `repos/${repo}/pulls/${IMPOSSIBLE}`, '-f', 'state=open']], - ['Contents: write (MERGE + push)', ['api', '--method', 'PUT', +]; +const needResults = needed.map(([label, args]) => probe(label, args)); + +console.log('\n[loop:gh] scopes the loop must NOT hold:'); +const forbidden = [ + ['Contents: write (MERGE + push + release)', ['api', '--method', 'PUT', `repos/${repo}/pulls/${IMPOSSIBLE}/merge`]], + ['Actions: write (trigger/disable workflows)', ['api', '--method', 'POST', + `repos/${repo}/actions/workflows/${IMPOSSIBLE}/dispatches`, '-f', 'ref=main']], ]; -const results = writes.map(([label, args]) => probe(label, args)); -const canMerge = results.find((r) => r.label.startsWith('Contents'))?.present ?? false; +const forbiddenResults = forbidden.map(([label, args]) => probe(label, args)); +// Read-only probe: listing secret *names* needs the Secrets permission. The +// listing itself is never printed — only whether the permission is present. +const secrets = gh(['api', `repos/${repo}/actions/secrets`]); +console.log(` ${secrets.ok ? 'present' : 'ABSENT '} Secrets access (list/overwrite CI secrets)`); + +const held = forbiddenResults.filter((r) => r.present).map((r) => r.label.trim().split(/\s+/)[0]); const brokenReads = readResults.filter((r) => !r.ok); -const canLabel = results.find((r) => r.label.startsWith('Issues'))?.present ?? false; +const canLabel = needResults.find((r) => r.label.startsWith('Issues'))?.present ?? false; console.log(''); -if (canMerge) { - console.log('[loop:gh] WARNING: the token holds Contents:write. That single scope grants' - + ' merging (PUT /pulls/{n}/merge), branch pushes, and POST /releases — none of which' - + ' the loop should have. See "GitHub write credential" in scripts/loop/README.md.'); - console.log('[loop:gh] Recreate the PAT with: Issues RW, Pull requests RW, Contents read.'); +if (held.length) { + console.log(`[loop:gh] WARNING: the token holds ${[...held, ...(secrets.ok ? ['Secrets'] : [])].join(', ')}.` + + ' Each of those is a way to reach code or CI without going through a pull request —' + + ' Contents alone grants merging (PUT /pulls/{n}/merge), branch pushes and POST /releases.'); + console.log('[loop:gh] Recreate the PAT as: Issues Read and write, Pull requests Read and write,' + + ' Contents Read-only, and Nothing for Actions/Secrets/Administration/Workflows.' + + ' See "GitHub write credential" in scripts/loop/README.md.'); } else { - console.log('[loop:gh] Contents:write absent — merging is impossible for this credential,' - + ' which is the intended L2c property.'); + console.log('[loop:gh] none of the forbidden scopes are held — merging, branch pushes and' + + ' workflow control are impossible for this credential, which is the intended L2c property.'); } console.log(`[loop:gh] label/comment/close: ${canLabel ? 'available' : 'NOT available — the loop cannot do its job'}`); console.log(`[loop:gh] reads: ${brokenReads.length From bcc22c4194d1bf629b58c1fa2b531c44f51f7fb8 Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:25:44 +0800 Subject: [PATCH 5/9] chore(loop): keep a writable credential out of the agent's hands MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit `runPi` spawned the agent with `env: process.env`, so once `LOOP_GH_TOKEN` was configured the agent inherited the PAT — and the report boundary would have been enforced by the prompt alone. Measured against the real PAT, that mattered: it holds `Contents: write`, which is merging, branch pushes and `POST /releases` in one scope. The host still needs the real token (future `applyActions`), but the agent does not, so the two are now separated at the spawn boundary: GH_TOKEN the host's credential (PAT when configured) GH_READ_TOKEN `github.token`, capped at read by `permissions:` `runPi` hands the child `GH_READ_TOKEN` whenever `writeLevel !== 'auto'`, and logs which one it used. The agent therefore reaches GitHub read-only even if it ignores its prompt, and `auto` is unchanged — the agent gets the write token, which is what that mode is for. `gh-check.mjs` now also measures this: it probes the credential the *agent* will hold and says outright whether the report boundary is a platform guarantee or back to being a prompt instruction. That makes the property observable in a smoke run, with no agent tokens spent. Verified locally with a stub engine: under `report` the child sees `READ-ONLY-JOB-TOKEN`; under `auto` it sees the write token. --- .github/workflows/loop.yml | 6 ++++++ scripts/loop/README.md | 19 ++++++++++++++---- scripts/loop/entry.mjs | 1 + scripts/loop/gh-check.mjs | 37 ++++++++++++++++++++++++++++++----- scripts/loop/shared/agent.mjs | 22 +++++++++++++++++++-- 5 files changed, 74 insertions(+), 11 deletions(-) diff --git a/.github/workflows/loop.yml b/.github/workflows/loop.yml index 343a327..b9d5007 100644 --- a/.github/workflows/loop.yml +++ b/.github/workflows/loop.yml @@ -97,6 +97,12 @@ jobs: LOOP_MODEL: ${{ github.event.inputs.model || 'deepseek/deepseek-v4-flash' }} LOOP_RUN_TIMEOUT_MS: '3300000' # 55min, leaving headroom for push (job timeout 60min) GH_TOKEN: ${{ secrets.LOOP_GH_TOKEN || github.token }} + # Handed to the agent subprocess in place of GH_TOKEN under the report + # boundary (see runPi in shared/agent.mjs). `github.token` here is capped + # at read by the `permissions:` block above, so the agent cannot write even + # if LOOP_GH_TOKEN is a write-capable PAT. Host-side writes still use + # GH_TOKEN, in the host's own process. + GH_READ_TOKEN: ${{ github.token }} # Credentials: secrets only DEEPSEEK_API_KEY: ${{ secrets.DEEPSEEK_API_KEY }} R2_ACCESS_KEY_ID: ${{ secrets.R2_ACCESS_KEY_ID }} diff --git a/scripts/loop/README.md b/scripts/loop/README.md index e7f37aa..84c837b 100644 --- a/scripts/loop/README.md +++ b/scripts/loop/README.md @@ -63,10 +63,21 @@ locks, and the platform one does not depend on the model behaving. Configuring the secret lifts only the platform lock — `writeLevel` still gates `applyActions` in `state.mjs`, and its default is `report`, so ordinary runs -keep reporting. What *does* change: the agent subprocess inherits -`env: process.env` (`shared/agent.mjs`), so a PAT reaches it and the only thing -left between the agent and a write is the prompt. That is the trade, and it is -why the scope below matters. +keep reporting. + +It must not also lift the **agent's** lock. `runPi` spawns the agent with +`env: process.env`, which would hand it whatever `GH_TOKEN` holds — and since +that is now a PAT, the agent would inherit the PAT's full reach. So under the +report boundary the child gets `GH_READ_TOKEN` instead: `github.token`, which +`permissions:` caps at read. The agent therefore reaches GitHub read-only even +if it ignores its prompt, while host-side writes keep using the real `GH_TOKEN` +in the host's own process. Under `auto` the agent gets the write token, which is +the point of that mode. + +Both are measurable without spending a token: +`node scripts/loop/gh-check.mjs` probes each credential's scopes side-effect +free (write endpoints against an impossible id — `403` means absent, `404`/`422` +means present) and reports what the host holds and what the agent will hold. **Do not grant the agent's credential `Contents: write`.** Merging a PR is not under "Pull requests" — it is under "Contents" (GitHub's endpoint→permission diff --git a/scripts/loop/entry.mjs b/scripts/loop/entry.mjs index 4d73726..cdb559f 100644 --- a/scripts/loop/entry.mjs +++ b/scripts/loop/entry.mjs @@ -243,6 +243,7 @@ async function doRun({ decision, ctx, repo, writeLevel }) { prompt, cwd: ROOT, sessionDir, model: process.env.LOOP_MODEL, timeoutMs, + writeLevel, log, }); const events = parseEvents(res.stdout); diff --git a/scripts/loop/gh-check.mjs b/scripts/loop/gh-check.mjs index a71b538..930247f 100644 --- a/scripts/loop/gh-check.mjs +++ b/scripts/loop/gh-check.mjs @@ -25,11 +25,10 @@ import { spawnSync } from 'node:child_process'; const IMPOSSIBLE = '999999999'; // far beyond any real issue/PR number here /** Run `gh`, returning the HTTP status it saw and whether it succeeded. */ -function gh(args, { json = false } = {}) { - const r = spawnSync('gh', json ? [...args, '--jq', '.'] : args, { - encoding: 'utf8', - env: { ...process.env, GH_PAGER: '', NO_COLOR: '1' }, - }); +function gh(args, { token = null } = {}) { + const env = { ...process.env, GH_PAGER: '', NO_COLOR: '1' }; + if (token) env.GH_TOKEN = token; + const r = spawnSync('gh', args, { encoding: 'utf8', env }); const all = `${r.stdout || ''}\n${r.stderr || ''}`; const m = /HTTP (\d{3})/.exec(all); return { @@ -138,3 +137,31 @@ console.log(`[loop:gh] label/comment/close: ${canLabel ? 'available' : 'NOT avai console.log(`[loop:gh] reads: ${brokenReads.length ? `${brokenReads.length} FAILED — the host's sweep will break: ${brokenReads.map((r) => r.label.trim().split(/\s+/)[0]).join(', ')}` : 'all OK'}`); + +// The credential the *agent* will hold. `runPi` swaps GH_TOKEN for GH_READ_TOKEN +// under the report boundary, so this is the one that decides whether the agent +// could write if it ignored its prompt. Measuring it here makes that property +// observable without spending a single token on a real agent run. +const agentToken = process.env.GH_READ_TOKEN; +console.log('\n[loop:gh] credential handed to the agent under the report boundary:'); +if (!agentToken) { + console.log(' (GH_READ_TOKEN is not set — runPi would pass GH_TOKEN through unchanged,' + + ' i.e. the agent inherits the host credential above)'); +} else { + const agentProbes = [ + ['Issues: write', ['api', '--method', 'PATCH', `repos/${repo}/issues/${IMPOSSIBLE}`, '-f', 'state=open']], + ['Contents: write (merge/push)', ['api', '--method', 'PUT', `repos/${repo}/pulls/${IMPOSSIBLE}/merge`]], + ]; + const agentHeld = []; + for (const [label, args] of agentProbes) { + const r = gh(args, { token: agentToken }); + const present = hasScope(r.status); + if (present) agentHeld.push(label); + console.log(` ${present ? 'present' : 'ABSENT '} ${label}`); + } + console.log(agentHeld.length + ? ` WARNING: the agent holds ${agentHeld.join(', ')} — the report boundary is back to being` + + ' a prompt instruction only.' + : ' OK: the agent holds no write scope, so even ignoring its prompt it reaches GitHub' + + ' read-only. The report boundary is a platform guarantee.'); +} diff --git a/scripts/loop/shared/agent.mjs b/scripts/loop/shared/agent.mjs index 812b992..7212926 100644 --- a/scripts/loop/shared/agent.mjs +++ b/scripts/loop/shared/agent.mjs @@ -125,7 +125,9 @@ export function summarize(events) { */ export const SESSION_HINT = 'session files land under --session-dir (uploaded as an artifact)'; -export function runPi({ prompt, cwd, sessionDir, model, timeoutMs = 3600000, log = () => {} }) { +export function runPi({ + prompt, cwd, sessionDir, model, timeoutMs = 3600000, writeLevel = 'report', log = () => {}, +}) { return new Promise((resolve) => { const args = [ '--mode', 'json', @@ -140,8 +142,24 @@ export function runPi({ prompt, cwd, sessionDir, model, timeoutMs = 3600000, log // to pi, overridable with LOOP_ENGINE_CMD (e.g. run the same contract locally through // another CLI or wrapper script). const engine = (process.env.LOOP_ENGINE_CMD || 'pi').split(/\s+/).filter(Boolean); + + // `env: process.env` hands the agent everything the host sees, including + // `GH_TOKEN` — which is `secrets.LOOP_GH_TOKEN || github.token`, so as soon + // as a PAT is configured the agent inherits whatever that PAT can do. Under + // the report boundary nothing the agent does is supposed to reach GitHub, + // and that must not rest on the prompt alone: hand it the job token instead, + // which the workflow caps at `read-only` via `permissions:`. Writes by the + // *host* (applyActions) still use the real GH_TOKEN in its own process. + const childEnv = { ...process.env }; + if (writeLevel !== 'auto' && process.env.GH_READ_TOKEN) { + childEnv.GH_TOKEN = process.env.GH_READ_TOKEN; + log('[loop] agent is given a read-only GitHub token (report boundary)'); + } else if (writeLevel === 'auto') { + log('[loop] agent is given the write-capable GitHub token (auto mode)'); + } + const child = spawn(engine[0], [...engine.slice(1), ...args], { - cwd, env: process.env, stdio: ['pipe', 'pipe', 'pipe'], + cwd, env: childEnv, stdio: ['pipe', 'pipe', 'pipe'], }); let stdout = ''; let stderr = ''; From af8cc9fd02eaa83490ecbdc6636abc5dde60068d Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:28:20 +0800 Subject: [PATCH 6/9] chore(loop): cross-check the credential probes against GitHub's own permissions MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 403-vs-404 reading assumes GitHub checks permissions before resource lookup. The first Actions run reported the agent's github.token as holding Issues and Contents write, which contradicts the workflow's read-only `permissions:` block — so either the assumption is wrong or the token really does hold write. Both credentials now also report the `permissions` object GitHub returns for them, which needs no inference. --- scripts/loop/gh-check.mjs | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/scripts/loop/gh-check.mjs b/scripts/loop/gh-check.mjs index 930247f..ae9669f 100644 --- a/scripts/loop/gh-check.mjs +++ b/scripts/loop/gh-check.mjs @@ -112,6 +112,14 @@ const forbidden = [ `repos/${repo}/actions/workflows/${IMPOSSIBLE}/dispatches`, '-f', 'ref=main']], ]; const forbiddenResults = forbidden.map(([label, args]) => probe(label, args)); +// Independent confirmation of the probes above: GitHub's own `permissions` +// object, which needs no inference from a status code. Relevant because the +// `403` vs `404` reading assumes GitHub checks permissions *before* resource +// lookup, which is worth verifying rather than assuming. +const hostPerm = gh(['api', `repos/${repo}`, '--jq', '.permissions']); +console.log(` repo permissions: ${hostPerm.ok + ? hostPerm.out.replace(/\s+/g, ' ') + : `(unreadable — ${(hostPerm.err || hostPerm.status).toString().slice(0, 60)})`}`); // Read-only probe: listing secret *names* needs the Secrets permission. The // listing itself is never printed — only whether the permission is present. const secrets = gh(['api', `repos/${repo}/actions/secrets`]); @@ -159,6 +167,13 @@ if (!agentToken) { if (present) agentHeld.push(label); console.log(` ${present ? 'present' : 'ABSENT '} ${label}`); } + // Cross-check the probe, because `403` vs `404` is an inference about where + // GitHub checks permissions relative to resource lookup. The `permissions` + // object is reported by GitHub itself and needs no inference. + const agentPerm = gh(['api', `repos/${repo}`, '--jq', '.permissions'], { token: agentToken }); + console.log(` repo permissions: ${agentPerm.ok + ? agentPerm.out.replace(/\s+/g, ' ') + : `(unreadable — ${(agentPerm.err || agentPerm.status).toString().slice(0, 60)})`}`); console.log(agentHeld.length ? ` WARNING: the agent holds ${agentHeld.join(', ')} — the report boundary is back to being` + ' a prompt instruction only.' From b311f76a3d1d5c2267c9c5d1b55fbd62be68ef69 Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:31:47 +0800 Subject: [PATCH 7/9] chore(loop): replace the unsound permission probe with idempotent writes MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The 403-vs-404 method was wrong, and the first real measurement proved it: a `github.token` that the workflow caps to read-only replied `404` to a write, not `403`. So `404` means only "has *some* scope for this resource" — read or write — and every write probe was reporting a false positive. The agent's token was consequently reported as holding `Contents: write` when it holds nothing of the sort. Two sound methods now: - GitHub's own `permissions` object for the repository, which needs no inference. `push: true` is exactly `Contents: write` — the scope carrying merge, branch pushes and `POST /releases` together. - **Idempotent writes** for the scopes that object does not describe. The probe reads the current title and writes it straight back, so nothing changes, but only a real write scope returns `2xx`. Against an admin token all three answer `200`, confirming the probes discriminate. `403` is still meaningful for a scope the credential lacks entirely — Actions and Secrets keep using it, and they correctly report ABSENT for the PAT. The agent check now decides on *identity* instead of probing scopes: `github-actions[bot]` means `github.token`, whose reach the workflow's `permissions:` block sets, so read-only is guaranteed by construction rather than measured. That is the fact worth asserting, and it cannot be faked by a probe artifact. --- scripts/loop/gh-check.mjs | 210 +++++++++++++++++++++----------------- 1 file changed, 114 insertions(+), 96 deletions(-) diff --git a/scripts/loop/gh-check.mjs b/scripts/loop/gh-check.mjs index ae9669f..44ddad5 100644 --- a/scripts/loop/gh-check.mjs +++ b/scripts/loop/gh-check.mjs @@ -8,21 +8,32 @@ * moment a real PAT is configured it replaces the job token for **every** `gh` * call, including the read-only ones sweep depends on. A PAT that is narrower * than the job token therefore breaks reads silently, and one that is broader - * silently hands the agent write access nobody chose. Neither is visible from - * the workflow file, so it has to be measured. + * silently widens what the loop can do. Neither is visible from the workflow + * file, so it has to be measured. * - * Permission probing without side effects: a write endpoint is called against a - * resource number that cannot exist. `403` means the token lacks the scope; - * `404`/`422` means the request passed the permission check and only failed on - * the missing resource — i.e. the scope is present. Nothing is ever mutated, - * because every probe targets an id far beyond this repository's range. + * How permissions are measured — two methods, because the obvious one is wrong: + * + * 1. GitHub's own `permissions` object for the credential. No inference. + * 2. **Idempotent writes** for scopes the object does not describe (Issues and + * Pull requests). The probe reads the current value and writes it straight + * back, so nothing changes — but a `2xx` means the write really executed, + * which only happens with the write scope. + * + * The tempting method — call a write endpoint against an impossible id and read + * `403` as "no permission", `404` as "permission present" — does **not** work. + * Measured against a `github.token` capped to read-only by `permissions:`, the + * write probes returned `404`, not `403`. So `404` means only "the credential + * has *some* scope for this resource", read or write, and the method reports + * false positives. `403` remains meaningful for a scope the token lacks + * entirely, which is why the Actions and Secrets checks still use it. * * The token itself is never printed. Only the login it authenticates as. */ import { spawnSync } from 'node:child_process'; -const IMPOSSIBLE = '999999999'; // far beyond any real issue/PR number here +const IMPOSSIBLE = '999999999'; // beyond any real issue/PR number here +const SEP = '\u0001'; // unlikely in a real title /** Run `gh`, returning the HTTP status it saw and whether it succeeded. */ function gh(args, { token = null } = {}) { @@ -39,18 +50,28 @@ function gh(args, { token = null } = {}) { }; } +/** A scope the credential lacks entirely answers `403`; anything else is "has some access". */ +const hasAnyScope = (status) => status !== null && status !== 403 && status !== 401; + +/** Identity is the decisive fact for a credential we did not choose. */ +function identity(token) { + const r = gh(['api', 'user', '--jq', '.login'], { token }); + return r.ok ? r.out : null; +} + /** - * `403`/`401` = scope absent. Anything else means the permission check passed - * and the request failed later (missing resource, bad body) — scope present. + * Write nothing, change nothing — but require the write scope to succeed. + * Returns `true` (2xx), `false` (refused), or `null` (no target to probe with). */ -const hasScope = (status) => status !== null && status !== 403 && status !== 401; - -function probe(label, args) { - const r = gh(args); - const present = hasScope(r.status); - console.log(` ${present ? 'present' : 'ABSENT '} ${label}` - + ` (HTTP ${r.status ?? 'n/a'})`); - return { label, present, status: r.status, err: r.err }; +function idempotentWrite(label, { read, write }, token) { + const cur = gh(read, { token }); + if (!cur.ok || !cur.out) { + console.log(` UNKNOWN ${label} (nothing to probe against)`); + return null; + } + const w = gh(write(cur.out), { token }); + console.log(` ${w.ok ? 'present' : 'ABSENT '} ${label} (HTTP ${w.status ?? 'n/a'})`); + return w.ok; } const repo = process.argv[2] || process.env.GITHUB_REPOSITORY || ''; @@ -59,25 +80,25 @@ if (!repo) { process.exit(1); } +const hostToken = process.env.GH_TOKEN || null; +const agentToken = process.env.GH_READ_TOKEN || null; + console.log(`[loop:gh] repo: ${repo}`); -const who = gh(['api', 'user', '--jq', '.login']); -if (!who.ok) { +const hostWho = identity(hostToken); +if (!hostWho) { // A token that cannot even identify itself is the one hard failure: every // later `gh` call in the job would fail too, so fail loudly and early. - console.error(`[loop:gh] error: token cannot authenticate — ${who.err || who.status}`); + console.error('[loop:gh] error: GH_TOKEN cannot authenticate'); process.exit(1); } -console.log(`[loop:gh] token authenticates as: ${who.out}`); -const isJobToken = who.out === 'github-actions[bot]'; -// Only claim which credential this is inside Actions: run locally, `gh` uses -// whatever the developer is logged in as, and naming LOOP_GH_TOKEN there would -// be plainly wrong. -if (process.env.GITHUB_ACTIONS) { - console.log(`[loop:gh] credential: ${isJobToken - ? 'github.token (job token — LOOP_GH_TOKEN is unset or empty)' - : 'LOOP_GH_TOKEN (a PAT, replacing the job token for every gh call)'}`); -} +const hostIsJob = hostWho === 'github-actions[bot]'; +// Locally there is no GH_TOKEN — `gh` falls back to the developer's own login, +// so saying "github.token" there would be plainly wrong. +const hostSource = !hostToken + ? '(GH_TOKEN unset — gh used its own local login)' + : hostIsJob ? 'github.token (job token)' : 'LOOP_GH_TOKEN (a PAT, replacing the job token)'; +console.log(`[loop:gh] host credential: ${hostSource} — authenticates as ${hostWho}`); console.log('\n[loop:gh] reads — the host depends on these; a miss breaks sweep:'); // sweep lists open issues/PRs and calls `gh release list`; repo metadata is the @@ -89,57 +110,69 @@ const reads = [ ['releases list (sweep)', ['release', 'list', '-R', repo, '--limit', '1']], ]; const readResults = reads.map(([label, args]) => { - const r = gh(args); + const r = gh(args, { token: hostToken }); console.log(` ${r.ok ? 'OK ' : 'FAILED '} ${label}` + `${r.ok ? '' : ` — ${(r.err || r.status).toString().slice(0, 90)}`}`); return { label, ok: r.ok }; }); -console.log('\n[loop:gh] writes the loop needs (probes target an impossible id — nothing changes):'); -const needed = [ - ['Issues: write (label/comment/close)', ['api', '--method', 'PATCH', - `repos/${repo}/issues/${IMPOSSIBLE}`, '-f', 'state=open']], - ['Pull requests: write (open/edit PR, reviews)', ['api', '--method', 'PATCH', - `repos/${repo}/pulls/${IMPOSSIBLE}`, '-f', 'state=open']], -]; -const needResults = needed.map(([label, args]) => probe(label, args)); +console.log('\n[loop:gh] host write scopes (idempotent probes — each restores the value it read):'); +const issueProbe = { + read: ['api', `repos/${repo}/issues?state=open&per_page=1`, '--jq', `.[0] | "\\(.number)${SEP}\\(.title)"`], + write: (cur) => { + const [n, title] = cur.split(SEP); + return ['api', '--method', 'PATCH', `repos/${repo}/issues/${n}`, '-f', `title=${title}`]; + }, +}; +const prProbe = { + read: ['api', `repos/${repo}/pulls?state=open&per_page=1`, '--jq', `.[0] | "\\(.number)${SEP}\\(.title)"`], + write: (cur) => { + const [n, title] = cur.split(SEP); + return ['api', '--method', 'PATCH', `repos/${repo}/pulls/${n}`, '-f', `title=${title}`]; + }, +}; +const canLabel = idempotentWrite('Issues: write (label/comment/close)', issueProbe, hostToken); +idempotentWrite('Pull requests: write (open/edit PR, reviews)', prProbe, hostToken); + +// `permissions.push` is the repository-write flag, which on GitHub's model is +// exactly `Contents: write` — the scope that carries merging, branch pushes and +// `POST /releases` together. It is read from GitHub rather than probed. +const hostPerm = gh(['api', `repos/${repo}`, '--jq', '.permissions'], { token: hostToken }); +let canPush = null; +if (hostPerm.ok && hostPerm.out) { + try { + canPush = JSON.parse(hostPerm.out).push === true; + } catch { /* leave null */ } + console.log(` ${canPush ? 'present' : 'ABSENT '} Contents: write (MERGE + push + release)`); +} else { + console.log(' UNKNOWN Contents: write (repo permissions unreadable)'); +} -console.log('\n[loop:gh] scopes the loop must NOT hold:'); -const forbidden = [ - ['Contents: write (MERGE + push + release)', ['api', '--method', 'PUT', - `repos/${repo}/pulls/${IMPOSSIBLE}/merge`]], +console.log('\n[loop:gh] scopes reached by a credential that lacks them entirely:'); +const absent = [ ['Actions: write (trigger/disable workflows)', ['api', '--method', 'POST', `repos/${repo}/actions/workflows/${IMPOSSIBLE}/dispatches`, '-f', 'ref=main']], ]; -const forbiddenResults = forbidden.map(([label, args]) => probe(label, args)); -// Independent confirmation of the probes above: GitHub's own `permissions` -// object, which needs no inference from a status code. Relevant because the -// `403` vs `404` reading assumes GitHub checks permissions *before* resource -// lookup, which is worth verifying rather than assuming. -const hostPerm = gh(['api', `repos/${repo}`, '--jq', '.permissions']); -console.log(` repo permissions: ${hostPerm.ok - ? hostPerm.out.replace(/\s+/g, ' ') - : `(unreadable — ${(hostPerm.err || hostPerm.status).toString().slice(0, 60)})`}`); -// Read-only probe: listing secret *names* needs the Secrets permission. The +for (const [label, args] of absent) { + const r = gh(args, { token: hostToken }); + console.log(` ${hasAnyScope(r.status) ? 'present' : 'ABSENT '} ${label} (HTTP ${r.status ?? 'n/a'})`); +} +// Read-only probe: listing secret *names* requires the Secrets permission. The // listing itself is never printed — only whether the permission is present. -const secrets = gh(['api', `repos/${repo}/actions/secrets`]); +const secrets = gh(['api', `repos/${repo}/actions/secrets`], { token: hostToken }); console.log(` ${secrets.ok ? 'present' : 'ABSENT '} Secrets access (list/overwrite CI secrets)`); -const held = forbiddenResults.filter((r) => r.present).map((r) => r.label.trim().split(/\s+/)[0]); const brokenReads = readResults.filter((r) => !r.ok); -const canLabel = needResults.find((r) => r.label.startsWith('Issues'))?.present ?? false; - console.log(''); -if (held.length) { - console.log(`[loop:gh] WARNING: the token holds ${[...held, ...(secrets.ok ? ['Secrets'] : [])].join(', ')}.` - + ' Each of those is a way to reach code or CI without going through a pull request —' - + ' Contents alone grants merging (PUT /pulls/{n}/merge), branch pushes and POST /releases.'); - console.log('[loop:gh] Recreate the PAT as: Issues Read and write, Pull requests Read and write,' - + ' Contents Read-only, and Nothing for Actions/Secrets/Administration/Workflows.' - + ' See "GitHub write credential" in scripts/loop/README.md.'); -} else { - console.log('[loop:gh] none of the forbidden scopes are held — merging, branch pushes and' - + ' workflow control are impossible for this credential, which is the intended L2c property.'); +if (canPush) { + console.log('[loop:gh] WARNING: the host holds Contents:write. That single scope grants merging' + + ' (PUT /pulls/{n}/merge), branch pushes and POST /releases — none of which the loop needs' + + ' until L2. See "GitHub write credential" in scripts/loop/README.md. Recreate the PAT with:' + + ' Issues Read and write, Pull requests Read and write, Contents Read-only, and Nothing for' + + ' Actions/Secrets/Administration/Workflows.'); +} else if (canPush === false) { + console.log('[loop:gh] host holds no Contents:write — merging, branch pushes and releases are' + + ' impossible for it.'); } console.log(`[loop:gh] label/comment/close: ${canLabel ? 'available' : 'NOT available — the loop cannot do its job'}`); console.log(`[loop:gh] reads: ${brokenReads.length @@ -147,36 +180,21 @@ console.log(`[loop:gh] reads: ${brokenReads.length : 'all OK'}`); // The credential the *agent* will hold. `runPi` swaps GH_TOKEN for GH_READ_TOKEN -// under the report boundary, so this is the one that decides whether the agent -// could write if it ignored its prompt. Measuring it here makes that property -// observable without spending a single token on a real agent run. -const agentToken = process.env.GH_READ_TOKEN; +// under the report boundary, so this decides whether the agent could write if it +// ignored its prompt. Identity is the decisive check: `github-actions[bot]` is +// the job token, whose reach the workflow's `permissions:` block sets, and that +// block is read-only here. console.log('\n[loop:gh] credential handed to the agent under the report boundary:'); if (!agentToken) { console.log(' (GH_READ_TOKEN is not set — runPi would pass GH_TOKEN through unchanged,' - + ' i.e. the agent inherits the host credential above)'); + + ' i.e. the agent inherits the host credential above.)'); } else { - const agentProbes = [ - ['Issues: write', ['api', '--method', 'PATCH', `repos/${repo}/issues/${IMPOSSIBLE}`, '-f', 'state=open']], - ['Contents: write (merge/push)', ['api', '--method', 'PUT', `repos/${repo}/pulls/${IMPOSSIBLE}/merge`]], - ]; - const agentHeld = []; - for (const [label, args] of agentProbes) { - const r = gh(args, { token: agentToken }); - const present = hasScope(r.status); - if (present) agentHeld.push(label); - console.log(` ${present ? 'present' : 'ABSENT '} ${label}`); - } - // Cross-check the probe, because `403` vs `404` is an inference about where - // GitHub checks permissions relative to resource lookup. The `permissions` - // object is reported by GitHub itself and needs no inference. - const agentPerm = gh(['api', `repos/${repo}`, '--jq', '.permissions'], { token: agentToken }); - console.log(` repo permissions: ${agentPerm.ok - ? agentPerm.out.replace(/\s+/g, ' ') - : `(unreadable — ${(agentPerm.err || agentPerm.status).toString().slice(0, 60)})`}`); - console.log(agentHeld.length - ? ` WARNING: the agent holds ${agentHeld.join(', ')} — the report boundary is back to being` - + ' a prompt instruction only.' - : ' OK: the agent holds no write scope, so even ignoring its prompt it reaches GitHub' - + ' read-only. The report boundary is a platform guarantee.'); + const agentWho = identity(agentToken); + const agentIsJob = agentWho === 'github-actions[bot]'; + console.log(` authenticates as: ${agentWho ?? '(failed)'}`); + console.log(agentIsJob + ? ' OK: it is github.token, so `permissions:` caps it — the agent reaches GitHub read-only' + + ' even if it ignores its prompt. The report boundary is a platform guarantee.' + : ` WARNING: it is ${agentWho}, a user token — the agent inherits that user's reach and the` + + ' report boundary is back to being a prompt instruction only.'); } From 6160db1f5fd8e5638fda0c829e3a72b613b02e3a Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:34:30 +0800 Subject: [PATCH 8/9] chore(loop): check the agent credential on signals that actually exist Identity was the wrong signal. `github.token` is a GitHub App installation token and `GET /user` refuses it outright, so the check reported "(failed)" rather than distinguishing a job token from a PAT. Uses two signals that are observable instead: whether the credential can read the repository at all, and whether an idempotent write is refused. A PAT shows up as "acts as a user"; a read-only job token reads fine but cannot write. That combination is what the report boundary depends on, and it is now asserted directly rather than inferred from an identity lookup that cannot work. --- scripts/loop/gh-check.mjs | 36 +++++++++++++++++++++++++----------- 1 file changed, 25 insertions(+), 11 deletions(-) diff --git a/scripts/loop/gh-check.mjs b/scripts/loop/gh-check.mjs index 44ddad5..65ba998 100644 --- a/scripts/loop/gh-check.mjs +++ b/scripts/loop/gh-check.mjs @@ -181,20 +181,34 @@ console.log(`[loop:gh] reads: ${brokenReads.length // The credential the *agent* will hold. `runPi` swaps GH_TOKEN for GH_READ_TOKEN // under the report boundary, so this decides whether the agent could write if it -// ignored its prompt. Identity is the decisive check: `github-actions[bot]` is -// the job token, whose reach the workflow's `permissions:` block sets, and that -// block is read-only here. +// ignored its prompt. +// +// Identity cannot be read from `GET /user` here: `github.token` is a GitHub App +// installation token and is refused there entirely, which is why the check below +// combines two signals that *are* observable — whether the credential can read +// the repository at all, and whether a write is refused. console.log('\n[loop:gh] credential handed to the agent under the report boundary:'); if (!agentToken) { console.log(' (GH_READ_TOKEN is not set — runPi would pass GH_TOKEN through unchanged,' + ' i.e. the agent inherits the host credential above.)'); } else { - const agentWho = identity(agentToken); - const agentIsJob = agentWho === 'github-actions[bot]'; - console.log(` authenticates as: ${agentWho ?? '(failed)'}`); - console.log(agentIsJob - ? ' OK: it is github.token, so `permissions:` caps it — the agent reaches GitHub read-only' - + ' even if it ignores its prompt. The report boundary is a platform guarantee.' - : ` WARNING: it is ${agentWho}, a user token — the agent inherits that user's reach and the` - + ' report boundary is back to being a prompt instruction only.'); + const canRead = gh(['api', `repos/${repo}`, '--jq', '.full_name'], { token: agentToken }); + const asUser = gh(['api', 'user', '--jq', '.login'], { token: agentToken }); + console.log(` can read the repo: ${canRead.ok ? 'yes' : `NO — the agent cannot read GitHub (${canRead.status})`}`); + console.log(` acts as a user: ${asUser.ok ? `yes (${asUser.out}) — a PAT or user token` : 'no — a GitHub App token, i.e. github.token'}`); + const agentWrite = idempotentWrite('Issues: write', issueProbe, agentToken); + + if (!canRead.ok) { + console.log(' WARNING: the agent cannot read GitHub, so triage/sweep will find nothing.'); + } else if (asUser.ok) { + console.log(` WARNING: the agent holds a *user* token (${asUser.out}) — it inherits that` + + " user's reach, so the report boundary is a prompt instruction only."); + } else if (agentWrite) { + console.log(' WARNING: the agent can write (github.token is not capped to read — check the' + + " workflow's `permissions:` block)."); + } else { + console.log(' OK: github.token, read-only by the workflow\'s `permissions:` block — the agent' + + ' reaches GitHub read-only even if it ignores its prompt. The report boundary is a' + + ' platform guarantee.'); + } } From 86dbb07bf92c8500dbd0066acf059eefef2cb1c1 Mon Sep 17 00:00:00 2001 From: Alex Chao Date: Fri, 11 Sep 2026 23:47:47 +0800 Subject: [PATCH 9/9] chore(loop): measure Contents with a real write, not the repo permissions object MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The Contents check read `.permissions.push` from `GET /repos/{owner}/{repo}` and treated it as a token scope report. It is not: that object describes the *authenticated user's* role on the repository, so it answers `push: true` for any token belonging to an admin — however narrowly that token was scoped. The result was a false WARNING claiming a Contents Read-only PAT could merge and push, plus an instruction to recreate a token that was already correct. Reported by the owner, who had set Contents to Read-only. Contents is now probed like the other scopes: read the default branch's SHA and `PATCH /git/refs/heads/{branch}` with that same SHA. Setting a ref to the value it already holds is a no-op, but only a real write scope returns 2xx — verified locally (the ref came back unchanged after a 200) and on Actions, where the PAT now correctly reports `ABSENT ... (HTTP 403)`. The `.permissions` object is still printed, labelled as the user's role, so the next reader does not repeat the mistake. The agent check now probes Contents too: it is exactly the scope that carries merge and push, so it is the one worth asserting about the credential the agent holds. Both report ABSENT (HTTP 403) there. --- scripts/loop/gh-check.mjs | 36 ++++++++++++++++++++++++------------ 1 file changed, 24 insertions(+), 12 deletions(-) diff --git a/scripts/loop/gh-check.mjs b/scripts/loop/gh-check.mjs index 65ba998..13141f4 100644 --- a/scripts/loop/gh-check.mjs +++ b/scripts/loop/gh-check.mjs @@ -134,18 +134,29 @@ const prProbe = { const canLabel = idempotentWrite('Issues: write (label/comment/close)', issueProbe, hostToken); idempotentWrite('Pull requests: write (open/edit PR, reviews)', prProbe, hostToken); -// `permissions.push` is the repository-write flag, which on GitHub's model is -// exactly `Contents: write` — the scope that carries merging, branch pushes and -// `POST /releases` together. It is read from GitHub rather than probed. +// `PATCH /git/refs/{ref}` is in the Contents write set, and setting a ref to the +// SHA it already has is a no-op — so a 2xx means real write ability and nothing +// changes. +// +// This is the only sound way to measure Contents. The repository's +// `.permissions` object is NOT a token scope report: it describes the +// *authenticated user's* role, so it answers `push: true` for any token +// belonging to an admin, no matter how narrowly the token was scoped. Reading it +// here reported a Contents Read-only PAT as write-capable. +const defaultBranch = gh(['api', `repos/${repo}`, '--jq', '.default_branch'], + { token: hostToken }).out || 'main'; +const refProbe = { + read: ['api', `repos/${repo}/git/refs/heads/${defaultBranch}`, '--jq', '.object.sha'], + write: (sha) => ['api', '--method', 'PATCH', + `repos/${repo}/git/refs/heads/${defaultBranch}`, '-f', `sha=${sha}`], +}; +const canPush = idempotentWrite( + `Contents: write (MERGE + push + release)`, refProbe, hostToken, +); const hostPerm = gh(['api', `repos/${repo}`, '--jq', '.permissions'], { token: hostToken }); -let canPush = null; -if (hostPerm.ok && hostPerm.out) { - try { - canPush = JSON.parse(hostPerm.out).push === true; - } catch { /* leave null */ } - console.log(` ${canPush ? 'present' : 'ABSENT '} Contents: write (MERGE + push + release)`); -} else { - console.log(' UNKNOWN Contents: write (repo permissions unreadable)'); +if (hostPerm.ok) { + console.log(` (info) repo .permissions = ${hostPerm.out.replace(/\s+/g, ' ')}` + + ' — the *user\'s* role, not this token\'s scopes; do not read it as a scope report'); } console.log('\n[loop:gh] scopes reached by a credential that lacks them entirely:'); @@ -197,13 +208,14 @@ if (!agentToken) { console.log(` can read the repo: ${canRead.ok ? 'yes' : `NO — the agent cannot read GitHub (${canRead.status})`}`); console.log(` acts as a user: ${asUser.ok ? `yes (${asUser.out}) — a PAT or user token` : 'no — a GitHub App token, i.e. github.token'}`); const agentWrite = idempotentWrite('Issues: write', issueProbe, agentToken); + const agentPush = idempotentWrite('Contents: write', refProbe, agentToken); if (!canRead.ok) { console.log(' WARNING: the agent cannot read GitHub, so triage/sweep will find nothing.'); } else if (asUser.ok) { console.log(` WARNING: the agent holds a *user* token (${asUser.out}) — it inherits that` + " user's reach, so the report boundary is a prompt instruction only."); - } else if (agentWrite) { + } else if (agentWrite || agentPush) { console.log(' WARNING: the agent can write (github.token is not capped to read — check the' + " workflow's `permissions:` block)."); } else {