audit: re-derive doctrine §4 from evidence the first run actually produced

.claude-plugin/marketplace.json

@@ -21,7 +21,7 @@
     {
       "name": "audit",
       "source": "./audit",
-      "version": "0.1.3",
+      "version": "0.1.4",
       "description": "Bulk, read-only audits of a QuantEcon repository — issue triage, PR review, technical debt, translation parity — each producing an evidence-cited report bundle"
     }
   ]

audit/.claude-plugin/plugin.json

@@ -1,6 +1,6 @@
 {
   "name": "audit",
   "description": "Bulk, read-only audits of a QuantEcon repository — issue triage, PR review, technical debt, translation parity — each producing an evidence-cited report bundle",
-  "version": "0.1.3",
+  "version": "0.1.4",
   "author": { "name": "QuantEcon" }
 }

audit/CHANGELOG.md

@@ -6,6 +6,17 @@ Versions are [semver](https://semver.org) as a user of this plugin experiences i
 
 Repository: [QuantEcon/skills](https://github.com/QuantEcon/skills) ([every commit that touched this plugin](https://github.com/QuantEcon/skills/commits/main/audit)). How a release is made: [developing-skills § Versioning and releases](https://github.com/QuantEcon/skills/blob/main/docs/developing-skills.md#versioning-and-releases).
 
+## 0.1.4 — 2026-08-03
+
+Doctrine §4's rule survived the first measured run; its justification did not. The section is re-derived from what that run actually produced, and the cost figures the skill quotes are replaced with measured ones.
+
+**Changed**
+
+- Doctrine §4 is renamed from "Surviving a long run" to "Checkpointing", and rests on three reasons that hold at any duration rather than on the claim that audits outlive sessions. The first measured run refuted that claim outright — 230 items in 22 minutes, with no context exhaustion, rate limit or sleeping machine in play. The strongest replacement reason is checkable: the per-item log is what the final enumeration is assembled *from*, and what a reviewer counts the coverage numbers against.
+- `/audit:issues` no longer describes itself as "long-running by design — a hundred-issue repo is a multi-hour run". It now quotes the measured cost: roughly **10 seconds per open issue**, with a 230-item tracker carrying 56 open issues taking 22 minutes. Cost tracks open issues needing verification rather than total items, so a large tracker with a small open set is cheaper than a small one with a large set.
+- The cost figures are stated so the two measures cannot be confused. Previously a reader met "roughly 10 seconds per open issue" beside "a 230-item tracker with 56 open took 22 minutes" and could not reconcile them — 56 × 10 s is 9 minutes, not 22. The 10-second rate is phase 2 alone; 22 minutes is end to end, and the remaining phases are largely fixed. Both numbers now say which question they answer.
+- Checkpoint artifacts are named where they carry evidence rather than at every phase boundary out of symmetry — a checkpoint written and superseded minutes later without ever being read earns nothing.
+
 ## 0.1.3 — 2026-08-03
 
 **Added**

audit/README.md

@@ -41,7 +41,9 @@ Skills are thin; the method lives once at plugin level.
 /audit:issues QuantEcon/action-translation
 ```
 
-Audits are long. They work from a frozen snapshot and checkpoint each phase to disk, so a run that loses its session resumes at the last completed phase rather than restarting — and every number in the report refers to one point in time. Expect hours, not minutes, on a repo with a hundred items.
+Audits work from a frozen snapshot and checkpoint to disk as they go, so an interrupted run resumes rather than restarting — and every number in the report refers to one point in time.
+
+**Budget tens of minutes, not hours.** The first measured run covered a 230-item tracker in **22 minutes** end to end ([record](https://github.com/QuantEcon/skills/blob/main/reviews/audit-run-action-translation-2026-07-28.md)). What scales is the **open issue** count, not the item count: phase 2 verifies the open set at about 10 seconds each — 9 of those 22 minutes for 56 issues — while the remaining phases are largely fixed. So a 1000-item repo with a small open set is cheaper than a 300-item repo with a large one. That is one data point; a repo whose issues need deeper code archaeology will run slower per issue.
 
 Headless runs work the same way:
 

audit/references/doctrine.md

@@ -35,9 +35,17 @@ That boundary is what makes the family safe to run headlessly and safe to point
 
 Corollary: an audit never writes into the plugin directory, and its outputs never live in `QuantEcon/skills`. See [deliverables.md](deliverables.md) for where they go.
 
-## 4. Surviving a long run
+## 4. Checkpointing
 
-Bulk audits outlive sessions. Context runs out, rate limits bite, machines sleep. One rule follows from that, and it is about checkpointing rather than about structure: **write each phase's output to the working directory before starting the next**, so a lost session resumes where it stopped instead of restarting. How a skill divides itself into phases is its own business — the division below is one that worked, not a template to fill.
+An audit's intermediate work is worth writing down — but not for the reason this section used to give. It claimed bulk audits outlive sessions: context running out, rate limits biting, machines sleeping. The first measured run refuted all three at once — 230 items in 22 minutes, with none of those mechanisms in play ([run record](https://github.com/QuantEcon/skills/blob/main/reviews/audit-run-action-translation-2026-07-28.md)). The rule survived the measurement; its justification did not, and a doctrine that demands evidence for every claim owes one here.
+
+**Write each phase's output to the working directory before starting the next**, for three reasons that hold at any duration:
+
+1. **The checkpoint is evidence, not insurance.** A per-item verification log is what the final enumeration is assembled *from*, and what a reviewer counts against the coverage numbers — run 1's 56-of-56 reconciliation was done against the log, not the report. This is the strongest of the three, and unlike the claim it replaces it can be checked.
+2. **Interruption does not care how long the run is.** A cancelled session, a tool error, a rate limit. Twenty-two minutes of item-by-item judgement is still expensive to re-derive.
+3. **The cost is asymmetric.** A line per item is nearly free; the phase is not.
+
+How a skill divides itself into phases is its own business — the division below is one that worked, not a template to fill. Note what reason 1 implies, though: a checkpoint written and then superseded minutes later without ever being read earns nothing. Name artifacts where they carry evidence, not at every phase boundary out of symmetry.
 
 Two properties separate a checkpoint from a claim about one, and a skill that promises resumability owes both. The artifact needs a **name the next session can find without guessing** — an unnamed intermediate is only resumable if two sessions independently invent the same file. And a phase that iterates over many items must **append as it works, not write when it finishes**, because the phase long enough to be worth checkpointing is the phase a run dies *inside*. Output that exists only on completion is no checkpoint at all, exactly where one was needed.
 

audit/scripts/README.md

@@ -1,6 +1,6 @@
 # Deterministic audit machinery
 
-Fetching is mechanical, so it belongs here rather than in model judgement ([doctrine §4](../references/doctrine.md#4-surviving-a-long-run)). Stdlib only, driving `gh`; no install step.
+Fetching is mechanical, so it belongs here rather than in model judgement ([doctrine §4](../references/doctrine.md#4-checkpointing)). Stdlib only, driving `gh`; no install step.
 
 ## `fetch_tracker.py`
 

audit/skills/issues/SKILL.md

@@ -5,7 +5,7 @@ description: "Audit every issue in a GitHub repository, open and closed — veri
 
 # audit:issues
 
-Whole-tracker review of one repository. Long-running by design — a hundred-issue repo is a multi-hour run — so it works from a frozen snapshot and checkpoints every phase to disk.
+Whole-tracker review of one repository. It works from a frozen snapshot and checkpoints as it goes. One measured run, on a 230-item tracker with 56 open issues: **22 minutes end to end**, of which phase 2 — verifying those 56 — was about 9, or roughly **10 seconds per open issue**. The two figures answer different questions: budget the total from the whole run, and scale it by the *open* count, since that is what phase 2 walks. Phase 2 also carries nearly all of the judgement, which is what makes a run expensive to re-derive rather than long to sit through.
 
 > **Status: first runbook of the `audit` family.** The procedure below is the QuantEcon-adapted form of a runbook that has been executed once end to end; the gap that execution found (closed threads unread) is now doctrine rule 3 and is closed by the snapshot. Plan and open questions: [issue #12](https://github.com/QuantEcon/skills/issues/12).
 
@@ -51,7 +51,7 @@ State which was used in the report's method section. Everything the run produces
 | `links.md` | phase 3 | the cross-link graph |
 | `01-…` `02-…` `03-…` `README.md` | phase 4 | the delivered bundle |
 
-**Append to the checkpoint as you go, not when the phase ends** ([doctrine §4](../../references/doctrine.md#4-surviving-a-long-run)). Phase 2 is the long one — a hundred items of item-by-item judgement — so it is the phase a run dies inside rather than between. On restart, read `findings.md` and resume at the lowest number in `issues.json` that has no entry there; re-verify the last entry rather than trusting a possibly truncated write.
+**Append to the checkpoint as you go, not when the phase ends** ([doctrine §4](../../references/doctrine.md#4-checkpointing)). Phase 2 carries nearly all the judgement, so it is both the phase an interruption lands inside rather than between, and the phase whose log the catalog is later assembled from. On restart, read `findings.md` and resume at the lowest number in `issues.json` that has no entry there; re-verify the last entry rather than trusting a possibly truncated write.
 
 ## Phase 1 — snapshot
 

docs/tutorial-run-an-audit.md

@@ -2,7 +2,7 @@
 
 This walks `/audit:issues` end to end against **[QuantEcon/action-translation](https://github.com/QuantEcon/action-translation)** — 228 items, the repo the runbook was first executed against by hand.
 
-It differs from the [evaluation tutorial](tutorial-run-an-evaluation.md) in one important way. That one reproduces a committed reference, so every number you produce can be checked. Here there is no reference: `/audit:issues` has **never been run as a skill**, only by hand as a runbook, so its method is generalised from a single execution. This run *is* the experiment ([skills#16](https://github.com/QuantEcon/skills/issues/16)), and the part no automation can supply is your judgement of the output. Step 6 is therefore not optional garnish — it is the result.
+It differs from the [evaluation tutorial](tutorial-run-an-evaluation.md) in one important way. That one reproduces a committed reference, so every number you produce can be checked. Here there is no reference: `/audit:issues` has been run as a skill exactly **once** — run 1, against this same repo on 2026-07-28, which found seven plugin defects and is recorded [here](https://github.com/QuantEcon/skills/blob/main/reviews/audit-run-action-translation-2026-07-28.md). A method generalised from one execution is still a hypothesis, so your run is the next data point in the validation program ([skills#16](https://github.com/QuantEcon/skills/issues/16)), and the part no automation can supply is your judgement of the output. Step 6 is therefore not optional garnish — it is the result.
 
 Canonical references (this tutorial points, never restates): the procedure in [SKILL.md](../audit/skills/issues/SKILL.md), the method in [doctrine.md](../audit/references/doctrine.md), the org conventions in [quantecon-context.md](../audit/references/quantecon-context.md), the output contract in [deliverables.md](../audit/references/deliverables.md).
 
@@ -11,7 +11,7 @@ Canonical references (this tutorial points, never restates): the procedure in [S
 - The `audit` plugin installed (Step 0). It is deliberately not in the lecture-repo auto-install block — it is maintainer tooling.
 - `gh` authenticated. Preflight refuses to start without it, because the anonymous API returns nothing for the org's private repos and is capped at 60 requests/hour.
 - A checkout of the audited repo. Phase 2 verifies claims against its default branch, so a tracker-only run cannot do the job.
-- **Hours, not minutes**, and a session you can afford to interrupt — interrupting it is one of the tests.
+- **Tens of minutes**, and a session you can afford to interrupt — interrupting it is one of the tests. Run 1 took 22 minutes end to end for 230 items, of which phase 2 was about 9 — roughly 10 seconds for each of its 56 open issues. Budget the total, and scale it by the *open* count rather than the item count.
 
 ## Step 0 — install the plugin
 
@@ -22,7 +22,7 @@ claude plugin install audit@quantecon
 
 **Then restart your session** — plugins register at startup, so the skill does not appear until you reopen.
 
-The `/plugin marketplace add …` slash form does the same job, but it is a *terminal-CLI built-in*: the VS Code extension and the web app answer `/plugin isn't available in this environment`, while the `claude plugin` CLI above works from any shell. Confirm with `claude plugin list` — the version should read **0.1.2**.
+The `/plugin marketplace add …` slash form does the same job, but it is a *terminal-CLI built-in*: the VS Code extension and the web app answer `/plugin isn't available in this environment`, while the `claude plugin` CLI above works from any shell. Confirm with `claude plugin list` — the version it reports should match the `audit` entry in [`marketplace.json`](../.claude-plugin/marketplace.json). (Naming a number here would go stale on the next release; if the two disagree, the install did not pick up the latest — `claude plugin update audit@quantecon`.)
 
 If `/audit:issues` is still unrecognised after restarting, the plugin-prefixed slash form needs Claude Code 2.1.216+; the bare `/issues` works on older builds, and natural-language invocation ("audit every issue in this repo, output to …") works on any version ([using-skills § troubleshooting](using-skills.md#updating-and-troubleshooting)).
 
@@ -108,7 +108,7 @@ Then confirm the boundary held: `git status --short` shows nothing but your igno
 Findings belong in this repo; the bundle does not. Write `reviews/audit-run-action-translation-2026-07-28.md`, following the [ge_arrow validation run](../reviews/validation-run-ge_arrow-2026-07-22.md):
 
 - **Setup** — repo, snapshot timestamp, `fetched_by`, item counts, unaccounted numbers, plugin version.
-- **Cost** — wall clock and rough token spend per phase, so "expect hours" stops being a guess.
+- **Cost** — wall clock and rough token spend per phase. Run 1's figures are in [its record](https://github.com/QuantEcon/skills/blob/main/reviews/audit-run-action-translation-2026-07-28.md); a second data point at a different repo type is what turns one measurement into an estimate.
 - **The ten checks** from Step 6, each held or broken, with the evidence.
 - **Interruption log** — where you killed it, what resuming actually did.
 - **What the doctrine did and did not transfer** — the payload. A rule that was cited and load-bearing, a rule that never came up, and a rule the run had to work around are three different verdicts, and only the third is a bug.