artifact-audit: Sep 2026 pass — billing-claim drift, native dollar ceiling, ralph caveats (3 fixed, 8 flagged)

[espi]

Artifact drift

Monthly artifact-audit pass (2026-09-25), run per .claude/skills/artifact-audit/SKILL.md. Fan-out: 4 audit agents (skills / templates / guardrails / runbooks+docs), every proposed fix re-verified against the KB by the coordinating session before applying. Source of truth: knowledge/ as merged on main (last substantive update 2026-08-31).

Yields honoured: open PR #21 is mid-editing loop-guardrails/SKILL.md, update-knowledge/SKILL.md, artifact-audit/SKILL.md, guardrails/README.md, guardrails/checklist.md, and runbooks/staying-current.md — none of those files are touched here; their findings are flagged below instead. Open PRs #22/#23 touch only knowledge/ (never edited by this skill); findings resting solely on their unmerged content are marked pending-KB and not applied.

Fixed

File	What was stale	Evidence
.claude/skills/new-loop/SKILL.md	Step 3's budget bullet offered "remind the user of the session budget" as the fallback ceiling — an alert, not enforcement. Now names --max-budget-usd (halts background subagents since v2.1.217).	primer §4, v2.1.216–220 entry (High, re-verified in primer directly)
.claude/skills/new-loop/SKILL.md	The ralph row named the ralph-wiggum plugin without its trap: --max-iterations defaults to unlimited.	primer §4 + sources.md plugin README entry (High)
README.md	"The billing layer will not save you (soft alerts only)" — predates the KB's dated correction that Anthropic ships real enforcement (Spend Limits API, gateway spend limits, Managed Agents session budgets). Conclusion unchanged (gateway fails open by default; ceiling stays in the harness), premise now accurate.	primer §6 "Correction (2026-08-31)" (High, read directly)
templates/ralph/run.sh	Comments only, no behavioural change: (1) Fable 5.1-class models take more turns per unit of work, so iteration/stall caps tuned on older models trip earlier — early stop is a re-tuning signal; (2) --permission-prompts none (v2.1.259+) noted as the narrower deny-by-default alternative to --dangerously-skip-permissions for headless runs.	primer §4 Fable 5.1 breaking changes; sources.md headless entry (High)

Flagged for you

Hard-stop / policy-protected — human decision, not rewritten:

1. CLAUDE.md (never edited by this skill): "Anthropic's billing layer has soft alerts but does not auto-disable" is the same stale premise fixed in README above. The rule it supports (ceiling lives in the harness) still holds — the KB itself notes gateway enforcement fails open by default — but the justification sentence is now factually behind the KB. Suggest updating alongside a review of the guardrails/README lead sentence below. (Medium; primer §6 correction, High evidence.)
2. guardrails/README.md lines 3–6 (yielded to guardrails: containment as a required companion, verify-the-agent rule, and retiring a subagent cap that no longer exists #21, and guardrails: containment as a required companion, verify-the-agent rule, and retiring a subagent cap that no longer exists #21 does not cover this): same "does not auto-disable" claim, in the document's lead sentence. Proposed softening: acknowledge Spend Limits API / gateway 429 enforcement, note the fail-open default (enforcement.fail_closed_on_error: true required for a true ceiling), keep the harness-ceiling conclusion. (High visibility; High evidence.)
3. guardrails/budget.env (flag-preferred by policy, left untouched): (a) LOOP_TURN_TIMEOUT and LOOP_MAX_RETRIES are referenced nowhere else in the repo — no skill or template wires them, so the header's "skills read these as defaults" over-claims for those two; either wire them (e.g. timeout wrapper in ralph run.sh) or relabel as not-currently-enforced. (b) The 20-iteration / $10 defaults are sized for Sonnet-5-class economics; a Fable-5.1 loop ($10/$50, more turns per unit of work) will trip them earlier — consider a comment making the model-dependency explicit. Values deliberately unchanged.

Yielded to PR #21's files, not covered by its diff (route to that PR or a follow-up):

4. .claude/skills/loop-guardrails/SKILL.md: (a) same stale "soft alerts" billing premise; (b) the bash-loop budget guidance omits --max-budget-usd as the first-party ceiling for claude -p-driven ralph loops (sends authors to build an external meter instead); (c) the /ralph-loop check line could carry "defaults to unlimited" where it names the flag. (All Medium/Low; High evidence.)

Already fixed by open PR #21 — no action, just don't lose them if #21 stalls:

5. artifact-audit/SKILL.md (stale CLAUDE_CODE_MAX_SUBAGENTS_PER_SESSION fan-out claim), guardrails/README.md "Native in Claude Code" paragraph, guardrails/checklist.md containment block, and all three CLAUDE_CODE_MAX_SUBAGENTS_PER_SESSION=12 occurrences in runbooks/staying-current.md (lines 79/89/143 — confirmed all three are in guardrails: containment as a required companion, verify-the-agent rule, and retiring a subagent cap that no longer exists #21's patch). These are High-severity drifts (the env var is a no-op since v2.1.224); merging guardrails: containment as a required companion, verify-the-agent rule, and retiring a subagent cap that no longer exists #21 is the fix.

Pending-KB (supported only by unmerged #22/#23 — re-check after those merge):

6. templates/ralph/run.sh conflates cap-hit and failure in its exit path (STOP: + nonzero exit for all three hard stops). PR knowledge: Sep 14–21 pass — the first primary runaway-loop incident, and two corrections this routine owes itself #23's draft KB text (SaltBench, arXiv:2609.11076) argues a capped run needs a third outcome distinct from failure. Worth a HALT (capped)/CAPPED marker if that lands.
7. guardrails/README.md "Native in Claude Code" could cite claude plugin eval (v2.1.269, per PR knowledge: Sep 7–14 pass — Anthropic ships the three hard stops as a CLI flag set, plus two corrections to this KB #22) as the first first-party command shipping all three hard stops + a deterministic gate in one interface.

Verified clean (no drift)

update-knowledge/SKILL.md (already current on the subagent-cap story), templates/goal/recipes.md (every claim verified against live code.claude.com/docs/en/goal), templates/loop/loop.md, templates/ralph/PROMPT.md, all four templates/ui-audit/ files, runbooks/first-loop.md, runbooks/overnight-build.md (claude --cloud / deprecated --remote alias verified against the live CLI reference — the runbook is right and the KB's --remote-first phrasing is the stale side, noted for the next update-knowledge pass), runbooks/babysit-prs.md (auto-fix-PR mechanism verified against live docs), runbooks/ui-audit-overnight.md, and CLAUDE.md's "Tooling version notes" section.

Note on the branch name

The skill names claude/artifact-audit-YYYY-MM-DD; this session's environment pins pushes to its designated branch claude/keen-archimedes-buozmn, so that is used — same effect (a claude/-prefixed, PR-only branch; never main).

---

Human reviews and merges; this routine never merges to main.

🤖 Generated with Claude Code

https://claude.ai/code/session_01DexL4ENdcH1wDBuzVCs3mP

---

Generated by Claude Code