knowledge: hard stop #1 failed open three ways; correct five live claims (Sep 28 – Oct 5)

[espi]

Weekly update-knowledge pass, window Sep 28 – Oct 5 2026. Five research agents; every load-bearing claim below was re-fetched and verified by the lead against a primary, not relayed from an agent.

⚠️ This PR is stacked on #26. Its base is #26's branch, not main, because #26 is open and unmerged and rewrites knowledge/ heavily (primer 3,303 → 661 lines, plus kb-size.yml). Per the skill's step 1 I branched from #26's head (09e3980) rather than from the stale main. Review or merge #26 first; GitHub will retarget this to main automatically when it lands. The diff shown here is my one commit only.

Routine self-improvements

Applied (auto-gated): none this pass.

I ran the step 7 scan and found no auto-apply candidate. Specifically, I verified on disk that every path reference in SKILL.md resolves (kb-size.yml, self-edit-guard.yml, all four knowledge/ files, guardrails/budget.env, runbooks/staying-current.md, CLAUDE.md, and the ../../../knowledge/archive/resolved-caveats.md relative link) and that all eleven step-number cross-references point at the right steps. Nothing stale, no broken link, no typo — so default-deny applies and SKILL.md is untouched (git diff --name-only confirms).

Suggested (needs your confirm):

1. Replace CLAUDE.md's two bare version floors with the feature-scoped minimums the docs now use. "Tooling version notes" says /loop is v2.1.72+ and /goal v2.1.139+. Neither number appears anywhere in current docs — not contradicted, but the docs no longer express minimums per command. They are now per feature: "Check-ins require Claude Code v2.1.234 or later.", "Idle check-ins require Claude Code v2.1.236 or later.", "Before v2.1.246, idle check-ins were uncapped.", and for scheduled tasks "...both require Claude Code v2.1.248 or later." → Routed to you because the edit is to CLAUDE.md, which step 7 puts in the always-suggest lane.
2. Consider a corpus-level check in self-edit-guard. The strongest research finding this pass is that per-edit gates do not compose (below). This routine's envelope is a per-pass ≤10-line cap with no re-test of the accumulated SKILL.md. → Routed to you: it changes the gate itself, which step 7 forbids the routine from touching.
3. Decide whether this repo's own Routine needs a cumulative ceiling. A Managed Agents deployment budget "bounds each run separately rather than the deployment's cumulative spend" — so an hourly loop with a $20 cap has no weekly ceiling. That is this routine's exact shape. → Routed to you: guardrails/ and runbooks/.
4. A note on the research path, not a change request. A research agent reported a benign PyPI dating lookup (curl https://pypi.org/pypi/agentguard47/json) denied by the permission classifier on three attempts, while the identical command shape succeeded for six other packages in the same session. The agent did not route around it and I did not run it on the agent's behalf. Consequence recorded rather than worked around: the AgentGuard "no in-window release" negative rests on tag existence, not an independent re-dating. Worth knowing because PyPI dating is this routine's main defence against the hallucinated-year trap.

---

The headline: hard stop #1 has now failed open three distinct ways

Last pass found the first. Two more are now on record, both verified verbatim against the raw changelog:

1. v2.1.281 — "Fixed a turn that could retry indefinitely, ignoring --max-turns…" (prior pass)
2. v2.1.285 — *"Fixed CLAUDE_CODE_RESUME_INTERRUPTED_TURN re-running a turn that had ended at --max-turns"` — a resume could undo the cap.
3. Still true today, by design — with --input-format stream-json, a message still queued when the limit ends a turn "starts a new turn with its own limit." So --max-turns bounds a turn-chain, not a session.

None needed a misconfiguration. Two further failure shapes that aren't "the cap was wrong" at all:

• ⚠️ A ceiling can leak through its meter. v2.1.286 fixed the gateway spend meter "pricing 1-hour prompt cache writes at the cheaper 5-minute rate, and counting only the first model call's input tokens on streamed turns that run a server-side tool." The meter is what the enforcing 429 is computed from, so a correctly-configured hard ceiling silently admitted more spend than it was set to — and agentic loops are the shape that maximises the error.
• ⚠️ A zero can mean "no cap". max_turns: 0 "runs the session without a turn limit", while max_budget_usd: 0 is rejected and CLAUDE_CODE_MAX_TURNS=0 is rejected — three polarities in one product. Any harness that computes a limit and can reach 0 is uncapped.

/goal's documented bound is a self-reported cap

This repo's doctrine turned on the reference implementation. goal.md says to bound a run with a clause like "or stop after 20 turns", which "Claude reports progress against … and the evaluator judges … from the conversation" — while the evaluator "does not call tools." The bound is narrated by the worker and graded from that narration, so by §5A.3 and §5A.9 it is a pacing hint, not hard stop #1. It does ship real no-progress detection keyed on absence of tool use, but it "returns control to you with the goal still set" — unattended, it pauses awaiting a human rather than terminating.

Relatedly: CLAUDE_CODE_RETRY_WATCHDOG, the setting Anthropic recommends for unattended loops, "Retries 429 and 529 capacity errors indefinitely" and "removes the cap of 15" — and the thing that still binds it is the spend limit.

Five corrections to claims currently in the knowledge base

Claim in the KB	What's true
LiteLLM "has now fixed" max_budget = 0	It has not. Read from budget_limiter.py on main: the falsy-zero guard is live in all three router paths (lines 251/266/281). Only the proxy org/project budget was fixed; PR #43292 was closed unmerged 2026-10-03; #44259 is open.
arXiv:2608.18744 is "the anti-pattern this repo's doctrine forbids"	It complies. Withheld external oracle ("hidden unit tests give exact ground truth"), operators may abstain, maker ≠ checker, validated on 428 unseen tasks. It is published support, mis-titled.
Google Cloud Spend Caps are "private preview"	Public preview since Jul 28; they enforce; still not GA.
LoopGain is "a retirement candidate (no releases)"	Live — v0.6.5 published 2026-09-30. Scoped to stop #2; ⚠️ reject its pitch to replace max_iterations.
Databricks Unity AI Gateway "GA Aug 4"	The gateway is GA; budgets are Beta, enforced "approximately", with the vendor saying "Do not use this feature as a way to ensure an absolute spend cap."

Plus two matrix corrections: Aider is dormant (no release since 2026-02-12) and the Gas City row pointed at the wrong repo (gastown, dormant since June, not gascity, which shipped v1.5.0 on Oct 5).

A hyperscaler shipped the mode this repo had only argued for

AWS project spend limits (Sep 16, limited availability) enforce — "AWS pauses your project and stops all resources" — and implement the graduated throttling AWS's own AGENTCOST07-BP01 had merely recommended: ~7 days out "new resources stop launching", ~5 days idle resources pause, ~4 days the top cost drivers pause, from "EC2, RDS, Lambda, Bedrock, and SageMaker", explicitly to catch "a runaway Lambda or unexpected Bedrock spike" — so it reaches model spend. AWS also states this KB's "a ceiling needs a runway" point first-hand. ⚠️ Destructive tail: after 90 days paused, "AWS permanently deletes your project data."

roborev v0.71.0 is the counter-example, in its own words: "Routing is off by default. The limit is soft, so reviews keep running after spend reaches it." The pair settles a standing backlog question: graduated throttling is sound as the approach to a hard ceiling and unsound as a replacement for one.

The research finding that indicts this repo's own envelope

Per-edit gates do not compose. arXiv:2609.24130's corpus-level guard caught "declining performance as individually validated rules accumulate" — one configuration declined monotonically from +0.017 to −0.016 (p=0.022) as active rules grew from 13 to 70, each having passed its own gate. arXiv:2610.01073 adds that a stale cached pass is not a pass and that default-deny without rollback is not safe-by-default. See suggestion 2.

A citation rule this knowledge base was violating

arXiv:2609.25173 meta-analysed 259 agentic-security papers — the corpus much of §5A rests on — and found most report no variance estimate or repeated runs, that on a 100-instance benchmark "the minimum difference in ASR detectable at conventional power is 18.2 percentage points", and that "cross-paper ASR comparison is currently unsupported." Adopted in the primer header: trust mechanisms and directions across papers; trust magnitudes only within one paper. A live instance turned up the same pass — "SkillsBench" is 89 tasks in one in-window paper and 509 directions in another.

Three new practitioner primaries

• Willison, Oct 3 gives hard stop knowledge: 2026-06-22 update pass (7-day delta from Jun 15) #3 its first named outside voice: "Soft caps, 'after $X/month, send me a warning email', will not cut it." ⚠️ It also widens the cost surface to the pay-by-usage services a loop's code calls — which guardrails/ does not cover.
• Osmani, Sep 28: "A test written or controlled by the thing it's testing doesn't prove much" / "Real independence comes from a different source of truth: a spec a person wrote, a reference implementation, a proof."
• Huntley, Oct 2 (two posts, both readable — the "Sep 27, paywalled" baseline is superseded): "Types are a form of verification. They provide back pressure: compiler errors that the LLM picks up and fixes automatically, every loop", plus simulator-first as a way to build the deterministic check.

Clean negatives worth having

• The dollar-ceiling mechanism was not touched this window — zero in-window hits for --max-budget-usd/max_budget_usd/modelPricing; the only budget matches are retry budgets. Subagent caps (20 / depth 3 / per-session removed in v2.1.224) re-verified unchanged. Nothing released Oct 4–5; newest is 2.1.289.
• No peer harness shipped a hard stop, across seven tools — so "ceilings are still not a market norm" holds through Oct 5.
• No new citable primary cost-overrun incident, and no analyst data — the latter with a structural cause: Gartner's Symposium is Oct 19–22, after this window. Briefed for next pass.
• ⚠️ A method correction to this KB's own reasoning. Previous passes cited "fsmonitor appears ZERO times in the changelog" as evidence on the GitSpawn ultrareview item. The term has never appeared in any version, so its absence carries no information. Changelog grepping cannot resolve that item. It stays open and unconfirmed — not the same as "confirmed unpatched".
• Hooks are not deprecated by the new "mods" layer ("hooks can't rewrite events, draw new UI, or replace features. Mods can."), so /goal-as-Stop-hook and this repo's hook-delivered guardrails stand.
• AG2 shipped a Critical bypass of its own new guardrail four days after shipping it (GHSA-42mf-vpmr-gw5r): "all trivially bypassable via shell metacharacters" — an allowlist must be enforced on an argv vector, not a string.

Verification method

• Claude Code changelog fetched raw (942 KB, explicit per-version date attributes), in-window range sliced and grepped locally; every quote checked against that slice.
• All 35 arXiv IDs machine-verified in one batched id_list call (HTTP 200, 35/35) for ID ↔ exact title ↔ v1 published ↔ primary category. Every ID resolved; 28 of 35 have v1 in-window. Two of the agent's "category unconfirmed" caveats were resolved by this call.
• ⚠️ New trap, and the reason the machine-verify rule exists: an arXiv ID's YYMM prefix is not a date. The API reports 2610.00015v1 as 2026-07-27 and 2610.00197v1 as 2026-09-18 despite 2610 IDs. Both out of window.
• ⚠️ The hallucinated-year trap fired twice more — WebFetch rendered LiteLLM v1.104.0 and AG2 v1.1.2 release pages as "2024". GitHub per-tag release pages are now a confirmed instance, not just vendor changelogs.
• LiteLLM read from source on main; Loom dated from the git object (git clone --depth 1 works through the proxy: HEAD 7ba5132, 2026-02-11, zero tags).

Pruning and the size budget

The primer stayed inside its 750-line budget (749) — the first pass to run under kb-size.yml. That required condensing per-release and per-paper detail into sources.md per step 5, and it also surfaced two pre-existing mid-sentence breaks in §5A.1 and §5A.2 which I repaired. Per step 6 I also removed superseded text rather than leaving it beside the correction: the stale LiteLLM, Google-preview and Azure assertions in older dated sections now carry explicit supersede/resolve markers.

Six items resolved and archived (Azure first-party source; Gas City 1.4 backfill; the four-paper "read next pass" list; Loom's dormancy; "Diamond" → Graphite Agent; the /loop//goal version-minimum question).

Every currently-open re-verify item (41 → 56)

Per step 2, the full standing backlog, not only what's new. Twenty-one opened this pass, marked ⚠️ NEW.

Security — GitSpawn ultrareview (6th pass, now unconfirmed rather than confirmed-unpatched); does the PR-review gate bind what the reviewer sees to what merges?; is every write path to a protected region covered by self-edit-guard?; vendor artifact for 2609.01222's claimed fixes; does X-CPE reach this repo's knowledge/?; adopt PROCTOR canaries / hash-pinning / provisional-persistent split / annealed edit budget?; --restricted is not an OS sandbox; ⚠️ NEW per-edit gates don't compose; ⚠️ NEW a command prefix allowlist is not a sandbox; ⚠️ NEW can a mod load before sec-default?

Guardrails — fixed caps vs progress-authorized continuation + context growth as a 4th cost surface; verify an injected guardrail reached the model; zero-valued ceilings (corrected — LiteLLM is not fixed); LiteLLM fail-open family (#33323 state genuinely ambiguous, #27394, collector drop); Anthropic's pre-announced spend controls (still undelivered); Google Spend Caps (corrected to public preview; GA still open); adopt OpenRouter's cost endpoint?; ⚠️ NEW a ceiling can leak through its meter; ⚠️ NEW four status codes for an exhausted budget (402/403/422/429); ⚠️ NEW a per-run budget is not a campaign ceiling (this routine's shape); ⚠️ NEW AWS's limit is limited-availability with a destructive tail; ⚠️ NEW the Bedrock cell needs the first-party treatment Azure just got; ⚠️ NEW OpenRouter's zero-budget semantics undocumented.

Tooling — 1 GB tool-results cap is changelog-only (4th pass); SKILL.md cross-tool execution and governance; Google's Agent Plugins maintainership (7th consecutive negative — recommend archiving the claim); roborev's MCP path ungated (re-confirmed through 0.71.0); review-tool hygiene (Qodo/Kodus halves still open); ⚠️ NEW the Gas City row points at the wrong repo; ⚠️ NEW opencode's v2.0.x line is invisible; ⚠️ NEW Amp not trackable at release granularity; ⚠️ NEW Aider should be reclassified dormant; ⚠️ NEW agentskills.dev and agentplugins.org both 503'd, so two standards baselines are unverifiable; ⚠️ NEW the MCP Tasks extension path 404s; ⚠️ NEW Factory has a /loop-class primitive the matrix omits.

Research / infrastructure — correct the x.com note + add snowflake-ID decoding to runbooks/; Osmani's Sep 7 X post (text still unread on his surface); Cherny's absence is a structural limit (permanent Medium); Huntley's recap dating is inconsistent; the unchased YC-hackathon ralph field report; the "Jev" identity question; the do-not-cite blacklist; the four recycled dollar anecdotes; the Jun 15 billing split; Microsoft dropping Claude Code; the /goal origin timeline; the "costliest thing" slogan; ⚠️ NEW adopt the magnitudes-don't-cross-papers rule; ⚠️ NEW Osmani's two uncited statistics (570 review comments / 14%; Anthropic 8× merge rate); ⚠️ NEW word Steinberger's x.com coverage gap permanently, and record Ronacher as "published, off-topic" not "quiet"; ⚠️ NEW dated tripwires (Gartner Oct 19–22; Sonnet 4.5 retires Nov 30; the GPT-5.6 "25% rise" restated as a promo expiry of unknown magnitude; LiteLLM v1.105.0 RC); ⚠️ NEW blacklist additions and the trustgateai TLD gap; ⚠️ NEW the KB's Loom description is unverified against the current README.

Worth a closer look

• The five corrections — each reverses or qualifies something the KB currently asserts; the LiteLLM one in particular was wrong in a direction that would have made a reader trust a ceiling that doesn't hold.
• arXiv:2610.01023's over-rejection cost (0.66–0.67) is quoted alongside its catch rate deliberately. The headline ("a weak reviewer with good evidence beats a strong one without") is real but the deployable version is expensive, and I didn't want the primer carrying only the flattering half.
• Nothing in guardrails/, budget.env, CLAUDE.md, permissions or the gate was touched — this PR is knowledge/ only. The three guardrail-shaped findings (status-code table, cumulative ceiling, argv allowlists) are suggestions above, not edits.
• CI: kb-size should pass at 749/750; self-edit-guard has no self-edit: commit to inspect.

🤖 Generated with Claude Code

https://claude.ai/code/session_01Dv4X6E1AZFTZ2oYYAGD3eD

---

Generated by Claude Code