Repository navigation
knowledge: Aug 24–31 pass — gateway spend limits fail open by default, CC v2.1.243–251, two loop benchmarks converge at ~25% - #19
Merged
Conversation
…43–251, LoopsBench+LoopArena ~25% convergence Five-agent fan-out over the Aug 24–31 window. Headline is a correction to this repo's own primer §6 rather than an external finding. §6 correction (High, primary read verbatim): Anthropic ships real spend enforcement, not only alerts — the Spend Limits API (Enterprise) and Claude apps gateway spend limits both hard-stop requests (429 / billing_error). But gateway enforcement FAILS OPEN by default when its Postgres spend store is unreachable or times out; enforcement.fail_closed_on_error: true is required for "no unmetered spend". The ceiling degrades into an alert exactly when infrastructure is unhealthy. Same shape as LiteLLM's fail_closed_budget_enforcement: on both gateways the true ceiling sits behind a non-default flag. §5B's "alerting layer, not confirmed to hard-stop" characterization corrected in place. Claude Code v2.1.243–251 (7 releases, no new model): /usage Loops breakdown (first per-loop cost attribution); subagent maxTurns results now marked partial instead of appearing finished (closes a false-"done" hazard); /goal idle sessions capped at three check-ins per goal (a new cap on top of the v2.1.239 back-off, not a restatement — confirmed by spot-verification); new --restricted containment flag. Constraint for this repo: locally configured MCP servers cannot attach to cloud routines. All four headline attributions spot-verified verbatim; v2.1.242, 244 and 249 do not exist, which explains prior passes drifting one version off. Sonnet 5's introductory $2/$10 pricing made permanent — verified on the stated last day of the promotion; the Sept 1 increase to $3/$15 will not occur. Expiring primer fact replaced rather than annotated. Also: Anthropic weekly Claude Code limits move Sept 14 (+25% vs baseline, ~-17% vs today); Osmani "Audit your Agent files" and Yegge "Fences, not Sandboxes" as opposing in-window positions on scaffolding, with the tension recorded; LoopsBench (backlog read, 25.00%) paired with independent in-window LoopArena (24.69%), both naming verification/regression management as the binding constraint; a verification arXiv cluster with machine-verified v1 dates (EvoUndo: 0/197 conventional repair on failed recoverability); skills co-evolution data (78% of SKILL.md/script co-changes functionally coupled); roborev v0.67.0 with an explicit null result. Rejections logged: the "Agent SDK billing split went live July 10" claim for a third time (paused status upgraded to High against a directly-read primary); a snippet conflating the VB Pulse survey as n=573/27% (actual n=107/21%); RedHub AI's unsourced spending-limits piece; arXiv:2608.27086 as a no-result position paper. Backlog: LoopsBench resolved and archived; 12 items carried forward unchanged; 7 new, including whether --max-budget-usd meters at contracted or list price under modelPricing, and unverified aggregator claims held back pending primary sourcing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BGke13bsngqBGuPLGX8ueT
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Routine self-improvements
Applied (auto-gated) — none this pass.
No auto-apply candidate arose. I re-checked this skill's own repo-internal state against the may-auto-apply categories and everything resolves correctly: the step cross-references in steps 7 and 8 point at the right steps, the relative link
../../../knowledge/archive/resolved-caveats.mdresolves to a tracked file, and.github/workflows/self-edit-guard.yml(plus.github/self-edit-guard.py) exist as the gate text claims. No stale cross-reference, broken path, or typo to fix — so the default-deny outcome is "nothing to apply," not "something suppressed."Worth noting the guardrail text was checked and held: nothing in-window changed
CLAUDE_CODE_MAX_CONCURRENT_SUBAGENTS(20) orCLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH(3), and the Routines docs still confirm the daily-run-cap-plus-metered-overage behaviour the "Budget" bullet depends on.Suggested (needs your confirm) — three, all outside this
SKILL.mdand therefore always-suggest, never auto-apply:guardrails/: consider--restrictedas a documented default for untrusted-input loops. v2.1.248 added--restricted/CLAUDE_CODE_RESTRICTED=1, which "removes the built-in tools that run commands or code andWebFetch… keeps file tools inside the working directory, refusesbypassPermissions, and ignores user, project and local settings files." That is a cheap blast-radius floor for the §5A "Friendly Fire" threat model. Routed to a human: edits toguardrails/**are on the always-suggest list, and the supporting claim that it is not an OS-level sandbox is secondary-sourced only (now a re-verify item).runbooks/staying-current.md: record that cloud routines cannot use locally configured MCP servers. v2.1.251's/schedulenow says so explicitly — a connector must live on the claude.ai account or in a committed.mcp.json. This routine is unaffected (its GitHub access is an account connector), but the runbook documents the Routine setup path and someone following it withclaude mcp addwould hit this. Routed to a human:runbooks/**is always-suggest.Baseline and scope
Last pass: 2026-08-24. Window: Aug 24 – Aug 31. Per step 1, I checked for an open
update-knowledgePR before starting: none open (PR #18 merged; the branch sat at that HEAD), so this is a clean baseline with no expected merge conflict.Five parallel research agents (tooling & versions, ecosystem & techniques, key voices, guardrails & cost, verification & skills); all five returned cleanly. A substantive window in every lane.
The headline is a correction to our own §6
Anthropic ships real spend enforcement, not only alerts — §6 previously read the 75%/90% figures as the whole story, but those are just the Claude Code warning layer. Underneath sit the Spend Limits API (Enterprise) and Claude apps gateway spend limits, which genuinely block:
429,error.type: billing_error,x-should-retry: false, daily/weekly/monthly each enforced independently, with client aborts billed on a floor estimate "so aborting requests early doesn't evade a cap."But the default failure mode is the thing. The gateway's pre-check queries Postgres on a two-second timeout, and — verbatim — "if the store is unreachable or times out, enforcement fails open by default: the request proceeds, the gateway logs a warning." You need
enforcement.fail_closed_on_error: truefor "no unmetered spend."So the ceiling silently degrades into an alert precisely when infrastructure is unhealthy — which is when a runaway loop is most likely to be running. It is the same shape as LiteLLM's
fail_closed_budget_enforcement: on both gateways, the true ceiling is behind a non-default flag. I read and verbatim-verified this primary doc myself rather than relying on the research agent.What's new
--max-budget-usd/max_turns/subagent caps). Loop-relevant:/usageLoops breakdown (first per-loop cost attribution — visibility, not enforcement); a subagent stopping atmaxTurnsnow returns output marked partial "instead of appearing finished", closing a real false-"done" hazard;/goalidle sessions capped at three check-ins per goal; and--restricted. Constraint we run into: locally configured MCP servers can't attach to cloud routines.skills/directories, instruction files and their scripts co-evolve with 78% of co-changes functionally coupled; and agent-first repos without committed AI config grow cognitive complexity ~2× faster, with 73.8% of AI-config artifacts committed once and never modified. Flagged explicitly as observational, not causal.Corrections, rejections and traps caught
/goalitem as possibly a restatement of v2.1.239's back-off. Spot-verification against the primary shows it is a separate, new cap — the opposite of the usual one-version-off error.Quiet lanes, stated rather than padded
Gas Town/Gas City, Huntley/ralph/Loom, OpenHands, LangGraph/ADK/CrewAI/AG2, Devin, OpenRouter, Databricks, Cloudflare, AgentGuard, LoopGain, Helicone/Portkey, Agent Plugins, AAIF, Gartner/Forrester/IDC, and Steinberger/Cherny/Huntley all produced nothing in-window. No new cost-overrun incident. No Week 35
whats-newdigest.Archived
LoopsBench-not-yet-read →
archive/resolved-caveats.md(read in full, promoted, Low → High). Per step 6 the superseded text was removed, not annotated: the expired "Sonnet 5 promo through Aug 31" wording and §5B's "alerting layer, not confirmed to hard-stop" characterization were both replaced in place.Standing re-verify backlog — every currently-open item
Per step 2, the full live backlog, not only what is new this pass.
Carried forward unchanged (12):
SKILL.mdcross-tool execution is still not fully settled/goal"Codex invented it, Claude copied in 11 days" timeline rests on one secondary$47K / 11-day loopand overnight-billing figures are self-reported anecdotes$500M in one month— unnamed company, no primary disclosureNew this pass (7):
13.
modelPricingvs--max-budget-usd— the docs say contracted rates drive/cost, status line and telemetry, but not whether the harness ceiling meters at contract or list price. Changes what a given budget number buys.14. The Sept 14 weekly-limit change — help-center page not updated as of Aug 31; re-verify after it lands
15. Claude Managed Agents' $0.08/session-hour runtime billing — a wall-clock cost dimension token-only budgeting misses; ship date not established, so not promoted
16.
--restricted"is not an OS-level sandbox" — secondary-sourced only; verify before recommending it inguardrails/17. arXiv:2608.27299 "When Context Gets Root" — reports escalation via ordinary
/goal- and/schedule-shaped features; only the abstract was read, and it bears on the routine-autonomy model this repo runs on18. LiteLLM v1.99.0 — does it GA or get skipped for v1.100.0?
19. Unverified aggregator claims — notably an alleged Hugging Face agent-escape incident. Would matter to §5A's threat model if true; must not be promoted on current evidence.
Files:
knowledge/00-primer.md,knowledge/sources.md,knowledge/CHANGELOG.md,knowledge/archive/resolved-caveats.md. No changes toguardrails/,budget.env,CLAUDE.md, permissions, or any skill file.Generated by Claude Code