Findings: the front lane's rate limit is a size ceiling, and a session grows into it

[jerome-queck]

Findings: the front lane's rate limit is a size ceiling, and a session grows into it

Not a request for a fix. This is what a morning of failures measured, written down while the evidence is still in the log, plus what the six pull requests around it taught. The Owner decides what, if anything, to do with each.

The headline

Groq's free tier is 8,000 tokens per minute, and one Syrax turn is bigger than that. It is not a rate limit in any useful sense — waiting does not help, because a single request exceeds the whole minute's budget:

413 Request too large for model `openai/gpt-oss-120b` ... service tier `on_demand`
on tokens per minute (TPM): Limit 8000, Requested 8831

Measured across the morning: syrax-gemini/gemini-3.5-flash-lite took 313 calls with zero 429s. syrax-groq/openai/gpt-oss-120b took 62 calls with 13, every one of them a 413 dressed as a 429.

Finding 1 — a retry that grows cannot converge on a size limit

The runtime answers a 413 with rate-limit same-model retry 1/3 … delayMs=10000, then 20s, then 30s. Each retry appends the previous failure to the session, so each attempt is larger than the one that just failed:

firing	attempt 1	attempt 2	attempt 3	attempt 4
05:43	8631	8631	8659	8687
06:13	8715	8743	8771	8799
07:13	8939	8967	8995	9023
08:43	9275	9303	9331	9359

+28 tokens per retry, +112 per firing, monotonic for three hours. Backoff is the right answer to a rate limit and the wrong one to a size limit: the only thing that shrinks the request is not sending the failures back.

Finding 2 — a session grows by heartbeat, unprompted

Something pokes every chat on a 30-minute cadence (observed at :13:53 and :43:53, all four agents firing together) and each poke lands in the session as a turn. Counted in the trajectories:

general/5a043b53….trajectory.jsonl      545 heartbeats
general/950b9bac….trajectory.jsonl      450
general/01c35ae3….trajectory.jsonl      129
academic/c29de0ed….trajectory.jsonl      42

Nothing in openclaw.json configures this, so it is a runtime default. It is the mechanism behind "my chat gets slower and then breaks": a session nobody has reset accumulates hundreds of turns that no one sent.

/new is currently the only lever, and its effect is dramatic — it is what stopped the runaway above at 08:49.

Finding 3 — context windows are not the constraint; the per-minute budget is

rung	context window	maxTokens
gemini-3.5-flash-lite	1,048,576	8192
ministral-3b-latest	131,072	1024
openai/gpt-oss-120b	131,072	1024

No turn is anywhere near a 131k context. The binding limit is Groq's 8k/minute, which is two orders of magnitude smaller than the window the same model advertises.

Unverified but worth checking: maxTokens: 1024 on two of the three front rungs may be its own bug. The shortlist tool call that #202 removed had eleven buttons carrying opaque tokens; a call like that could plausibly exceed a 1024-token output budget and arrive truncated, which is indistinguishable from the malformed calls seen. Nothing here measured it.

Finding 4 — the standing prompt is a floor under every turn

chat	AGENTS.md	rough tokens	tools
academic	5347 B	~1340	17
general	4533 B	~1130	10
system	3399 B	~850	7
media	1667 B	~420	4

Plus each tool's own schema and description, which for the search unit are long. ADR-0011 already says every line is paid on every turn; what is new is that against an 8k/minute ceiling the floor is a material fraction of the budget before the Owner has typed anything.

Finding 5 — a pin silently removes the fallback chain

/model syrax-groq/… collapses the lane to one candidate. The log records it plainly:

"attempt": 1, "total": 1, "fallbackConfigured": false

Offered as a workaround during an outage, it made things strictly worse: the pinned rung was the one with the 8k cap, and pinning removed the two rungs that would have answered. Nothing warns about this, in the chat or in /status.

Finding 6 — a provider finish_reason is not a failover reason

gemini-3.5-flash-lite aborting with function_call_filter: MALFORMED_FUNCTION_CALL produced failoverReason: null and no fallback decision at all — the chain was never consulted. openclaw classifies failover from transport signals (status codes, timeouts); a provider that returns 200 and then refuses in its finish reason falls outside that. Rate limits do route correctly (failoverReason: "rate_limit"), so this is a gap rather than a broken router.

A malformed call that fell through to the next rung would have been an annoyance instead of a morning-long outage.

What the six pull requests taught, which is not in any of them

• An instruction change does not take effect in a running session. Three fixes looked like failures because the model was copying its own earlier tool calls out of the context window. In-context examples beat instructions, reliably. Every instruction change needs /new before it can be judged, and that is written down nowhere.
• A test that scripts a tool call proves the surface, never the model. A shortlist's numbered list is dropped, because a text block loses to message #198 shipped an instruction describing one shape and a test scripting another; the test stayed green while the chat was broken. The scripted arguments must be the arguments the instruction asks for, or the test measures a shape nothing produces.
• Prose is not a schema. Three wordings produced three different malformed calls. What fixed the schema errors was showing the literal JSON. What fixed the rest was not asking the model to build the structure at all (ADR-0033).
• The suite that matters only runs on the mini, so a red test can sit on main indefinitely — The stand-down gateway test reads an already-read deployment #196 was exactly that, and CI never sees it.

Open questions for triage

1. Is Groq worth keeping in the front chain at 8k TPM, given one ordinary turn exceeds it? It is a rung that cannot answer.
2. Should a 413 stop retrying rather than back off — and should the retry drop the failures it is accumulating?
3. Should sessions be trimmed or reset on a schedule, given heartbeats grow them unprompted?
4. Should a pin warn that it removes the fallback chain?
5. Is maxTokens: 1024 correct for the two rungs that carry it?
6. Is finding 6 worth reporting upstream, or pinning a runtime that handles it?

[jerome-queck]

This was generated by AI during triage.

Triage outcome

Triaged as a findings report rather than a defect, and split into five tickets. Relabelled bug → decision and moved to ready-for-human: what remained here after the split was a set of questions for the Owner, and those were settled in a grilling session rather than by an agent.

This issue stays open as the parent record. Its evidence is what the five argue from.

What was cut

#205	The Mistral rung truncates its answers — question 5
#206	The heartbeat grows every session — finding 2, question 3
#207	Three records carry a spent claim about streaming
#208	A rung the session has outgrown stands down — findings 1 and 3, questions 1 and 2
#209	An instruction change cannot be judged in a running session

Findings 5 and 6 were not cut. Both are already recorded in ADR-0033 with the same evidence — the pin collapsing the chain to attempt: 1, total: 1, and failoverReason: null on a provider finish_reason. Question 6's upstream report is left for the Owner to decide separately, since it posts to someone else's repository under their name.

What measurement changed

Four things this issue got right were confirmed, and three claims moved.

Confirmed, against the live providers. Groq's 8,000 TPM is a per-request size ceiling: a full bucket (x-ratelimit-remaining-tokens: 8000) refuses Requested 8098. The reservation is debited whether or not it is used — a 3,598-token prompt reserving 128 left 4,274 remaining, and 8000 − 4274 = 3726 = 3598 + 128 exactly. And maxTokens: 1024 is its own bug on the Mistral rung, which this issue flagged as unverified and worth checking: it returns finish_reason: "length", cut mid-word.

/new is not the only lever. The pinned runtime configures the heartbeat's cadence, its delivery, its active hours, and whether a run retains conversation history at all. Syrax states none of it. #206.

The retry loop has a cause, one step below "backoff is the wrong answer to a size limit". The runtime's context-overflow detector returns false for any provider message naming tokens-per-minute — deliberately, and correctly for every provider whose TPM limit is only a rate. Groq's refusal names TPM, so the detector rejects it before testing the overflow signatures it would otherwise match, and the error takes the rate-limit path. The log shows it: HTTP 413 carrying failoverReason: "rate_limit".

Streaming makes no difference, contrary to ADR-0016 and to the glossary's Wall entry. Both measurements above were non-streaming. #207.

The finding that changed the decision

This issue reads as though the Groq rung cannot answer at all. It can, and the log says when:

12:59:41  200   ← 12 real streaming turns
13:19:51  200   ← the last one
13:17:47  413   ← the first refusal
16:44:56  413   ← still refusing, 3½ hours later

12 served, 37 refused, 13 rate-limited. A twenty-minute window where the rung worked, then never again. The rung answers a young session and dies as the session grows — so its viability is a function of session state, not of the rung, and removing it would throw away something that works. That is what #208 decides, and it is why it decides stand down rather than remove.

[jerome-queck]

This was generated by AI during triage.

Question 6, reported upstream

Finding 6's residue — the part of this issue that no configuration here can answer — is filed as
openclaw/openclaw#130096.

Two things were established while writing it that this issue did not know.

It is not fixed in any published channel. Confirmed by calling the runtime's own exported
classifier on the real refusal message, rather than by reading the code:

channel	version	isContextOverflowError(groq413)
extended-stable	2026.6.34	false
latest	2026.7.1-2	false
beta	2026.8.1-beta.3	false

A genuine overflow message returns true in the same builds, so the classifier works — this one
wording escapes it. extended-stable is still 2026.6.34, so ADR-0003's pin is current for its
channel and there is no upgrade waiting; and rolling forward would not have helped, which is now
measured rather than assumed.

The beta refactors this exact area and comes close. The pattern tables moved out to
@openclaw/ai/internal/runtime as named scopes, and isContextOverflowError splits so the
provider-plugin hook sits outside the TPM guard — which looks like it would let a plugin
re-classify the error. It does not: the hook's candidate gate is !isRateLimitErrorMessage(…), and
Groq's message satisfies that predicate. So there is still no local workaround, and #208's decision
to stand a rung down rather than wait for upstream stands on its own.

This closes the last of this issue's six questions. Nothing here changes #205–#209.