You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Last-resort tool-response truncation is O(candidates x rows) on the journal-backed send path #3857
On the journal-fed source send route from #854 (PR #3852), the last stage of the context enforcement ladder is truncateSourceToolResponses → truncateOversizedToolResponsesUnified (toolResultTruncator.ts). After every single tool-response replacement it re-projects the whole candidate from disk and writes the journal. With n rows and about n/3 tool-response candidates, a send that reaches this stage does O(n²) work.
It was measured in a test fixture during #854 CI work (tmp bench, not committed). With a 1M-token limit and mostly uncompressible 2 KiB text rows, the send stalled:
1,024 rows took 520 ms.
2,048 rows were still running at 180 s.
8,192 rows never reached provider preparation.
Under the limit, the same send path is linear: about 1.3 s at 8,192 rows.
This stage only runs after compression and the fallback stages have failed to bring the request under the limit, so most sessions never reach it. When a session does reach it, the turn appears to hang instead of failing fast with a context-overflow error.
Proposal
Decide all replacements in one pass, then publish a single atomic change. Use a disk-backed running estimate per row, so the loop does not re-project after each replacement.
Alternatively, cap the stage's work. When truncating tool responses cannot reach the limit, because the remaining rows are not tool responses, fail fast with ContextOverflowError before doing the per-candidate loop.
Acceptance
A counting-seam test shows that projections and journal writes per send in this stage do not grow quadratically with the row count.
An over-limit fixture at 8,192 rows either finishes or fails with ContextOverflowError within seconds.
Request bytes are unchanged for cases where truncation succeeds.
Fixed in PR #3852 (ba2947e). The last-resort stage now ranks and decides every replacement in one pass over a disk-backed running estimate and publishes them as a single change, instead of re-projecting the candidate and writing the journal after each replacement. If shedding every tool response cannot bring the request under the limit, it fails fast with ContextOverflowError before doing any work.
Measured on the same fixture (mostly 2 KiB text rows, tool-response truncation reached):
Rows
Before
After
1,024
722 s
84 ms
2,048
did not finish
227 ms
8,192
did not finish
537 ms
Checks: a counting-seam test shows row reads and writes grow about 4x for 4x the rows (it failed on the old code, 140 s). An 8,192-row over-limit send fails with ContextOverflowError in about 8.5 s without rewriting the journal. A differential corpus shows the history, pending rows and final token count equal the old per-replacement stage whenever truncation succeeds.
Activity
github-actions commentedon Oct 11, 2026
LLxprt Issue Planner — infrastructure failure
The autonomous planner failed with exit code 1. Please inspect the workflow logs (Run planner agent section) and re-run once resolved.
fix(#854): last-resort tool-response truncation decides all replaceme…
acoliver commentedon Oct 14, 2026
Fixed in PR #3852 (ba2947e). The last-resort stage now ranks and decides every replacement in one pass over a disk-backed running estimate and publishes them as a single change, instead of re-projecting the candidate and writing the journal after each replacement. If shedding every tool response cannot bring the request under the limit, it fails fast with ContextOverflowError before doing any work.
Measured on the same fixture (mostly 2 KiB text rows, tool-response truncation reached):
Checks: a counting-seam test shows row reads and writes grow about 4x for 4x the rows (it failed on the old code, 140 s). An 8,192-row over-limit send fails with ContextOverflowError in about 8.5 s without rewriting the journal. A differential corpus shows the history, pending rows and final token count equal the old per-replacement stage whenever truncation succeeds.