Repository navigation
Responses progressive input materializes the whole history when a tool row is present #3858
Description
Problem
#854 (PR #3852) makes the OpenAI Responses request body stream from journal rows. An RSS investigation during that work found one remaining materialization.
progressiveResponsesInput calls fill() when it reaches the first non-text row, so any history containing a tool call or response triggers it. fill() materializes the whole IContent[] and the whole ResponsesInputItem[] before the byte stream starts. That means the "no whole-body array" property holds only for text-only histories. Real sessions almost always contain tool rows.
The materialization is request-scoped and released after the call, so retained heap between turns is not affected. The repeated-turn proof stays within the 1 MiB allowance. Peak memory during a send is still proportional to the context.
Measurements at 8,192 rows (4.7 MB body), after the two fixes in commit 3e4fd0e (chunk-buffer amplification and quadratic tool pairing):
| Send | Peak RSS added during the send |
|---|---|
| OpenAI Responses | +344 MB |
| Anthropic | +270 MB |
The Anthropic figure includes its SDK body, which is a permitted request-scoped exception. The fill stage accounts for about +100 MB of the Responses figure.
buildInput also spends about 1.5 s at 8,192 rows, much of it in limitOutputTokens per tool response.
Proposal
Convert tool call and response pairing in the progressive input to a streaming pass. The Responses source input already pairs through a disk index, from #854 commit 44ab888. The progressive route should use the same bounded pairing instead of falling back to fill().
Acceptance
- Request bytes are unchanged against the existing byte oracles:
providerBody*.p05b4and thedisk-text-*tests. - A peak-heap test for a tool-heavy 8,192-row history shows no allocation proportional to the context beyond the body bytes in flight.
- Include a trap that reintroduces
fill(); the test must fail with it.
Activity
github-actions commentedon Oct 11, 2026
LLxprt Issue Planner — infrastructure failure
The autonomous planner failed with exit code 1. Please inspect the workflow logs (Run planner agent section) and re-run once resolved.
fix(#854): C2 Responses calls without request rows go through the dis…
acoliver commentedon Oct 14, 2026
Fixed in PR #3852: the progressive Responses fill path is deleted, and peak memory during a Responses send no longer grows with the size of the history.
Commit 253a7d5 (C2) removes
progressive-responses-body.tsandresponses-content-filler.ts. Calls that arrive without request rows are staged on scratch disk and go through the same disk text route as row-backed calls. Commit 6106d45 (C3) deletes the remaining lazy and array Responses machinery, includingLazyBoundedJsonBody. Nothing callsfill()any more, and noResponsesInputItem[]orIContent[]for the whole history is built.Peak numbers from the peak-during-send probe (heap growth when the body goes from 1.4 MiB to 11.6 MiB in a tool-heavy history, bound 1,048,576 bytes), measured on the same shape before and after:
The "before" figures are from the A2 probe run and the C1 fail-first run, recorded on the same test files. The "after" figures are from the final run on 58149a1: heap growth, arrayBuffer growth (0 to 8,304 bytes) and retained growth (0 to 390,330 bytes, Theil-Sen over repeated sends) are all inside the 1 MiB bound, with the body bytes equal to the pre-change oracle and Content-Length equal to the bytes sent.
The array trap (13.9 MB) and the fill trap (28.1 MB) break the heap bound, and the retain trap (80.3 MB retained) breaks the retained bound, so the probe fails on the behaviour this issue reported.
The RSS figures in the issue (+344 MB at 8,192 rows) were not re-measured at that row count. The probe measures heap and arrayBuffer growth between two body sizes, which is the property the fix targets.
buildInputand its per-tool-responselimitOutputTokenscost are gone with the array path.