Feature: configurable inbound body limit - 922k context sessions 413 on remote compact (256 MiB hard-coded in request-decompress)

[nowhere1975]

Area

Proxy and routing

What are you trying to accomplish?

With the GPT-5.6 1M (922k opt-in) context window configured, a long session eventually triggers Codex's automatic remote compaction. That compaction request replays the full conversation history to the compaction model, and once the session passes roughly 700-800k tokens the serialized request body exceeds the proxy's hard-coded 256 MiB inbound limit. The compact task then fails with:

Error running remote compact task: unexpected status 413 Payload Too Large: Decompressed request body exceeds 268435456 bytes, url: http://127.0.0.1:10100/v1/responses

Users on the large opt-in window cannot use automatic (or late manual) compaction at all, because the compaction request itself is what crosses the limit. I want the body limit to be configurable (or scaled with the model window) so large-context sessions can compact.

What prevents this today?

• src/server/request-decompress.ts hard-codes MAX_DECOMPRESSED_BODY_BYTES = 256 * 1024 * 1024 for /v1/responses inbound bodies (readJsonRequestBody always uses it; there is no config lever).
• A 922k-token session history serializes to well over 256 MiB of JSON, so remote compact (and any full-history request) returns 413 before parsing.
• Manual /compact suffers the same 413 once the history is large enough - the user is stuck until they abandon the session.
• The limit is independent of the configured context window: smaller windows (e.g. 272k) stay far below it, which is why this only bites the opt-in large-window setups.

What should OpenCodex do?

1. Make the inbound decompressed-body limit configurable (e.g. a config key with a sensible default), and/or
2. Derive it from the routed model's configured context window (limit >= window size serialized with headroom), so enabling the 1M opt-in does not silently break compaction, and/or
3. At minimum, return a clearer error explaining that the session is too large to compact through the proxy (instead of an opaque 413), and suggest a smaller window or manual compaction earlier.

Example usage or interface

# config.json
"inboundRequestBodyLimitBytes": 536870912   # 512 MiB opt-in

# or per-model auto-scaling when the 922k window is enabled
"providers": { "openai": { "modelContextWindows": { "gpt-5.6-sol": 922000 } } }
# -> inbound limit scales to cover the window, compaction works

Alternatives or workarounds

• Manually /compact before the session exceeds ~600k tokens (works, but easy to forget; automatic compaction still 413s if it fires late).
• Revert to the 272k default window (compaction always safe, but the large-window capability is lost).

Additional context

Related: the 1M opt-in mechanism for gpt-5.6 (NATIVE_GPT56_MAX_INPUT_TOKENS / modelContextWindows), and #3196 which added an opt-in ceiling for oversized outbound passthrough bodies - this is the inbound analogue. Reproduced on 2.42.0.

Checks

• I searched existing issues and documentation.
• This request describes a concrete OpenCodex workflow rather than merely naming a desired technology.
• I removed secrets and personal data.

[Ingwannu]

The 413 path is real: current dev still applies the shared 256 MiB decompressed admission cap before parsing Responses and compact bodies. I do not think the token-count explanation is established yet, though. A 700–922k text-token history normally should not serialize anywhere near 256 MiB; crossing this limit usually implies large base64 image parts, very large tool outputs, repeated embedded payloads, or another replay-shape expansion.

Please provide content-free measurements from one failing request: compressed Content-Length when present, decompressed byte count from the 413, Codex version, and counts plus total serialized bytes grouped by input item/content-part type. In particular, note the total bytes in input_image data URLs, file parts, and tool outputs. Do not share prompts, credentials, images, or raw request bodies.

A safe fix should not simply expose an unbounded/global 512 MiB knob. The parser temporarily owns raw, decoded, UTF-8 text, and parsed JSON representations, so doubling the admission cap can multiply peak memory and make a shared/non-loopback proxy vulnerable to memory exhaustion. Model context size is also unavailable until after this pre-parse gate. The likely safe direction is a bounded, hard-capped opt-in with aggregate admission accounting, plus a compaction-specific diagnostic; if one payload class is responsible, earlier compaction or pruning that class may be better than raising every data-plane route. I am keeping this open pending the sanitized size breakdown.

[lidge-jun]

리뷰 · 우선순위 50 / 80

이 이슈는 GPT-5.6 1M(922k opt-in) 같은 큰 컨텍스트 세션에서 Codex 자동(또는 늦은 수동) remote compact가 전체 히스토리를 다시 /v1/responses로 보낼 때, 프록시의 하드코딩된 256 MiB 인바운드 압축 해제 한도에 걸려 413으로 죽는다는 기능 요청이다. 제보 에러는 Decompressed request body exceeds 268435456 bytes이고, 대략 700–800k 토큰을 넘긴 세션에서 compact 자체가 한도를 넘기 때문에 창은 켰는데 compact를 못 해서 세션을 버려야 하는 상태에 빠진다. 원하는 것은 (1) 설정으로 인바운드 한도를 올리거나, (2) 모델 컨텍스트 창에 맞춰 자동 스케일하거나, (3) 최소한 413 대신 세션이 너무 커서 compact가 안 된다는 안내다.

지금 dev(HEAD 24cc558d5, package 2.43.0)에서 원인은 제보 그대로다. src/server/request-decompress.ts 22줄 MAX_DECOMPRESSED_BODY_BYTES = 256 * 1024 * 1024이고, 주석도 루프백 Codex 폭탄 위협은 약하고 OOM 가드라고 적혀 있다. readJsonRequestBody(237–238줄)는 항상 이 상수를 readBoundedJsonRequestBody에 넘긴다. 호출부는 Responses 데이터면 전반이다: src/server/responses/core.ts 2724줄(본요청), src/server/responses/compact.ts 498줄(remote compact), collaboration/encrypted-payload/policy-fallback도 같은 헬퍼를 쓴다. DecompressedBodyTooLargeError는 core.ts 1438줄대에서 처리되어 제보의 413 문구로 나간다. 아웃바운드 쪽에는 이미 #3196이 maxUpstreamBodyBytes(default-off, src/types/config.ts 761줄, src/config.ts zod, core.ts checkOutboundBodySize)를 넣어 큰 바디 가드의 나가는 짝을 만들었다. 이 이슈는 그 들어오는 짝이다.

컨텍스트 창과 한도가 분리돼 있다는 점도 맞다. modelContextWindows와 GPT-5.6 922k/272k 쌍은 routing/capability·input-admission·registry·관리 API에 이미 있다. 창을 키워도 MAX_DECOMPRESSED_BODY_BYTES는 안 따라간다. 272k 기본 창에서는 JSON이 256 MiB 아래에 잘 남고, 922k opt-in + 긴 히스토리(+이미지 base64)에서만 compact가 먼저 터지는 구조다. 다만 한도를 창 토큰 × 어떤 바이트 계수로 자동 추론하는 건 추정이 거칠고, 이미지·툴 결과가 있으면 토큰 대비 바이트가 크게 뛴다. decompress 주석이 말하듯 12장 풀해상도 스크린샷만으로도 수십 MB가 나온다. 그래서 (2) 자동 스케일보다 (1) opt-in 설정 키(예: inboundRequestBodyLimitBytes, 기본 256 MiB 유지)가 #3196 패턴과 가장 잘 맞고, OOM 리스크도 운영자가 의도적으로 연다. (3) 에러 메시지는 설정 없이도 싸게 개선할 수 있다.

현재 머지 트레인(OAuth/Kiro #3561/#3562, #3348 combo 바운디드 스택, Windows suite)과는 파일이 겹치지 않는다. types/config 스플릿 캠페인에 새 키를 넣을 때는 types/config zod 위치에만 조심하면 되고, 이 이슈 자체는 close-don't-rebase 대상이 아니다. 열린 이슈 중 같은 inbound 256 MiB / remote compact 413 중복은 보이지 않는다. 우선순위는 922k opt-in 사용자만, 워크어라운드(일찍 /compact 또는 272k 유지) 있음, 보안·메모리 트레이드오프 있음이라 중상위 enhancement로 둔다.

src/server/request-decompress.ts 22 / 237–238 - 인바운드 한도가 상수 256 MiB로 고정이고 readJsonRequestBody가 설정을 받지 않는다.
src/server/responses/compact.ts 498 - remote compact가 같은 readJsonRequestBody를 써서, 큰 세션일수록 compact 요청 자체가 413으로 실패한다.
src/server/responses/core.ts 2724 / 1438대 - 일반 /v1/responses도 동일 캡·에러 경로라, compact만이 아니라 full-history 재전송 전반에 영향이 있다.
src/types/config.ts maxUpstreamBodyBytes / #3196 - 아웃바운드 opt-in 선례. 인바운드도 기본 유지 + 상한 올리기 옵션이 대칭이다.
경로 modelContextWindows / GPT-5.6 922k - 창 설정과 decompress 캡이 연결되어 있지 않아 opt-in 큰 창이 silent break를 만든다.

메인테이너의 판단이 필요한 지점

• (1) 설정 키만 둘지, (2) 창 크기 연동 자동 스케일까지 갈지, (3) 메시지 개선만 먼저 할지.
• 상한을 올릴 때 프로세스 RSS/번역 버퍼(TranslatorBudget)·동시 연결과의 OOM 가드 정책(절대 상한 캡이 필요한지).
• 키 이름을 inboundRequestBodyLimitBytes로 할지, #3196과 대칭인 다른 이름/네임스페이스로 할지.
• types.ts/config.ts 스플릿·대형 트레인 뒤 순서 vs help wanted로 외부 PR 대기.

너의 추천
라벨 enhancement/proxy 맞다. 1차 구현은 #3196 대칭으로 기본 256 MiB 유지 + opt-in 상한 설정을 readJsonRequestBody/readBoundedJsonRequestBody에 주입하고, 413 본문에 inbound decompressed body limit / earlier compact or raise limit 안내를 넣는 쪽을 추천한다. 창 자동 스케일은 2차로 두자(바이트 추정 부정확). 922k를 쓰는 운영자만 설정을 올리면 되고, 기본 사용자는 OOM 가드가 그대로다. 우선순위 50 — 재현·원인 명확, 영향 범위는 opt-in 대형 창, 지금 핫픽스 트레인은 아님.

이 댓글은 grok-bot이 작성했습니다

[nowhere1975]

Thanks for the detailed review — here are content-free measurements from the actual failing request, plus a before/after control that directly brackets the request size.

Environment

• Client: Codex App 26.901.41123, bundled codex CLI 0.153.3 (release 26.901.41123), macos arm64
• Proxy: opencodex 2.42.0 (at failure time), inbound via /v1/responses
• Model: gpt-5.6-sol, native (openai forward provider)
• Model context window: 922000 (opt-in via modelContextWindows); session input at the time ≈ 578K–650K tokens/request (measured from usage records of the same conversation before/after)

The failing request (09-05 01:27–01:34 UTC)

The 413 burst came from one explicit compaction of a long-lived rolling session (created 07-31; rollout journal is 50,257 lines with 27,255 response_items; the session had already performed 56 window compactions — "compacted" records with replacement_history — so it is a multi-window session, not a single giant context).

• Codex App log shows thread/compact/start at 01:28:28 UTC (conversation 019fb74b-…).
• Proxy usage log records 21 consecutive 413s between 01:27:59 and 01:34:10 UTC:
  Decompressed request body exceeds 268435456 bytes
  • provider/model = unknown/unknown in all of them → the body is rejected before parsing (as your review notes, at the decompress admission cap), so no upstream error body exists to quote.
  • Each attempt failed in ~0.8–1.7 s (no upstream round-trip).
• The 21 attempts are the Codex client's own retry loop re-sending the same oversized compact body (intervals 3–15 s).

Session content composition (rollout journal, serialized bytes per item type)

This is the full journal, not the compact payload, but it shows where the bytes live:

item type	count	serialized bytes	avg bytes
reasoning	9,557	23,487,688	2,457
custom_tool_call	7,132	9,266,888	1,299
custom_tool_call_output	7,132	92,006,806	12,900
message	1,951	37,892,151	19,421
function_call	719	340,230	473
function_call_output	719	27,543,019	38,307
agent_message	43	116,934	2,719
tool_search_call/output	2	17,712	—
TOTAL	27,255	190,671,428 (181.8 MiB)	—

Tool outputs dominate (custom_tool_call_output 92.0 MB + function_call_output 27.5 MB ≈ 62.7% of bytes). There are no input_image data-URL parts in this session (text/tool-output only); the 256 MiB crossing therefore comes from the compaction replay shape carrying the session's accumulated tool outputs across window boundaries.

Before/after control (local patch, same session, same client)

I temporarily raised the inbound decompress cap to 1.5 GiB in the proxy source (issue #3573 workaround) and restarted:

• Before (256 MiB cap): the compact request → 413 ×21, session unusable.
• After (1.5 GiB cap): same conversation resumed and ran normally:
  • 09-05 01:45:57 — input 578,493 tokens → 200
  • 09-05 01:47:04 — input 580,235, cached 578,432 → 200
  • later requests up to input 650,395 tokens → 200 (cached 650,112)
• Prompt cache also recovered immediately (cached ≈ input − ~2K), so the higher cap did not perturb the upstream cache.

Since the same client+session succeeds at 1.5 GiB and fails at 256 MiB, the failing compact body is definitively between 256 MiB and 1.5 GiB decompressed — consistent with the 181.8 MiB journal being expanded further by compaction replay framing (system prompt + all window summaries + repeated tool outputs).

Requests

1. An opt-in inboundRequestBodyLimitBytes config key (default 256 MiB) as suggested — that would have unblocked this session without a source patch.
2. For comparison: same-day failures on third-party providers show the replay shape is the issue, not just the proxy cap — DeepSeek rejected the same kind of compact with "you requested 7,499,487 tokens" (>1M cap) and zhipu/kimi with body/context-limit errors. A diagnostic that reports the compact input's decompressed size + item counts (like the table above) in the 413/400 message would help operators decide (compact earlier vs raise limit).

Happy to provide more measurements (e.g. gzip Content-Length from a re-captured attempt with the cap restored) if useful.

[Ingwannu]

The exact decompression error is strong evidence that OCX's 256 MiB admission limit blocked these requests before routing. Thank you for providing content-free measurements.

One qualification: the posted post-change 200 rows are normal model requests. Unless the same compact request was observed completing under the larger limit, they do not by themselves prove an upper bound for the original compact payload. Likewise, the journal's serialized size is not the transmitted compact body's size. The decisive next measurement is the decompressed byte count for that exact compact attempt and its endpoint/result, without its contents.

An opt-in limit cannot be implemented by changing the decoder constant alone. Current src/server/index.ts also passes that constant to Bun.serve({ maxRequestBodySize }); the runtime listener and decompression reader must use one validated policy. Preserve the default, bound the opt-in maximum, and cover plain/compressed requests and concurrency/memory behavior. A 1.5 GiB body can cost substantially more than that once JSON and replay structures are materialized, so it is not a safe general default or a sufficient memory budget by itself.

A bounded byte-count diagnostic is useful; item counts may not be available when admission rejects the body before parsing, so the 413 path must not parse/retain the rejected payload merely to report them. Keeping the issue open; no global size increase or retry-policy change has been applied.