Skip to content

serve: ACCEPT frame — oversized prompt is a clean 400, not a broken 200 stream (#597 item 6) - #609

Merged
JustVugg merged 1 commit into
devfrom
fix/597-context-accept-frame
Jul 25, 2026
Merged

serve: ACCEPT frame — oversized prompt is a clean 400, not a broken 200 stream (#597 item 6)#609
JustVugg merged 1 commit into
devfrom
fix/597-context-accept-frame

Conversation

@JustVugg

Copy link
Copy Markdown
Owner

Fixes #597 item 6@mkelcb's diagnosis and protocol design.

The bug

A streaming chat request committed its HTTP 200 (SSE headers) before the engine tokenised the prompt and checked it against CTX. An oversized prompt was only discovered afterwards (ERROR CONTEXT_EXCEEDED) — too late to send the 400 the client needs, so the client got a broken/empty stream instead of a clean context_length_exceeded. (Non-streaming was already correct: it sends 200 only after generation completes.)

The fix — an acceptance frame in the engine↔gateway protocol

  • c/colibri.c mux_submit() — after all cheap validation (encode, non-empty, fits CTX) and before prefill, emit ACCEPT <id> <ntok>. Every early failure already returns first with an ERROR, so a request yields exactly one of ACCEPT or an early ERROR.
  • Engine._dispatch_stdout() — parse ACCEPT("accept", …) without popping the pending request (DATA/DONE still follow).
  • Engine.generate() — new on_accept callback; commit exactly once on the first of ACCEPT / DATA / DONE. DATA/DONE before an ACCEPT still commit, so an older engine with no ACCEPT frame keeps working; a second ACCEPT is a protocol error.
  • APIHandler.generation() streaming — defer the SSE 200 + keepalive into start_stream(), fired by on_accept. An ERROR before ACCEPT now propagates as an APIError with nothing committed → do_POST returns a normal JSON 400.

Tests

4 Engine-level (fake engine process): ACCEPT-before-DATA fires on_accept first; ERROR-before-ACCEPT raises 400 and never commits; DATA-before-ACCEPT still commits (old engine); duplicate ACCEPT is a protocol error.
1 end-to-end HTTP: a streaming context-exceeded request returns a clean 400 context_length_exceeded, not a 200 stream.
Full server suite 83/83; the C engine builds clean.

#597 is now complete

# item status
1 <|user|> turn termination #535
2 reverse-proxy Host allowlist #606
3 HTTP/1.1 desync mitigated on dev — awaiting reporter retest
4 reasoning via reasoning_content #607
5 quiet prefill keepalive #606
6 CONTEXT_EXCEEDED after 200 ✅ this PR

…en 200 stream (#597 item 6)

A streaming chat request committed its HTTP 200 (SSE headers) before the engine
tokenised the prompt and checked it against CTX. An over-long prompt was only then
found -- ERROR CONTEXT_EXCEEDED -- far too late to send the 400 the client needs, so
the client saw a broken/empty stream instead of a clean context_length_exceeded.
(Non-streaming was already correct: it sends its 200 only after generation.)

Add an acceptance frame to the engine<->gateway protocol (mkelcb's design in #597):

- c/colibri.c mux_submit(): after all cheap validation (encode, non-empty, fits CTX)
  and BEFORE prefill, emit `ACCEPT <id> <ntok>`. Every early failure already returns
  first with an ERROR, so a request yields exactly one of ACCEPT or an early ERROR.
- Engine._dispatch_stdout(): parse ACCEPT -> ("accept", {...}) without popping pending.
- Engine.generate(): new on_accept callback; commit exactly once on the first of
  ACCEPT / DATA / DONE. DATA/DONE before an ACCEPT still commit (an older engine with
  no ACCEPT frame keeps working); a second ACCEPT is a protocol error.
- APIHandler.generation() streaming: defer the SSE 200 + keepalive into start_stream(),
  fired via on_accept. An ERROR that arrives before ACCEPT now propagates as an APIError
  with nothing committed, so do_POST returns a normal JSON 400.

Tests: 4 Engine-level cases over a fake engine process (ACCEPT-before-DATA fires
on_accept first; ERROR-before-ACCEPT raises 400 and never commits; DATA-before-ACCEPT
still commits for an old engine; duplicate ACCEPT is a protocol error) + 1 end-to-end
HTTP case (streaming context-exceeded returns a clean 400 with context_length_exceeded,
not a 200 stream). Full server suite 83/83; the C engine builds clean.

Completes the #597 series (items 1/2/4/5 already on dev via #535/#606/#607).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant