Skip to content

Reject incomplete structured LLM streams - #85

Merged
RANGROO merged 1 commit into
unstablefrom
codex/reject-incomplete-json-stream
Aug 12, 2026
Merged

Reject incomplete structured LLM streams#85
RANGROO merged 1 commit into
unstablefrom
codex/reject-incomplete-json-stream

Conversation

@RANGROO

@RANGROO RANGROO commented Aug 12, 2026

Copy link
Copy Markdown
Contributor

Problem

OpenRouter could end a streamed structured response after only a JSON prefix such as { "character":. StobeServer treated any non-empty HTTP 200 stream as successful, then fell back to speaking and persisting the raw fragment.

Root cause

  • the OpenAI-compatible streaming connector discarded provider error, finish_reason, and native_finish_reason metadata
  • the structured dialogue path accepted a JSON-looking fallback even when it had no usable message

Changes

  • capture and audit provider stream errors and finish reasons
  • reject normalized truncation, content-filter, safety, and error completions
  • reject JSON-looking structured responses without usable dialogue
  • preserve plain-text provider fallback and salvageable partial JSON that already contains dialogue
  • extend the existing structured-dialogue regression for the reported fragment

Compatibility

No database migration or configuration change. Existing plain-text fallback and usable partial structured responses remain supported.

Validation

  • PHP lint: connector/openaijson.php, lib/chat_helper_functions.php, tests/structured_dialogue_contract_regression.php
  • focused stream-failure classification probe passed, including length and Gemini-style MAX_TOKENS
  • STOBE_DB_NAME=stobe_test php tests/structured_dialogue_contract_regression.php
  • STOBE_DB_NAME=stobe_test php tests/rechat_flow_regression.php
  • STOBE_DB_NAME=stobe_test php tests/chat_flow_regression.php
  • git diff --check
  • outgoing diff secret/local-path scan

Deployment

Not deployed.

Untested limits

No live OpenRouter request or in-game Kenshi retest was performed. If a provider fails after complete dialogue sentences have already streamed, those earlier sentences cannot be retracted; the failed completion is still rejected and no raw JSON fallback is accepted.

@RANGROO
RANGROO marked this pull request as ready for review August 12, 2026 00:13
@RANGROO
RANGROO merged commit 14a2276 into unstable Aug 12, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant