Skip to content

fix(provider): scope OpenAI Responses text reconciliation to one output item - #421

Open
kevin9327 wants to merge 1 commit into
leookun:mainfrom
kevin9327:fix/responses-text-reconcile-per-item
Open

fix(provider): scope OpenAI Responses text reconciliation to one output item#421
kevin9327 wants to merge 1 commit into
leookun:mainfrom
kevin9327:fix/responses-text-reconcile-per-item

Conversation

@kevin9327

Copy link
Copy Markdown
Contributor

The bug

reconcile_response_text repairs a stream whose output_text deltas are incomplete: when response.output_text.done or response.output_item.done reports a final text longer than what was streamed, the missing suffix is emitted as one more TextDelta.

Its baseline is wrong. text accumulated every delta of the whole response, but final_text only ever describes one output item:

let mut text = String::new();          // per response
...
"response.output_text.done" => {
    if let Some(final_text) = value.get("text").and_then(Value::as_str) {
        for event in reconcile_response_text(&mut text_open, &mut text, final_text) { yield event; }
    }
if final_text.starts_with(streamed.as_str()) && final_text.len() > streamed.len() {

A Responses stream routinely carries more than one text-bearing item — the model writes a sentence, calls a tool, then writes more, and each block is its own message item. From the second item onward, streamed holds the concatenation of all previous items, starts_with is false, and the repair silently stops happening.

Failure mode

An assistant turn that ends in blank text.

When an upstream relay reports text only through terminal events and never sends response.output_text.delta — a common shape for OpenAI-compatible gateways — the reconciler is the only thing emitting text. Item 0 is emitted correctly, and every later item is dropped entirely, because the accumulator it is checked against is no longer empty:

event streamed before final_text emitted
output_item.done (message, output_index 0) "" "checking the file" TextStart, TextDelta("checking the file")
output_item.done (function_call, 1) tool call, fine
output_item.done (message, output_index 2) "checking the file" "the file looks fine" nothing

The same wrong baseline also disables the partial-delta repair for any later item, so a truncated delta stream loses its tail instead of being completed.

The fix

Track the output_index the accumulator belongs to, and clear it when the provider moves to a different item, so each item reconciles against its own deltas. This mirrors how tool state is already keyed by output_index in this file.

Two properties are preserved deliberately:

  • Events that omit output_index keep the current scope, so lenient relays behave exactly as before — nothing new is made required_u64.
  • Repeated reports for one item (response.output_text.done followed by response.output_item.done) resolve to the same index, so they still share a baseline and cannot replay the same text twice.

Verification

cargo +1.95 test --package cursor-server --lib provider::openai_responses

Before (fix neutered in place, tests kept):

---- a_later_output_item_reconciles_against_its_own_text stdout ----
assertion `left == right` failed
  left: []
 right: [TextStart, TextDelta("the file looks fine")]

---- a_later_output_item_still_repairs_a_truncated_delta_stream stdout ----
assertion `left == right` failed
  left: []
 right: [TextDelta("looks fine")]

test result: FAILED. 2 passed; 2 failed

After:

test provider::openai_responses::tests::repeated_reports_for_one_output_item_do_not_replay_its_text ... ok
test provider::openai_responses::tests::a_later_output_item_reconciles_against_its_own_text ... ok
test provider::openai_responses::tests::a_later_output_item_still_repairs_a_truncated_delta_stream ... ok
test provider::openai_responses::tests::reasoning_replay_projects_response_items_to_valid_input_items ... ok

test result: ok. 4 passed; 0 failed

Gates (make check, Rust half):

  • cargo fmt --all -- --check — clean
  • cargo clippy --workspace --all-targets -- -D warnings — clean on CI's stable; see note below
  • cargo test --workspace --all-targets — passes, apart from a pre-existing local flake in cursor_trace_queue::trace_producers_do_not_wait_for_sqlite_and_artifacts_stay_ordered (a 10 ms-poll timing assertion, 1 failure in 5 runs on this Windows box, unrelated to this change; green in CI on main).

Toolchain note: this machine's stable is broken, so the gates were run with +1.95. Clippy 1.95 reports one collapsible_match error in server/src/cursor/compile/model.rs:109 on unmodified main — that is a 1.95-only false positive (its own suggestion, "fast" if parse_bool(parameter)? =>, does not compile, since ? is not allowed in a match guard) and CI's 1.98.1 does not emit it, so main is green. Clippy here was therefore run with -A clippy::collapsible_match; nothing in this diff is affected either way.

…ut item

`reconcile_response_text` exists to repair a stream whose `output_text`
deltas are incomplete: when `response.output_text.done` or
`response.output_item.done` reports a final text longer than what was
streamed, the missing suffix is emitted as one more delta.

The comparison baseline was wrong. `text` accumulated every delta of the
whole response, but `final_text` only ever describes a single output
item. A Responses stream routinely carries more than one text-bearing
item -- the model writes a sentence, calls a tool, then writes more, and
each block is its own `message` item -- so from the second item onward
`final_text.starts_with(streamed)` compares against the concatenation of
all previous items and is false. The repair silently stops happening.

The visible failure is an assistant turn that ends in blank text. When
an upstream relay reports text only through terminal events and sends no
`output_text.delta` (a common shape for OpenAI-compatible gateways), the
first item is emitted correctly and every later one is dropped entirely,
because the accumulator it is checked against is no longer empty. The
same baseline also breaks the partial-delta repair for any later item.

Track the `output_index` the accumulator belongs to and clear it when the
provider moves to a different item, so each item reconciles against its
own deltas. Events that omit `output_index` keep the current scope, so
lenient relays behave exactly as before, and repeated reports for one
item (`output_text.done` followed by `output_item.done`) still reconcile
against the same baseline and cannot replay text twice.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@leookun

leookun commented Sep 4, 2026

Copy link
Copy Markdown
Owner

Are you a robot 😂?

@kevin9327

Copy link
Copy Markdown
Contributor Author

Human, I promise 😄 A Korean developer in my 30s who basically codes around the clock, so the PRs land at odd hours. Happy to answer anything on any of them.

Xiashangning added a commit to Xiashangning/cursor-byok that referenced this pull request Sep 5, 2026
与 Rust 侧 leookun#421(97feb2a)同源:output_text.done/output_item.done 报告的
终态文本只描述单个输出 item,整条流累积的补全基线从第 2 个文本 item 起
前缀恒不匹配,补齐(及只发终态的中继的整段文本)被静默丢弃。按事件
output_index 切换作用域并清空基线;缺 output_index 沿用当前作用域;同
item 重复终态不重放。仅改插件 SDK 协议层,codex/grok 等共用该实现的
Responses 插件同受其益。
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants