[Bug]: an empty forced-answer pass fails the whole web-search turn instead of retrying once

[cortes-ventures]

Client or integration

Codex App

Area

Tools / MCP / web search

Summary

A hosted web-search turn aborts with stream disconnected before completion: forced-answer pass produced no usable assistant output when the forced-answer pass ends with a terminal done, no visible assistant text, and no tool call.

#1001 fixed the original silent-success defect by turning that case into a hard LoopError(502, …) at src/web-search/loop.ts:853, and the merged fix (PR #1030) took the second of the two options the report offered — "surface an explicit in-stream error". The first option, "recover with a bounded final-answer retry", was never implemented. As a result one empty final pass ends the entire turn, even though every search result needed to answer is already in the request and nothing about the response is structurally broken.

Expected: when the forced-answer pass produces no usable output and the response is not malformed, re-ask once with the results already gathered; fail the turn only if that retry is also empty. Malformed tool calls should keep failing immediately, because they report an upstream protocol problem.

The upstream model in the observed failure is deepseek-v4.1-flash over the OpenCode Go subscription. An occasionally empty assistant message on a tool-free request is upstream behavior, not a protocol violation, and it is recoverable — the search results are already in context.

Reproduction

1. ocx start with hosted web search and Codex App connected to the proxy.
2. Route a model that intermittently returns an empty assistant message for a tool-free request (observed: deepseek-v4.1-flash via the OpenCode Go subscription).
3. Ask a question that consumes the search budget (default maxSearches=3), so the loop sends the forced-answer request with the synthetic web_search tool removed.
4. The forced pass returns HTTP 200, a terminal done, no assistant text, and no tool call.
5. The stream disconnects before completion with the message above and the gathered results are discarded.

Deterministic reproduction on dev at 7a0513c (2.52.0) with the existing runWithWebSearch harness:

• Pass 1 emits a batched web_search call, consuming the budget.
• The forced pass emits done with no text_delta and no tool call.

Current dev replays the empty passthrough and then throws:

throw new LoopError(502, "forced-answer pass produced no usable assistant output");

Version

@bitkyc08/opencodex 2.51.0 installed; the defect is present on dev at 7a0513c (2.52.0).

Operating system

macOS 26.6.2 arm64; client is Codex App.

Provider and model

opencode-go (Console Go) subscription route to deepseek-v4.1-flash, adapter openai-chat. The failing request is the forced-answer pass of the hosted web-search loop.

Logs or error output

stream disconnected before completion: forced-answer pass produced no usable assistant output

Code path: src/web-search/loop.ts:853 (LoopError(502, …) inside the forceAnswer terminal check).

Screenshots and supporting files

A fix with regression coverage is linked below as a pull request.

Checks

• I searched existing issues and documentation.
• I removed secrets, tokens, account details, request credentials, personal data, local paths, session IDs, and request IDs.

[lidge-jun]

리뷰 · 우선순위 55 / 80

이 이슈는 호스티드 웹서치의 forced-answer 패스가 빈 done으로 끝나면 턴 전체가 forced-answer pass produced no usable assistant output으로 끊기는 문제입니다. 지금 dev(8fc63277d)의 src/web-search/loop.ts는 #1001/#1030 이후 그 지점에서 바로 LoopError(502, …)를 던집니다. 목적(침묵 성공 금지)은 맞지만, 검색 결과가 이미 요청 안에 있는 상태에서 upstream이 가끔 빈 어시스턴트 메시지를 주는 경우까지 복구 없이 죽입니다. 보고된 경로(OpenCode Go → deepseek-v4.1-flash, Codex App)는 "프로토콜 깨짐"이 아니라 "빈 최종 답"에 가깝습니다.

이슈가 기대하는 동작도 분명합니다. malformed 툴 콜은 즉시 실패(재시도해도 같은 프로토콜 문제), 텍스트도 툴도 없는 침묵만 결과 유지한 채 한 번 더 묻고, 그것도 비면 그때 502. #1001이 적어 둔 두 옵션 중 "bounded final-answer retry"를 이제야 집어 올리는 후속입니다.

코드 위치 지적은 현재 HEAD와 맞습니다. forceAnswer 분기에서 terminal done을 찾고 hasMalformedToolCall 또는 visible text 없음을 본 뒤 throw합니다. harness로 pass1=web_search, pass2=빈 done을 돌리면 결정적으로 재현된다는 설명도 runWithWebSearch 구조와 일치합니다.

이미 수정 PR #4316이 있습니다. emptyAnswerRetries 1회, HARD_CAP +1, 복구 nudge + toolChoice: "none", malformed는 재시도 안 함, 테스트 4개가 이슈의 Expected와 같은 표입니다. 키풀/Z.AI 트레인과 무관한 웹서치 루프 국소 수정입니다.

src/web-search/loop.ts (forced-answer throw) - HEAD에서는 여전히 즉시 실패합니다. #4316 랜딩 전까지 증상 유지입니다.

malformed vs empty - 이슈가 둘을 나눈 기준이 맞습니다. malformed는 upstream 프로토콜 신호라 재시도가 독이 되고, empty는 같은 컨텍스트로 한 번 더 묻는 편이 사용자 피해가 작습니다.

중복/ Closable - 별도 중복 이슈로 보이진 않고, #1001은 이미 닫힌 선행입니다. 이 티켓은 그 후속 복구 이야기로 단독 유지가 맞습니다.

메인테이너의 판단이 필요한 지점

• #4316을 얼마나 빨리 draft 해제·머지할지
• 복구 정책을 1회로 고정할지(이슈/PR 모두 1회)
• 특정 모델(deepseek flash 계열)만 우회할지 — 비추천, 루프 일반 복구가 맞음

너의 추천
서버/모델 특수 케이스로 키우지 말고 #4316을 랜딩한 뒤 이 이슈를 닫으세요. 재현·기대 동작·회귀 테스트가 이미 PR에 묶여 있습니다. types/config 스플릿과 무관합니다.

이 댓글은 grok-bot이 작성했습니다

[Ingwannu]

Reviewed the proposed recovery in #4316 at 61b3bbe. A single retry for a genuinely empty normal completion is useful, but the current condition also retries explicit content_filter/refusal/max_tokens outcomes, and its answer-only pass retains ordinary tools in context.tools. I requested changes with separate negative controls for those paths and the clean-empty positive control. @lidge-jun this should remain open until the corrected PR lands; increasing retries or hiding a refusal behind a completed response is not the fix.

[cortes-ventures]

Corrected PR #4316 is up at 05e776325, rebased onto dev dcd13b435. What changed against the review on this issue:

• recovery is limited to a genuinely clean empty done. A terminal whose stopReason is in the bridge's truncation vocabulary (content_filter, refusal, max_tokens, and the raw provider spellings mapped to them) is replayed unchanged, so the turn still ends response.incomplete and spends no second upstream call — a filtered turn cannot become a success;
• the recovery request advertises no tools at all: context.tools = [] and toolChoice: "none", pinned by a new fixture that carries web_search plus an ordinary client tool (a toolChoice-only regression now fails the test);
• HARD_CAP is maxSearches + 2; cancellation does not spend the recovery attempt; the persistent-empty and malformed controls are unchanged; the search/result history stays untouched;
• structure/runtime.md records the retry contract and its cost: one extra upstream call on the clean-empty path, none for a truncated terminal.

Verification on that head: bun test tests/web-search/web-search.test.ts 71 pass / 0 fail; bun run typecheck, bun run structure:check, bun run privacy:scan clean; bun run test:changed 14,565 pass / 0 fail across 647 files.

The PR stays in draft until the reviewer verifies this exact head, so this issue stays open until then.

[lidge-jun]

Landed via #4356 at dcd2d07 — emptyAnswerRetries + truncation-safe replay is on dev (src/web-search/loop.ts).