Skip to content

fix(anthropic): degrade thinking replay before the turn is rejected - #4172

Open
probepark wants to merge 1 commit into
devfrom
fix/anthropic-unreplayable-latest-thinking
Open

fix(anthropic): degrade thinking replay before the turn is rejected#4172
probepark wants to merge 1 commit into
devfrom
fix/anthropic-unreplayable-latest-thinking

Conversation

@probepark

@probepark probepark commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

What

Detect locally that the latest assistant turn cannot replay its thinking block, and degrade the replay during the first build — instead of sending the request, being rejected, and repairing afterwards.

Why

Anthropic opens and closes a thinking block with no thinking_delta and no signature_delta when it withholds reasoning. The block lands in history with empty text and no signature, convertAnthropicMessages drops it, and the assistant turn goes back carrying only its tool_use. Anthropic validates the latest assistant message against the turn it produced and rejects the missing block:

messages.N.content.M: `thinking` or `redacted_thinking` blocks in the latest assistant
message cannot be modified. These blocks must remain as they were in the original response.

Captured from a live rejected request (headroom HEADROOM_DEBUG_DUMP=full):

  • 1214 messages, model claude-opus-5, stream=true
  • 29 signed-but-empty thinking blocks forwarded intact
  • latest assistant message: ['tool_use(bash)'] — no thinking block
  • the proxy's pre-compression snapshot is byte-identical for that turn, so all 29 empty blocks came from the client and the proxy emptied none

The rejection was already recoverable through the existing replay repair, but only after a full round trip. This makes the degradation happen before the request is sent.

Scope is deliberately the latest assistant turn only, because that is the turn Anthropic validates against its own output; an older hollow block must not cost the rest of the replay.

Testing

New packages/ai/test/anthropic-unreplayable-thinking.test.ts:

  • hollow latest turn -> no native thinking replayed (before fix: Expected 0, Received 1)
  • healthy signed latest turn -> both thinking blocks preserved
  • hollow block on an earlier turn -> latest turn's thinking preserved

Also:

  • bun test packages/ai/test: 2293 pass / 337 skip / 4 fail — the same 4 failures a clean checkout has (buildAnthropicAuthConfig base URL, 3x openai-responses cache affinity), confirmed by stashing this diff
  • bun --cwd=packages/ai run check: biome + tsc clean
  • artifacts/issue-3670-anthropic-cache-eval.json regenerated (it pins the sha256 of providers/anthropic.ts)

Related: #4159 stops the same rejection from recurring every turn; this one stops it from being paid at all.

GJC verdict

gajae.pr-review-verdict.v1 needs-human sha256:b8c00d85b50676cd1032fbdbe210d094dfdef5c1 reviewer:human evidence:local bun test packages/ai/test + bun --cwd=packages/ai run check

  • Target branch dev
  • bun check passes
  • Tested locally
  • CHANGELOG updated (if user-facing)
  • Verdict matches exact PR head, not earlier commit

@probepark
probepark force-pushed the fix/anthropic-unreplayable-latest-thinking branch from 471b8ad to b8c00d8 Compare August 10, 2026 08:43
@probepark
probepark requested a review from Yeachan-Heo August 10, 2026 10:01
@probepark
probepark force-pushed the fix/anthropic-unreplayable-latest-thinking branch 2 times, most recently from e7b2a3a to 75c2bcb Compare August 10, 2026 12:54
Anthropic opens and closes a thinking block without sending any delta when it
withholds reasoning. That block lands in history empty and unsigned, the message
converter drops it, and the assistant turn goes back carrying only its tool_use.
Anthropic checks the latest assistant message against the turn it produced and
rejects the missing block, so the request dies before a token streams.

The rejection was recoverable but cost a full round trip to discover. The block
is inspectable locally, so the replay degrades during the first build instead.

Captured production evidence: 1214 messages, 29 signed-but-empty thinking blocks
forwarded intact, latest assistant turn reduced to a bare tool_use, and headroom
proven not to have emptied any of them.

Lore-id: 4b83fe10
Constraint: only the latest assistant turn is checked -- Anthropic validates that turn against its own output
Rejected: send the hollow block back | no signature, Anthropic rejects the shape
Rejected: drop thinking always | throws away valid reasoning replay on healthy turns
Confidence: high
Scope-risk: narrow
Reversibility: easy
Tested: hollow latest turn degrades, healthy latest turn preserved, older hollow turn ignored
Not-tested: live long-session request against api.anthropic.com
@probepark
probepark force-pushed the fix/anthropic-unreplayable-latest-thinking branch from 75c2bcb to 4ddac2e Compare August 10, 2026 13:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant