Skip to content

fix(ai): normalize tool-result image data URLs - #1421

Open
Altairpaca wants to merge 4 commits into
code-yeongyu:mainfrom
Altairpaca:fix/1260-tool-result-image-data-url
Open

Altairpaca wants to merge 4 commits into
code-yeongyu:mainfrom
Altairpaca:fix/1260-tool-result-image-data-url

Conversation

@Altairpaca

@Altairpaca Altairpaca commented Sep 7, 2026

Copy link
Copy Markdown

Summary

Fixes #1260 by restoring the ImageContent.data contract at the shared message-normalization boundary before provider serialization.

Tool results occasionally contain an already-prefixed data:<mime>;base64,... string even though ImageContent.data is defined as raw base64. Responses adapters then add their own prefix and produce an invalid double-prefixed image_url.

Root cause

transformMessages() normalized unsupported media and tool-call IDs but passed supported tool-result image payloads through unchanged. Provider adapters therefore had to trust that ImageContent.data was raw base64, and an already-prefixed value leaked into the Responses serializer.

Changes

  • Normalize tool-result media blocks in transform-messages.ts.
  • If data starts with the exact canonical prefix for its declared MIME type (data:${mimeType};base64,), strip that prefix and keep raw base64 internally.
  • Leave ordinary raw base64 byte-identical.
  • Do not accept/normalize arbitrary or mismatched data: payloads.
  • Add deterministic regression coverage for raw base64 and already-prefixed image data through convertResponsesMessages().
  • Add exact-nearest packages/ai/src/changes.md coverage for the shared normalization path.

Scope / risk

The change is limited to tool-result media normalization. User attachments are unchanged. Provider serializers retain their existing raw-base64 contract and continue adding the wire prefix themselves.

Validation / current sync

The branch is synchronized with upstream main at 464f1a54e8569cdfe8e4620583e19d594a6b96e9 (head bdc33c87e57faeb1ca34719972707ef5248fc45d). Current main still lacks tool-result data-URL normalization in transform-messages.ts, so #1260 remains live upstream. The semantic runtime diff remains exactly the production normalizer plus its focused regression; the third changed file is repository-policy tracker coverage.

PR-triggered CI, Changelog gate, and model-catalog workflows for this fork contribution may stop at action_required before jobs are created; that is an approval gate rather than executable test evidence. Any executable failure remains blocking.

Per CONTRIBUTING.md, release CHANGELOG.md entries remain maintainer-owned.

Closes #1260

Copy link
Copy Markdown
Author

Implementation is ready for review. The branch is current-main based and the diff is limited to the shared tool-result normalization seam plus the focused #1260 regression. All three fork workflows are currently action_required with no jobs executed; when convenient, please approve the workflow run. I’ll own any CI/review follow-up through merge.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Tool-result image data URLs are double-prefixed for Codex Responses, producing invalid image_url

1 participant