fix(ai): normalize tool-result image data URLs - #1421
Open
Altairpaca wants to merge 4 commits into
Open
Altairpaca wants to merge 4 commits into
Altairpaca wants to merge 4 commits into
Conversation
Author
|
Implementation is ready for review. The branch is current-main based and the diff is limited to the shared tool-result normalization seam plus the focused #1260 regression. All three fork workflows are currently |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1260 by restoring the
ImageContent.datacontract at the shared message-normalization boundary before provider serialization.Tool results occasionally contain an already-prefixed
data:<mime>;base64,...string even thoughImageContent.datais defined as raw base64. Responses adapters then add their own prefix and produce an invalid double-prefixedimage_url.Root cause
transformMessages()normalized unsupported media and tool-call IDs but passed supported tool-result image payloads through unchanged. Provider adapters therefore had to trust thatImageContent.datawas raw base64, and an already-prefixed value leaked into the Responses serializer.Changes
transform-messages.ts.datastarts with the exact canonical prefix for its declared MIME type (data:${mimeType};base64,), strip that prefix and keep raw base64 internally.data:payloads.convertResponsesMessages().packages/ai/src/changes.mdcoverage for the shared normalization path.Scope / risk
The change is limited to tool-result media normalization. User attachments are unchanged. Provider serializers retain their existing raw-base64 contract and continue adding the wire prefix themselves.
Validation / current sync
The branch is synchronized with upstream
mainat464f1a54e8569cdfe8e4620583e19d594a6b96e9(headbdc33c87e57faeb1ca34719972707ef5248fc45d). Currentmainstill lacks tool-result data-URL normalization intransform-messages.ts, so #1260 remains live upstream. The semantic runtime diff remains exactly the production normalizer plus its focused regression; the third changed file is repository-policy tracker coverage.PR-triggered CI, Changelog gate, and model-catalog workflows for this fork contribution may stop at
action_requiredbefore jobs are created; that is an approval gate rather than executable test evidence. Any executable failure remains blocking.Per
CONTRIBUTING.md, releaseCHANGELOG.mdentries remain maintainer-owned.Closes #1260