feat(api): forward pasted images through the claude CLI transport#159
Merged
Conversation
Image blocks from the latest user message ride the stream-json stdin envelope in the same Anthropic wire shape the direct API path uses, placed before the text block. Image-only prompts omit the empty text block. Earlier turns' images are not re-sent: a resumed session already has them and a flattened restart lost the non-text content. Capability image_input flips to true for the CLI transport.
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
There was a problem hiding this comment.
Pull request overview
This PR updates the Claude CLI transport (the keyless path that shells out to the claude binary) to forward pasted images from the latest user message as standard Anthropic image content blocks, making image prompts behave consistently with the direct API transport.
Changes:
- Add
last_user_images()to extract image blocks from the latest user message only. - Update stdin stream-json envelope generation to include image blocks (before text) and to omit the text block for image-only prompts.
- Thread images through the CLI spawn paths and advertise
image_input: truein provider capabilities, with targeted tests for ordering and extraction rules.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+119
to
+124
| .map(|message| match &message.content { | ||
| MessageContent::Blocks(blocks) => blocks | ||
| .iter() | ||
| .filter(|block| matches!(block, ContentBlock::Image { .. })) | ||
| .cloned() | ||
| .collect(), |
Comment on lines
+196
to
+199
| let mut content: Vec<Value> = images | ||
| .iter() | ||
| .filter_map(|block| serde_json::to_value(block).ok()) | ||
| .collect(); |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
claudebinary as standard Anthropic image blocks, so image prompts work the same as on the direct API path.Context
image_input: falseand dropped pasted images silently.crates/api/src/providers/claude_cli.rsonly — image extraction from the latest user message, the stdin envelope shape, and the capability flag.pub(crate)for a nonexistent "opencode CLI provider" — dropped as out of scope (providers are Claude and Codex only).Changes
last_user_images(): image blocks from the latest user message.stdin_line(prompt, images): images serialized in the Anthropic wire shape, placed before the text block (matchinganthropic.rs); image-only prompts omit the empty text block.spawn_turn(...)threads images through both the resume and fresh-session paths.image_input: truefor the CLI transport.Validation
git diff --check— cleancargo fmt --all—--checkcleancargo check --workspace— cleancargo clippy --workspace --all-targets -- -D warnings— cleancargo test --workspace— not run in full; see targetedcargo test --package claurst-api -- claude_cli(12 passed, incl. 3 new image tests)PR Readiness