Commit cbd37a8
bcode
fix(provider/openai-compatible): trust user-declared modalities, don't inject fake ERROR text
unsupportedParts (transform.ts) replaces image/file parts with an inline 'ERROR: Cannot read <name> (this model does not support <modality> input)' text part when model.capabilities.input.<modality> is false. The capability is derived in provider.ts:1298-1304 — for @ai-sdk/openai-compatible (the user-configured proxy) there is no models.dev entry, so image/audio/video/pdf all default to false unless the user explicitly declares modalities in opencode.json.
This is a silent footgun in the user-configured-proxy case. The screenshot is stripped before the wire, and the model receives the fabricated 'ERROR: Cannot read image' text as if it came from the user — producing nonsensical replies like 'I can't see the image' even though upstream supports vision.
Forwarding the part for openai-compatible providers is honest: if upstream truly can't handle it, we get a real provider error from the API call, which is far more debuggable than fabricated capability text. Native providers (@ai-sdk/anthropic, @ai-sdk/openai, @ai-sdk/google, @ai-sdk/amazon-bedrock) keep the existing check because models.dev IS authoritative for them — there the filter prevents a real 4xx upstream and gives the user a clear local error.
Reproed against browser-use cloud's V4 LLM gateway (an openai-compatible proxy in front of Anthropic): vision smoketest replied 'I cannot see the image' on every run; gateway-side request audit confirmed zero image_url / file / data:image/ signals in the body even when the agent attached a screenshot via browser_execute. With this change image parts reach the wire and the model produces real vision replies.1 parent 06293d5 commit cbd37a8
1 file changed
Lines changed: 15 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
390 | 390 | | |
391 | 391 | | |
392 | 392 | | |
| 393 | + | |
| 394 | + | |
| 395 | + | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
| 403 | + | |
| 404 | + | |
| 405 | + | |
| 406 | + | |
| 407 | + | |
393 | 408 | | |
394 | 409 | | |
395 | 410 | | |
| |||
0 commit comments