ltmoerdani · ltmoerdani · Jul 20, 2026 · Jul 20, 2026
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -2,6 +2,17 @@
 
 All notable changes to the **OpenCode Go BYOK Provider** extension are documented here.
 
+## [Unreleased]
+
+### Added
+
+- **`[Vision]` Multimodal tool results — MCP screenshot forwarding (#77).** Images returned inside a `LanguageModelToolResultPart` (e.g. screenshots from `chrome-devtools-mcp`, `playwright-mcp`) are now forwarded to vision-capable models. Previously these images were silently dropped by the serialization layer and the model would report "I cannot see the image". Images are encoded as OpenAI-style `image_url` content parts and translated into the native multimodal shape for each transport: chat-completions (native array content), Anthropic messages (`tool_result.content: AnthropicContentBlock[]`), Google Gemini (`functionResponse.response.parts: [{inlineData}]`). Oversized images (>1 MB raw bytes) are replaced with an actionable placeholder note so a single full-page MCP screenshot can't push the request payload past the upstream limit. The Responses API cannot carry images in tool output and degrades to a placeholder note on that transport only.
+
+### Fixed
+
+- **`[Logging]` Model registration log spam during UI refresh.** VS Code refreshes model info on roughly a 300 ms cadence during chat UI activity; each call previously produced one log line per registered model (22+ lines per call). `provideLanguageModelChatInformation` now emits a single summary line per invocation (`Models registered: count=N provider=… first=… last=…`). Output channel is dramatically cleaner during testing.
+- **`[Logging]` Transient model-list fetch failures no longer pop a modal warning.** OpenCode's shared gateway occasionally returns transient 400/503 responses that resolve on retry within seconds, and the previous behavior called `showWarningMessage` on every failure — including from auto-registered provider variants the user may not actively use (e.g. `OpenCode Zen (Agents)`). Failures now log to the Output channel only; the bundled `fallbackModels` snapshot keeps the picker functional.
+
 ## [0.4.1] — 2026-07-15
 
 ### Added

diff --git a/docs/devlog.md b/docs/devlog.md
@@ -1,17 +1,30 @@
 # 🧠 OPENCODE COPILOT CHAT DEVLOG
-**Branch:** `main` | **Updated:** 2026-07-15 Asia/Jakarta | **Current Phase:** v0.4.1 — Vision Proxy + Context Overflow Fix + Output Popup Fix ✅ Merged
+**Branch:** `fix/issue-77-mcp-image-tool-result` | **Updated:** 2026-07-20 Asia/Jakarta | **Current Phase:** Issue #77 — MCP Tool Result Image Forwarding ✅ Fixed, pending PR
 
 ---
 
 ## ⚡ Session Handoff
 
 | Field | Value |
 |-------|-------|
-| **Last Session** | 2026-07-15 |
-| **Worked On** | Reviewed PR #76 (@Wallacy, vision proxy for text-only models + context overflow safety margin + output pane focus steal fix). Two review rounds: first round flagged 1 blocker (context overflow fix missing from code, only in CHANGELOG) + 2 formatting issues (README table collapsed, CHANGELOG `[0.3.7]` heading removed) + 2 nice-to-haves (dummy CancellationToken, quota not documented). Wallacy force-pushed `8a0d813` addressing all blockers. Re-verified: 64-token margin in `modelLimits()`, README rows split, heading restored, real `token` wired to `proxyVision`. Merged via merge commit `d2fcbe4` (NOT squash, preserved 4 commits). Post-merge: wrote feature doc `docs/features/11-20260715-vision-proxy.md`, bumped version to 0.4.1, updated CHANGELOG, updated this devlog. |
-| **Stopped At** | v0.4.1 docs + version bump complete. Pending `npm run compile` + `vsce package` + local VSIX install + commit. |
-| **Next Action** | → Compile, package VSIX, install locally for smoke test. Then commit docs + version bump. Push only after user approval. |
-| **Open Issues** | (1) VS Code API gap: thread ID → session cost. (2) Qwen image quota. (3) `qwen3.6-plus-free` tool-call loop. (4) #57/#58 agent model visibility. (5) Vision proxy quota not documented in README (minor, logged to Output channel). |
+| **Last Session** | 2026-07-20 |
+| **Worked On** | Investigated issue #77 (MCP tool result images dropped — vision-capable models couldn't see screenshots from `chrome-devtools-mcp`). Root cause: `convertMessage()` serialized `LanguageModelToolResultPart.content` via `partToText()` which silently dropped nested image `LanguageModelDataPart` (catch-all returned `""`). Pasted image attachments worked because they hit the separate top-level image handler. Fix: rewrote the tool-result branch to walk `part.content` and emit multimodal `OpenAiContentPart[]` when an image is present; updated all 4 transports (chat-completions native, Anthropic `tool_result.content: AnthropicContentBlock[]`, Google `functionResponse.parts: [{inlineData}]`, Responses API degraded to placeholder note). Two follow-up bugs uncovered during manual testing: (a) 4.6 MB payload → upstream 400 because MCP screenshot loops accumulate full-page PNGs; fixed with `MAX_TOOL_RESULT_IMAGE_BYTES = 1_000_000` size guard; (b) model registration log spam (22 lines × ~3 calls/sec) + transient model-list fetch failures popping modal warnings — both replaced with Output-channel logs. Wrote `docs/issues/34-*` + `docs/features/12-*` + updated CHANGELOG `[Unreleased]`. All tests pass (107/107), compile clean. |
+| **Stopped At** | Ready to commit on `fix/issue-77-mcp-image-tool-result` and open PR. User will push + open PR after reviewing. |
+| **Next Action** | → User reviews commit → push → open PR `fix/issue-77-mcp-image-tool-result` → merge with merge commit (NEVER squash) → closes #77 automatically via `Fixes #77` in PR body. |
+| **Open Issues** | (1) VS Code API gap: thread ID → session cost. (2) Qwen image quota. (3) `qwen3.6-plus-free` tool-call loop. (4) #57/#58 agent model visibility. (5) Vision proxy quota not documented in README (minor). (6) `estimateTokenCount` under-counts base64 payloads — no history-level image trimming yet (tracked in issue doc #34 limitations). |
+
+---
+
+## 🔬 Issue #77 — MCP Tool Result Image Forwarding — Session 2026-07-20 ✅ FIXED
+
+**Action:** Triaged issue #77 (reported by @yinhx3). Deep-dived VS Code Chat API to confirm `LanguageModelToolResultPart.content: unknown[]` may contain nested image `LanguageModelDataPart` (that's how MCP screenshot tools deliver images). Confirmed Kimi K2.7 is vision-capable via `VISION_CAPABLE_MODELS`. Designed and implemented a 4-transport fix with per-image size guard. Two pre-existing log-noise bugs fixed in the same session because they made manual testing very hard to follow.
+
+**Branch:** `fix/issue-77-mcp-image-tool-result` (created from `main` @ `55fb6ad`)
+
+**Compile:** `./node_modules/.bin/tsc -p ./` exit 0
+**Tests:** 107/107 pass, 0 regression (vision proxy, metadata, thinking, retry, usage, goUsageTracker, usageProfile)
+
+**Manual verification:** Chrome DevTools MCP + OpenCode Go model — model successfully read and described the screenshot.
 
 ---
 

diff --git a/docs/features/12-20260720-mcp-tool-result-image-support.md b/docs/features/12-20260720-mcp-tool-result-image-support.md
@@ -0,0 +1,137 @@
+**Status:** 🟢 Active
+
+# Multimodal Tool Results — Images Returned by MCP Tools
+
+**Topic:** vision / tool-calling / streaming / provider / mcp
+**Updated:** 2026-07-20
+**Tags:** #vision #tool-calling #streaming #provider #mcp
+**Issues:** [#77](https://github.com/ltmoerdani/opencode-copilot-chat/issues/77)
+**Released:** `Unreleased` (post-`0.4.1`)
+
+---
+
+## Overview
+
+Extension-side support for forwarding images that arrive inside a
+`LanguageModelToolResultPart` to vision-capable models. This is the shape
+MCP tools such as `chrome-devtools-mcp` and `playwright-mcp` use to return
+screenshots. Before this feature, those images were silently dropped at
+serialization time and the model received an empty tool result. Now they
+are encoded as OpenAI-style `image_url` content parts and translated into
+the native multimodal format of each supported transport.
+
+---
+
+## Architecture
+
+```
+MCP tool returns image
+  (chrome-devtools-mcp screenshot, etc.)
+  ↓
+VS Code delivers LanguageModelToolResultPart.content = [LanguageModelDataPart]
+  ↓
+convertMessage() walks part.content:
+  • TextPart / internal DataPart / string  → joined into toolTextParts
+  • image DataPart                        → encoded as OpenAiContentPart
+                                            {type:"image_url", image_url:{url:"data:…"}}
+                                            (subject to MAX_TOOL_RESULT_IMAGE_BYTES = 1 MB)
+  ↓
+Tool message content:
+  • string                         if no images present (byte-identical to old behavior)
+  • OpenAiContentPart[] multimodal if ≥1 image present
+  ↓
+Per-transport builder converts to native shape:
+  • chat-completions: passes the array through as-is
+  • Anthropic messages: tool_result.content becomes string | AnthropicContentBlock[]
+  • Google Gemini: functionResponse.response gains parts:[{text},{inlineData}]
+  • Responses API: image replaced with placeholder note (API limit: string only)
+```
+
+---
+
+## Configuration
+
+No settings, no toggle. If a model reports `imageInput: true` (i.e. it is in
+`VISION_CAPABLE_MODELS` or `models.dev` says it supports vision), nested tool
+result images are forwarded automatically. If the model is text-only, the
+existing **vision proxy** (see
+[`docs/features/11-20260715-vision-proxy.md`](11-20260715-vision-proxy.md))
+handles it.
+
+---
+
+## Size Guard
+
+```ts
+const MAX_TOOL_RESULT_IMAGE_BYTES = 1_000_000; // 1 MB raw bytes
+```
+
+Single images larger than 1 MB are replaced with an inline placeholder note so
+the request payload stays bounded. Typical MCP screenshots are 50–300 KB; the
+cap exists to prevent agent loops that re-capture full-page screenshots from
+producing multi-MB payloads that get rejected upstream with
+`400 Upstream request failed`.
+
+The placeholder text is:
+
+> `[Image attachment omitted: N bytes exceeds the 1000000-byte limit for tool
+> results. Ask the tool to produce a smaller screenshot or save it to a file.]`
+
+---
+
+## Per-Transport Behavior
+
+| Transport | Image in tool result | Notes |
+|-----------|----------------------|-------|
+| OpenAI chat-completions | ✅ Native — array content forwarded as-is | Used by Kimi, Mimo, GLM, DeepSeek, Grok on OpenCode Go |
+| Anthropic messages | ✅ Native — `tool_result.content: AnthropicContentBlock[]` | Used by MiniMax M2.5/M2.7, Qwen3.5/3.6/3.7-max |
+| Google Gemini | ✅ Native — `functionResponse.response.parts: [{inlineData}]` | Used by Gemini family (when available on OpenCode) |
+| OpenAI Responses API | ❌ API limit — replaced with placeholder note | `function_call_output.output` is string-only |
+
+---
+
+## Code Locations
+
+| Concern | Location |
+|---------|----------|
+| `convertMessage` tool-result image branch | `src/extension.ts` `convertMessage()` |
+| `MAX_TOOL_RESULT_IMAGE_BYTES` constant | `src/extension.ts` (near `IMAGE_TOKEN_ESTIMATE`) |
+| Anthropic tool result content | `src/extension.ts` `anthropicToolResultContent()` |
+| Responses API tool output | `src/extension.ts` `responsesToolOutput()` |
+| Google tool response content | `src/extension.ts` `googleFunctionResponseContent()` |
+| Anthropic content-block type widening | `src/extension.ts` `AnthropicToolResultBlock` |
+
+---
+
+## Verification
+
+- `tsc -p ./` — compile pass, no errors
+- `node --test out/test/**/*.test.js` — 107/107 pass, no regression
+- Manual test with `chrome-devtools-mcp` + Kimi K2.7 Code on OpenCode Go:
+  model successfully read and described the returned screenshot
+
+See [`docs/issues/34-20260720-mcp-tool-result-image-dropped.md`](../issues/34-20260720-mcp-tool-result-image-dropped.md)
+for the full investigation, timeline, and follow-up bugs uncovered during
+manual testing.
+
+---
+
+## Limitations
+
+1. **Responses API has no image support in tool output.** Only `gpt-5-codex`
+   is affected today. Images are replaced with an actionable placeholder.
+2. **No history-level image trimming.** The size guard bounds each image
+   individually but does not trim smaller images that collectively exceed
+   the model's context window. VS Code's own trimming is expected to handle
+   that, but our `estimateTokenCount` under-counts base64 payloads.
+3. **Top-level image attachments still have no size cap.** Only tool-result
+   images are bounded. Consistent limit can be added in a follow-up if
+   users hit oversized pasted images.
+
+---
+
+## Related Docs
+
+- [`docs/features/01-20260514-vision-image-input.md`](01-20260514-vision-image-input.md) — top-level image attachments
+- [`docs/features/11-20260715-vision-proxy.md`](11-20260715-vision-proxy.md) — proxy for text-only models
+- [`docs/issues/34-20260720-mcp-tool-result-image-dropped.md`](../issues/34-20260720-mcp-tool-result-image-dropped.md) — root-cause + fix writeup