[Verlintas]

Problem

web_research could read DOM text but multimodal models could not see the page. Charts, canvas, images, maps and visual layout were invisible. take_screenshot returned only a file path, and tool results had no image channel.

What changed

• ToolResult.Success gains images: List<ToolImage> (base64 kept in memory only).
• AgentLoop inserts a synthetic user image message after a tool batch, reusing the existing OpenAI / Anthropic / Gemini user-image serialization. The message is marked as untrusted visual data.
• Images are sent for the next model round only; later rounds keep the text marker and OCR, so screenshots are not re-sent or persisted.
• web_research gains screenshot:
  • viewport PNG (max dimension 1600) captured through WebView View.draw();
  • ML Kit OCR is always returned;
  • known vision families get the image automatically; vision=true/false overrides.
• VisionModels covers GPT-4o/4.1/5/6, Claude 3/4/5, Gemini, Grok, Qwen-VL, LLaVA, Pixtral, GLM-V, etc. Unknown models stay OCR-only.
• web_search description now points to web_research open + screenshot for rendered pages.
• Docs/changelog/roadmap updated; debug smoke logs image count.

Why View.draw() instead of the virtual-display ImageReader

On the API 35 emulator the reader frame contained only the WebView window background (white + scrollbar) while Chromium composited the page on another layer. Drawing the viewport directly produced the real page. A four-attempt retry handles first-draw warmup; completely blank frames return an actionable failure.

Verification

• API 35 emulator, local CSS chart page: 720×1600 PNG / 135 KB; OCR read chart values and the phrase cobalt lantern 47; images=1 with vision=true.
• Canvas-only page (DOM contains only a <canvas>): 76 KB PNG; OCR read 31415 and the bar values; proves DOM-invisible content now reaches the model.
• vision=false: same OCR, images=0.
• Unit tests: ToolResult image round-trip, VisionModels detection, synthetic user image message, and old screenshots not resent on later rounds.
• ./gradlew test lintDebug :app:assembleDebug green.

Limits / follow-ups

• Viewport only; no full-page/stitched capture yet.
• View.draw() may miss video/SurfaceView layers; 2D canvas, HTML/CSS and images are covered.
• Screenshots leave the device only when vision is enabled; password/payment/captcha pages should use show_to_user first.
• Phase 2: full-page capture, user-visible screenshot receipt, unify take_screenshot / screen_ocr, sensitive-page policy.