v0.8.63: Add visual inspection artifacts for browser and UI tasks

[Hmbown]

Research signal

Cursor's browser/canvas Design Mode is not just visual polish. It gives the agent a richer evidence loop for UI work: selected elements, layout relationships, code context, screenshots, and queued follow-up input.

Sources checked:

• Cursor changelog, Design Mode Improvements: https://cursor.com/changelog
• Cursor Cloud Agents docs, artifacts and remote desktop control: https://cursor.com/docs/cloud-agent
• Cursor 0.50 changelog, Agent integration and faster precise edits: https://cursor.com/changelog/0-50

For CodeWhale, the lesson is not to clone Cursor's IDE UI. The lesson is to make visual evidence a first-class harness artifact so an agent can verify what changed instead of only editing text and guessing.

CodeWhale substrate

CodeWhale already has:

• runtime APIs for future GUI/editor integrations
• durable task timelines and artifact references in the architecture
• tool execution receipts and post-edit diagnostics
• local-first privacy boundaries
• v0.8.60: Group background tasks into workflows with phase summaries and collapsed Bash runs #3082 for workflow/task grouping
• v0.8.60: Add agent run ledger with follow-up, takeover, and artifact receipts #3142 for run-level artifact receipts once filed

Proposal

Add a visual inspection artifact loop for browser/UI tasks.

Minimum viable 0.8.59 shape:

• A local browser/UI inspection tool can capture screenshots and basic metadata for a target URL or app surface.
• The artifact is attached to the active task/run/thread, not pasted as opaque prose.
• The agent can ask for an element selection or accept a selector/path/screenshot crop from the user.
• The run summary records before/after screenshots, console/log excerpts, and verification status.
• The TUI can show a compact artifact reference even if full image viewing belongs to future desktop/web clients.

Possible later shape:

• DOM selector plus bounding-box mapping.
• Visual diff/crop attachments.
• Click-to-code mapping when a GUI/editor client can provide it.
• Remote-desktop-like takeover for cloud or VM-backed runs.

Acceptance criteria

• A task/run/thread can attach screenshot/log artifacts with stable ids and file paths.
• A browser/UI inspection command or tool captures at least screenshot, URL, viewport, timestamp, and optional console errors.
• Artifacts are visible from TUI and API/headless inspection without dumping binary data into chat history.
• A before/after artifact pair can be attached to a verification step.
• The agent receives a concise, structured summary of the visual artifact and can reference the artifact id in follow-up work.
• The implementation preserves local-first behavior and does not upload screenshots unless the user explicitly chooses a remote integration.
• Tests cover artifact metadata persistence and redaction/omission of oversized or sensitive artifact contents.

Non-goals

• Do not build a full IDE Design Mode in 0.8.59.
• Do not require a hosted browser or cloud desktop.
• Do not make screenshots part of the model prompt by default; attach summaries and refs first, with opt-in image inclusion later.

Related

• v0.8.60: Group background tasks into workflows with phase summaries and collapsed Bash runs #3082: workflow/task grouping.
• v0.8.61: Split sub-agents into a headless worker runtime with lightweight TUI projections #3096: headless worker runtime.
• The new run-ledger issue should become the canonical parent surface for these artifacts.

[Hmbown]

Related Cursor-harness follow-up issues filed from the same research pass:

• v0.8.60: Add agent run ledger with follow-up, takeover, and artifact receipts #3142
• v0.8.59: Add prompt source map and context-usage report for rules, tools, memory, and skills #3143
• v0.8.64: Add natural-language auto-review policy and a pre-push review gate #3144