images[] legazy deuda técnica

[Saul-Gomez-J]

---

id: 0012
title: MCP Tool Results — Remove Legacy images[] Format After Canonical Rollout
labels: [mcp, technical-debt, replay, persistence, cleanup]
priority: P1
role: Platform Engineer
depends_on: [0003]

Summary

Levante currently carries legacy compatibility for rich tool outputs that still use an internal images[] array with inline base64 payloads. This format must be removed after the canonical tool-result rollout is fully implemented and historical conversations have been normalized.

The long-term target architecture is:

• no new images[] outputs produced anywhere;
• no images[] persisted in messages.tool_calls;
• no replay path that depends on images[];
• a single canonical persisted format based on CanonicalToolResultV1 + image-ref assets on disk.

Problem Analysis

Why this debt exists

The current codebase mixes:

• legacy rich tool outputs with images[];
• sanitized MCP content[] tombstones;
• replay from persisted tool_calls;
• toModelOutput() logic that should be the single source of truth.

To avoid breaking existing history, the canonical rollout keeps temporary read compatibility for legacy images[] payloads. That compatibility is necessary during migration, but it must not become permanent.

Risks of keeping legacy images[] indefinitely

• Ambiguous data model for rich tool outputs
• More branches in sanitizer / replay / persistence code
• Higher chance of accidentally persisting base64 again
• Harder reasoning about what format is authoritative
• More maintenance cost when adding providers or new tool result types

Scope

In Scope

• Remove legacy images[] support from new code paths after migration window
• Delete compatibility branches that only exist for legacy tool outputs
• Tighten tests to only accept canonical tool results for new flows
• Verify that all persisted histories have either:
  • canonical image-ref based outputs, or
  • safe degraded text-only legacy fallbacks

Out of Scope

• The initial canonical rollout itself
• Reworking unrelated MCP output budget logic
• UI redesign of screenshot or widget rendering

Acceptance Criteria

• processToolResult() never produces images[]
• sanitizeMessagesForModel() no longer contains legacy images[] handling for normal flows
• materializeToolResultForModel() no longer needs the legacy { text, images[] } branch
• chatService no longer rewrites legacy images[] on read because all supported histories are already normalized
• toolOutputSanitizer.ts no longer preserves or reasons about images[]
• All relevant tests assert canonical tool result shapes only
• No output.images[].data appears in context diagnostics for supported flows

Deliverables

• Removal of legacy images[] branches from:
  • src/main/services/ai/toolMessageSanitizer.ts
  • src/main/services/toolResults/canonicalToolResultService.ts
  • src/shared/toolOutputSanitizer.ts
  • any replay compatibility helpers created during canonical rollout
• Test cleanup removing legacy fixtures that are no longer supported
• Short migration note in the canonical tool-result plan/handoff documenting that the cleanup was completed

Suggested Execution Strategy

Phase 1 — Verify migration is complete

• Confirm canonical rollout is in production locally
• Scan representative DB histories and confirm no active rows depend on images[]
• Confirm new conversations only persist canonical tool results

Phase 2 — Delete compatibility code

• Remove legacy parsing/materialization branches
• Remove legacy tests and fixtures
• Update comments/docs that still mention images[] as supported runtime shape

Phase 3 — Validate replay and persistence

• Replay old normalized conversations
• Verify screenshots still reach the provider as image-data
• Verify no base64 remains in persisted tool_calls

Success Metrics

• Single canonical mental model for rich tool outputs
• Fewer branches in replay and sanitizer code
• Zero persisted base64 image payloads from MCP tool results
• Lower maintenance burden for future multimodal changes

Notes

This issue should only be tackled after the canonical tool-result migration plan is fully landed.

Primary reference:

• docs/PLAN_MCP_CANONICAL_TOOL_RESULTS.md