Interactive tool for Snapchat ambiguous overlay matching

[ethanriverpage]

Current behavior

When a Snapchat message contains multiple videos with multiple overlays at the same timestamp, Memoria can't determine which overlay belongs to which video. These cases are saved to needs_matching/ for manual review.

Users must read match_info.json, view each video and overlay, determine correct pairings, and manually process them outside of Memoria. This is tedious and requires technical knowledge.

Why this is a problem

• Manual work required - users must handle these outside Memoria
• Technical knowledge needed - not obvious how to apply overlays manually
• Videos remain unprocessed - stuck in needs_matching folder
• Feels like failure - even though it's just ambiguous matching

When this happens

Common scenario: User sends 2 videos with custom text overlays in a single message, all timestamped identically. Creates 4 possible pairings - which overlay goes on which video?

Frequency: Relatively rare (most messages have 1 video or no overlays), but painful when it happens.

Proposed solution

Create an interactive CLI tool that guides users through matching:

./tools/match_snapchat_overlays.py /path/to/needs_matching/

The tool would:

1. Show message context (conversation, sender, message text, timestamp)
2. Display video/overlay info (resolution, duration, file names)
3. Prompt for matches with simple input
4. Process matched pairs using same overlay embedding logic
5. Move to output and clean up needs_matching folder

Implementation ideas

• Basic CLI tool first:
  • Read match_info.json for context
  • List videos and overlays with metadata
  • Prompt user for letter-number matches
  • Confirm before processing
• Enhanced features (optional):
  • Extract video thumbnails with ffmpeg for visual preview
  • Show overlay previews
  • Batch process multiple ambiguous cases
  • Allow skipping/deferring
• Web UI (future enhancement):
  • Drag-and-drop matching interface
  • Side-by-side video playback with overlay preview

Why this matters

• Completes the workflow - users can fully process all Snapchat content
• Guided process - no manual file operations needed
• Visual preview - helps make correct matches
• Preserves context - videos get proper metadata and conversation info

Alternative approach

Could implement heuristic matching (compare durations, resolutions, OCR on overlay text), but this is much more complex and error-prone. Interactive tool is simpler and more reliable.

Related

• Snapchat Export Guide: "Planned Feature: Automated interactive matching tool"
• FAQ: "Why are some Snapchat messages in needs_matching?"

Files to create

• tools/match_snapchat_overlays.py - main interactive tool
• docs/Snapchat-Matching-Tool.md - usage guide (optional)