Roadmap: H3 multishot era — battery results, unlock stack, and the next build wave

[taskmasterpeace]

Tracking issue for everything the multishot validation opened up (2026-08-06). Results update as tests land.

Learning battery (plan: docs/plans/2026-08-06-h3-multishot-learning-battery.md)

Test	Question	Status	Result
Captain 30s	Does chaining work at all?	✅ done	PASS — 30.3s continuous, identity through both seams
T1 rap, 9:16	Musical performance + vertical	✅ rendered	30.3s @ 480×832 with audio — awaiting Robert's grade
T2 artist photo anchor	REAL artist identity through a chain	✅ rendered	Likeness PASS at 25s (frame-checked). ⚠️ FINDING: opening frames stretched — the pack resizes the portrait start_image to 16:9 with stretch mode (_resize(..., \"disabled\")); recovers as generation takes over. Fix for our integration: center-crop/pad the start image to target aspect before feeding.
T3 coverage (CU→wide→tracking)	Reframing tolerance	🔄 rendering
T4 hard location cuts	Scene-change tolerance (decides MV architecture)	🔄 queued
T5 60s six-shot verse	Long-chain drift + runtime	🔄 queued
T6 voice reference (ref2va + captain's voice)	Voice conditioning lever	🔄 queued

The unlock stack (researched, sources in repo memory)

• H3 character LoRAs: ostris AI-Toolkit added H3 T2V/I2V LoRA training 2026-08-03 (NVFP4, consumer GPUs) — already installed here; Fizgig trains from still photos. → artist identity without reference images.
• H3 native sing-lipsync claims (feed a singing voice → lip-synced video; re-lipsync existing footage). Untested locally — follow-up test after T6.
• LTX-2.3 IC-LoRA family: Motion-Track-Control (official), Cameraman (camera-move transfer from reference videos), Canny/Depth/Pose transfer. → real MV camera language on generated footage.
• LTX as H3 upscaler (community workflows; upscaler model already on disk).

Build wave (from Robert's review, 2026-08-06)

• H3-aware Prompt Enhancer redesign: click enhance → LLM analyzes the prompt → offers category directions (pick one) → generates 4 full enhanced prompts, all visible, click-to-use. Bake the H3 prompting guide into the system prompt. Edge-case tests: very long prompts (5000 chars), empty prompt, unicode, malformed LLM output fallbacks.
• Gallery: click-through lightbox — prev/next arrows + keyboard navigation instead of X-and-reopen.
• Gallery: bulk erase — multi-select + one delete.
• Prompt persistence everywhere — every render carries its prompt (.meta.json sidecars); battery outputs backfilled by hand this time.
• Characters in Playground discoverability — the machinery EXISTS (ReferencePicker + @name mentions resolve library characters + quick modes) but reads as absent. Surface it (Character button chips like Gen Space).
• Multishot into DD (plan in memory): generate_multishot on the h3 client + Playground Multishot toggle + crop-not-stretch start images + eventually Music Video sections as single chains.

Done this wave (context)

Music Video rename · compose-while-rendering · queue work surface (reorder/edit/duplicate/chains) · seed fix + seeds visible · Gallery duration/resolution/seed · credential split · model registry + honest ETAs + clapperboard.

[taskmasterpeace]

Progress (2026-08-06, overnight):

Shipped in 2611219:

• ✅ H3-aware Director's Enhance — analyze → one creative question with 4 directions → 4 full prompts all visible, click-to-use. Works TODAY on the Palette-only key (guided expander × 4); upgrades to full LLM analysis automatically when a Gemini/OpenRouter key is added. 6 edge-case tests (long/empty prompts, flaky providers).
• ✅ Gallery click-through lightbox (arrows + keyboard)
• ✅ Gallery bulk erase (Select mode + one delete)
• ✅ Battery outputs carry prompts via .meta.json sidecars (backfilled)

Battery status: T1 ✅ · T2 ✅ (likeness PASS at 25s; stretch finding logged) · T3 ✅ rendered · T4/T6/T5 re-running — ComfyUI hard-died silently at T4 shot 1 after ~3.5 chained tests (no traceback; VRAM/driver fatigue suspected). Restarted clean, resubmitted with the 60s test last. Reliability lesson for Night Shift (idea #7): a watchdog that detects silent engine death and auto-restarts/resumes is mandatory, not optional.

Characters in Playground: confirmed the machinery already exists (ReferencePicker + @name mentions + quick modes) — discoverability fix still open.

[taskmasterpeace]

Battery complete — 6/6 rendered. Results:

Test	Verdict	Evidence
T1 rap 9:16	✅ rendered clean (30.3s, 480×832 + audio)	awaiting Robert's watch
T2 artist photo anchor	✅ LIKENESS PASS	frame @25s = the exact reference artist (face/shirt/tattoos/watch), 2 seams deep. ⚠️ opening frames stretched (pack resizes portrait start_image with stretch mode — fix: center-crop before feeding)
T3 coverage	✅ rendered clean	awaiting watch
T4 hard location cuts	✅ ARCHITECTURE PASS	day sidewalk → night rooftop skyline → back to day, same violinist+scarf+instrument through every cut. Chains are NOT same-scene-locked — one chain per MV section works across locations. Finding: scripted "she", model cast a man in shot 1 and kept HIM consistent — identity machinery beats prompt details; cast deliberately in shot 1 or via reference.
T5 60s six-shot verse (MEMORY)	✅ LONG-FORM PASS	same singer/jacket/room at 5s, 30s, 55s — five seams, zero identity drift; coverage varied naturally with the script; ~70 min render
T6 voice reference (ref2va)	✅ structurally works (rendered with audio, scene as scripted, visual casting independent of voice ref)	voice-match verdict needs ears — compare against the captain clip

Reliability finding: ComfyUI hard-died silently (no traceback) after ~3.5 consecutive chain renders; clean restart + resubmit recovered everything. Night Shift needs a death-watchdog with auto-resume.

Product conclusions: the MV-as-chains design is green — T2 (real artist) + T4 (location cuts) + T5 (60s) together cover a full music-video section. Next builds, in leverage order: (1) wire generate_multishot into DD (+ crop-not-stretch start images), (2) train an artist character LoRA (AI-Toolkit H3 support already installed), (3) if T6's voice passes Robert's ears — voice refs into the chain pipeline.

[taskmasterpeace]

Benchmark intel absorbed (Robert-supplied H3 vs LTX-2.3 community test, RTX 5080) + model-aware enhancement shipped (8420939).

Model-choice rules the test confirms:

• LTX-2.3 = speed lane (2.4–6.5× faster): talking heads (slightly better skin/lip timing), drafts, IC-LoRA control. Weak: camera adherence, fast/complex motion (anatomy collapse), 1080p+ instability.
• H3 = adherence lane: precise camera, superior lip-sync/expression on 3D-cartoon + anime characters (directly the children's-stories lane), holds structure through complex motion. Native 1888p impractical (~2 min/gen-sec).
• Pruned INT8 + NVFP4 == full INT8 visually (re-confirmed; it's what we ship).

Enhance flow now speaks the selected model's language: H3 performed-audio guide / new LTX-2.3 guide (subject→action→camera→mood, specificity, simple named camera, no fast motion, audio described) / new Seedance guide (formula, camera-first, timeline prompting). _guide_for_model in enhance_prompt_handler; 8 tests.

Speed unlocks queued for a Night-Shift A/B (NOT installed): Spectrum node (+30%, xmarre/ComfyUI-Spectrum-MiniMax-H3, zero deps — caveat: changes denoising trajectory, keep off identity-critical chains until tested) and H3 Turbo LoRA (20→6-8 steps, early prototype, needs non-pruned base or community conversion). Also adopting their QoL idea: LTX live latent previews (KJ-nodes) to kill anatomically-failed runs early.