From 5e8780b13712d6940c9d89b3cd12f4437ac8eca5 Mon Sep 17 00:00:00 2001 From: Calvin Grunewald Date: Wed, 21 Oct 2026 13:58:55 -0700 Subject: [PATCH] Plan for generated footage: a bible, cast, takes and briefs; crop Splice becomes the system of record for footage made with video models (Seedance through Higgsfield, driven by an assistant), and for any shoot: - Bible: characters, places, things and the movie's look, each with pictures (the first is its face) and the words that must not change. A second page of the Story view, built like the board. - Cast: beats name who and what is in them; faces on each beat card and a sheet to choose them. - Takes: footage tried for a beat with where it came from (tool, model, mode, prompt, references, job). Pending takes show while they are made; landing, choosing and removing touch only the clips takes laid in, in one undo step; extensions can skip their seam frames. - Briefs: everything to make a beat's footage, its references ranked, capped at five and written out with the frames around the beat, and an outline written the way video models read best. The Brief button copies it and shows the references in Finder. - Export a stretch (--from/--to, a clip, a beat), cut from the movie as drawn so motion matches. - Crop: cut any edge of a picture; it keeps its scale and what remains stays in place, in the viewer and in export. An adversarial review's findings are addressed (takes deleting other beats' footage, failed imports leaving takes pending, stretch exports replaying entrances, brief ordering and limits, and the layout of the bible, cast sheet and takes). Co-Authored-By: Claude Opus 5.5 --- Cargo.lock | 4 +- Cargo.toml | 2 +- README.md | 16 + crates/splice-app/src/ctl.rs | 73 +- crates/splice-app/src/editor.rs | 55 +- crates/splice-app/src/editor/agent.rs | 41 +- crates/splice-app/src/editor/inspector.rs | 61 + crates/splice-app/src/editor/plan.rs | 1640 +++++++++++++++++++++ crates/splice-app/src/editor/render.rs | 4 + crates/splice-app/src/editor/shots.rs | 4 +- crates/splice-app/src/editor/stage.rs | 14 +- crates/splice-app/src/editor/story.rs | 95 +- crates/splice-app/src/icons.rs | 1 + crates/splice-app/src/timeline.rs | 10 +- crates/splice-core/src/bible.rs | 222 +++ crates/splice-core/src/brief.rs | 364 +++++ crates/splice-core/src/export.rs | 98 +- crates/splice-core/src/lib.rs | 2 + crates/splice-core/src/look.rs | 123 ++ crates/splice-core/src/model.rs | 27 + crates/splice-core/src/story.rs | 231 +++ skill/splice/SKILL.md | 73 +- 22 files changed, 3102 insertions(+), 58 deletions(-) create mode 100644 crates/splice-app/src/editor/plan.rs create mode 100644 crates/splice-core/src/bible.rs create mode 100644 crates/splice-core/src/brief.rs diff --git a/Cargo.lock b/Cargo.lock index 2016123..a415e3c 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -5366,7 +5366,7 @@ dependencies = [ [[package]] name = "splice-app" -version = "0.2.0" +version = "0.3.0" dependencies = [ "anyhow", "blake3", @@ -5390,7 +5390,7 @@ dependencies = [ [[package]] name = "splice-core" -version = "0.2.0" +version = "0.3.0" dependencies = [ "anyhow", "blake3", diff --git a/Cargo.toml b/Cargo.toml index 76d318b..3cf2941 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -3,7 +3,7 @@ resolver = "2" members = ["crates/splice-core", "crates/splice-app"] [workspace.package] -version = "0.2.0" +version = "0.3.0" edition = "2024" rust-version = "1.90" license = "MIT" diff --git a/README.md b/README.md index fff49ad..e916b51 100644 --- a/README.md +++ b/README.md @@ -116,6 +116,8 @@ make install-app # Splice.app in /Applications length you choose. A slow zoom can run across the whole clip. - **Green screen:** key out green or blue, with strength, softness and fringe clean-up. + - **Crop:** cut any edge away. The picture keeps its scale, and what + remains stays where it was. - **Mask:** circle or box, with size, position, feather and invert. - **Sound:** volume. @@ -233,6 +235,20 @@ while you cut it. - They are kept in the project file, out of the media panel. - **Cues** (C) mark moments on the ruler, such as "VO starts" or "music lifts". +- **Bible** (the second page of Story view): the characters, places and + things the movie keeps coming back to, and its look. Each has pictures + (drop them on the card; the first is its face) and the words that must + stay the same in every shot. +- **Cast**: each beat card shows who and what is in it as faces. Click **+ + Cast** to choose from the bible. +- **Takes**: footage tried for a beat, generated or shot. Each remembers how + it was made (model, prompt, references). Click a take on the beat card to + lay it in. A take still being generated pulses, and the story lane says so. +- **Brief** copies the chosen beat's brief, a prompt to start from. It + carries the beat's story, framing, cast, look, length and shape. + `splice ctl story brief` also writes out the references and the frames + around the beat, for an assistant generating footage. Seedance through + Higgsfield works well; see the skill. - **Shot list** copies the plan as text. - Keys: B adds a beat, C marks a cue. diff --git a/crates/splice-app/src/ctl.rs b/crates/splice-app/src/ctl.rs index ff7b375..9108144 100644 --- a/crates/splice-app/src/ctl.rs +++ b/crates/splice-app/src/ctl.rs @@ -54,6 +54,8 @@ Titles, colours and looks --key green|blue|#rrggbb|none [--similarity F] [--blend F] [--despill/--no-despill] --mask ellipse|rect|none [--mask-x F --mask-y F --mask-width F --mask-height F] [--feather F] [--invert/--no-invert] (mask numbers are fractions of the clip's frame) + --crop-left F --crop-right F --crop-top F --crop-bottom F | --crop none + (cut edges away, fractions of the picture; it keeps its size and what remains stays put) C is #rrggbb, #rrggbbaa or a name: white, black, paper, ink, vermilion, gold, green, blue, clear. Music (a score: instruments and notes, played by General MIDI sounds) @@ -129,9 +131,32 @@ Story (planning: beats end to end along the timeline, and cues) story shot add --beat B [--image FILE] [--note TEXT] a reference shot: what it should look like (a picture, a frame of a video) story shot set --shot S --note TEXT | story shot move --shot S --index I | story shot remove --shot S + story cast --beat B [--add ENTRY] [--remove ENTRY] + who and what from the bible is in a beat + story brief --beat B [--dir DIR] + everything to make a beat's footage: what happens, what to capture, its + cast's words and pictures, the look, length and shape, the frames just + before and after (written to DIR), and an `outline` prompt to start from + story take pending --beat B [--tool T] [--model M] [--mode X] [--prompt P] [--refs a,b] [--job J] + a take being made (generated): shows on the beat while it runs + story take land --take T --video FILE [--seam] [--keep] + story take add --beat B --video FILE [origin as above] [--seam] [--keep] + footage for a take; it is laid into the beat unless --keep (and a beat + with nothing laid in takes it anyway). --seam skips an extension's + first three frames, which repeat what it extends. The reply's `take` + says if it was chosen, and if it is short of or longer than the beat + story take fail --take T a generation that failed: its pending take goes + story take choose --take T | story take remove --take T cue add [--at T] --label TEXT | cue set --cue C [--at T] [--label TEXT] | cue remove --cue C view edit|story switch the window between Edit view and the Story board +The bible (who and what the movie keeps coming back to, and its look) + bible read entries: kind, name, words, pictures, the beats each is in + bible add --kind character|place|thing|look --name N [--words W] [--image FILE] + bible set --entry E [--name N] [--words W] + bible picture --entry E --image FILE | --remove I (the first picture is the face briefs lead with) + bible remove --entry E E is an entry's id or name + Recording the screen record sources the displays and windows that can be recorded record start [--display N | --window ID|APP | --region X,Y,W,H] [--mic] [--clicks] [--for S] @@ -155,7 +180,8 @@ Playing and the window select [--clip C] select clips (none: deselect) command run a command from the command menu (zoom fit, versions) save [--label TEXT] keep a version - export [--draft|--high] [--wait] + export [--draft|--high] [--from T --to T | --clip C | --beat B] [--wait] + the movie, or just a stretch of it (a clip's span, a beat) say --text TEXT show one line in Splice's status bar Talking with the person @@ -399,11 +425,12 @@ fn build(args: &[String]) -> Result { let mut next = |name: &str| rest.next().ok_or(format!("{name} needs a value")); match arg.as_str() { "--font" | "--color" | "--box" | "--align" | "--enter" | "--exit" | "--drift" | "--key" | "--mask" - | "--query" => { + | "--query" | "--crop" => { request.insert(arg[2..].into(), json!(value(next(arg)?)?)); } "--size" | "--x" | "--y" | "--scale" | "--opacity" | "--similarity" | "--blend" | "--feather" - | "--mask-x" | "--mask-y" | "--mask-width" | "--mask-height" | "--enter-for" | "--exit-for" => { + | "--mask-x" | "--mask-y" | "--mask-width" | "--mask-height" | "--enter-for" | "--exit-for" + | "--crop-left" | "--crop-top" | "--crop-right" | "--crop-bottom" => { let raw = next(arg)?; let v = match raw.parse::() { Ok(v) => json!(v), @@ -419,17 +446,22 @@ fn build(args: &[String]) -> Result { "--no-bold" | "--no-italic" | "--no-shadow" | "--no-invert" | "--no-despill" => { request.insert(arg[5..].into(), json!(false)); } - "--at" | "--to" | "--start" | "--end" | "--duration" => { + "--at" | "--to" | "--from" | "--start" | "--end" | "--duration" => { let raw = next(arg)?; request.insert(arg[2..].into(), time_value(arg, raw)?); } - "--track" | "--fps" | "--version" | "--beat" | "--cue" | "--index" | "--display" | "--after" | "--shot" => { + "--track" | "--fps" | "--version" | "--beat" | "--cue" | "--index" | "--display" | "--after" | "--shot" + | "--take" => { let n: u64 = next(arg)?.parse().map_err(|_| format!("{arg} expects a number"))?; request.insert(arg[2..].into(), json!(n)); } - "--image" => { + "--image" | "--video" | "--dir" => { let file = next(arg)?; - request.insert("image".into(), json!(absolute(file))); + request.insert(arg[2..].into(), json!(absolute(file))); + } + "--entry" => { + let e = next(arg)?; + request.insert("entry".into(), e.parse::().map_or(json!(e), |id| json!(id))); } "--window" => { let w = next(arg)?; @@ -459,8 +491,9 @@ fn build(args: &[String]) -> Result { let m = next(arg)?; request.insert(arg[2..].into(), m.parse::().map_or(json!(m), |id| json!(id))); } - "--text" | "--label" | "--name" | "--new" | "--as" | "--aspect" | "--layer" | "--into" | "--remove" - | "--json" | "--add" | "--title" | "--story" | "--capture" | "--region" | "--note" => { + "--text" | "--label" | "--name" | "--new" | "--as" | "--kind" | "--words" | "--prompt" | "--model" + | "--mode" | "--tool" | "--refs" | "--job" | "--aspect" | "--layer" | "--into" | "--remove" | "--json" + | "--add" | "--title" | "--story" | "--capture" | "--region" | "--note" => { request.insert(arg[2..].into(), json!(value(next(arg)?)?)); } "--file" => { @@ -484,7 +517,7 @@ fn build(args: &[String]) -> Result { let v: f64 = next(arg)?.parse().map_err(|_| format!("{arg} expects points"))?; request.insert(arg[2..].into(), json!(v)); } - "--ripple" | "--wait" | "--keep" | "--none" | "--all" | "--reset" | "--clear" => { + "--ripple" | "--wait" | "--keep" | "--none" | "--all" | "--reset" | "--clear" | "--seam" => { request.insert(arg[2..].into(), json!(true)); } "--draft" | "--high" => { @@ -574,6 +607,10 @@ fn build(args: &[String]) -> Result { "symbol" } "placeholder" => "placeholder", + "bible" => { + request.insert("sub".into(), json!(words.first().map_or("read", |w| w.as_str()))); + "bible" + } "hold" => "hold", "ui" => "ui", "story" | "cue" | "record" => { @@ -585,7 +622,7 @@ fn build(args: &[String]) -> Result { |w| w.as_str(), ); request.insert("sub".into(), json!(sub)); - if let Some(action) = words.get(1).filter(|_| sub == "shot") { + if let Some(action) = words.get(1).filter(|_| sub == "shot" || sub == "take") { request.insert("action".into(), json!(action)); } command.as_str() @@ -756,6 +793,20 @@ mod tests { ); let r = build(&args("symbol fit --clip 4 --as canvas")).unwrap(); assert_eq!(r, json!({"cmd": "symbol", "sub": "fit", "clips": [4], "as": "canvas"})); + let r = build(&args("bible add --kind character --name Ada --words silver")).unwrap(); + assert_eq!( + r, + json!({"cmd": "bible", "sub": "add", "kind": "character", "name": "Ada", "words": "silver"}) + ); + let r = build(&args("story take pending --beat 3 --model seedance_2_5 --job j1")).unwrap(); + assert_eq!( + r, + json!({"cmd": "story", "sub": "take", "action": "pending", "beat": 3, "model": "seedance_2_5", "job": "j1"}) + ); + let r = build(&args("export out.mp4 --from 2 --to 5")).unwrap(); + assert_eq!((r["from"].clone(), r["to"].clone()), (json!(2.0), json!(5.0))); + let r = build(&args("look --clip 2 --crop-left 0.25")).unwrap(); + assert_eq!(r["crop_left"], json!(0.25)); let r = build(&args("symbol update --all")).unwrap(); assert_eq!(r, json!({"cmd": "symbol", "sub": "update", "all": true})); } diff --git a/crates/splice-app/src/editor.rs b/crates/splice-app/src/editor.rs index 88eff76..a9d3964 100644 --- a/crates/splice-app/src/editor.rs +++ b/crates/splice-app/src/editor.rs @@ -33,6 +33,7 @@ mod agent; mod input; mod inspector; mod made; +mod plan; mod record; mod render; mod reuse; @@ -135,6 +136,11 @@ enum Overlay { microphone: bool, clicks: bool, }, + /// Who and what from the bible is in a beat. + Cast { + beat: splice_core::story::BeatId, + index: usize, + }, /// Naming a symbol: a new one, a renamed or copied one, or a use's copy. SymbolName { naming: reuse::Naming, @@ -245,6 +251,12 @@ pub struct Editor { shown_shots: HashMap, /// Takes to lay into a beat once they are imported. fill_after_import: HashMap, + /// Footage being imported for a take: what it becomes when it lands. + take_after_import: HashMap, + /// Which page of the Story view shows. + page: plan::Page, + /// The picture on show on each bible card. + shown_pictures: HashMap, /// The score or scene editor, when one is open. workbench: Option, _ctl: Task<()>, @@ -380,6 +392,9 @@ impl Editor { story_focus: cx.focus_handle(), recording: None, fill_after_import: HashMap::new(), + take_after_import: HashMap::new(), + page: plan::Page::default(), + shown_pictures: HashMap::new(), shown_shots: HashMap::new(), _ctl: ctl, _autosave: autosave, @@ -599,6 +614,7 @@ impl Editor { let mut errors = Vec::new(); let mut added: Vec<(Media, PathBuf)> = Vec::new(); let mut placed = Vec::new(); + let mut takes: Vec = Vec::new(); self.doc.edit(|project| { let mut at = place; for result in probed { @@ -622,7 +638,16 @@ impl Editor { content: None, }); let media = project.media(id).cloned().expect("just added"); - if let Some(beat) = self.fill_after_import.remove(&path) { + if let Some(landing) = self.take_after_import.remove(&path) { + // The take is made (or lands) in the same step as + // its import: one undo, and nothing if it fails. + if let Some(report) = landing.land(project, id) { + if let Some(clip) = report["clip"].as_u64() { + placed.push(ClipId(clip)); + } + takes.push(report); + } + } else if let Some(beat) = self.fill_after_import.remove(&path) { if let Some(clip) = project.fill_beat(beat, id) { placed.push(clip); } @@ -653,13 +678,16 @@ impl Editor { .map(|(m, _)| serde_json::json!({ "id": m.id.0, "name": m.name })) .collect(); let clips: Vec<_> = placed.iter().map(|c| c.0).collect(); - let answer = serde_json::json!({ + let mut answer = serde_json::json!({ "ok": errors.is_empty(), "media": media, "clips": clips, "errors": errors, "revision": self.doc.revision, }); + if let Some(take) = takes.first() { + answer["take"] = take.clone(); + } let _ = reply.send(answer.to_string()); } if !placed.is_empty() { @@ -1340,7 +1368,7 @@ impl Editor { path.set_extension("mp4"); } this.update(cx, |editor, cx| { - if let Err(e) = editor.start_export(path, quality, true, cx) { + if let Err(e) = editor.start_export(path, quality, true, None, cx) { editor.say_error(e); cx.notify(); } @@ -1357,12 +1385,18 @@ impl Editor { output: PathBuf, quality: Quality, reveal: bool, + range: Option<(Time, Time)>, cx: &mut Context, ) -> Result<(), String> { if let Some(waiting) = self.unrendered_in_use() { return Err(format!("Cannot export yet: {waiting}")); } - let plan = match export::plan(&self.doc.project, &self.resolver(), &output, quality) { + // Just a stretch: cut from the movie as drawn, so it matches. + let planned = match range { + Some((a, b)) => export::plan_range(&self.doc.project, &self.resolver(), &output, quality, a, b), + None => export::plan(&self.doc.project, &self.resolver(), &output, quality), + }; + let plan = match planned { Ok(plan) => plan, Err(e) => return Err(format!("Cannot export: {e}")), }; @@ -1512,6 +1546,9 @@ impl Editor { let key = event.keystroke.key.as_str(); let typed = event.keystroke.key_char.clone(); let command = event.keystroke.modifiers.platform || event.keystroke.modifiers.control; + // The cast sheet's rows, counted before the overlay is borrowed. + // Its entries, then a new character, place and thing. + let castable = self.castable().len() + 2; match &mut self.overlay { Overlay::None => return, Overlay::Palette { query, index } => match key { @@ -1661,6 +1698,16 @@ impl Editor { } _ => {} }, + Overlay::Cast { beat, index } => match key { + "escape" => return self.close_overlay(window, cx), + "up" => *index = index.saturating_sub(1), + "down" => *index = (*index + 1).min(castable), + "enter" | "space" => { + let (beat, index) = (*beat, *index); + return self.cast_pick(beat, index, window, cx); + } + _ => {} + }, Overlay::SymbolMenu { name, index } => match key { "escape" => return self.close_overlay(window, cx), "up" => *index = index.saturating_sub(1), diff --git a/crates/splice-app/src/editor/agent.rs b/crates/splice-app/src/editor/agent.rs index 01d802e..ee406d9 100644 --- a/crates/splice-app/src/editor/agent.rs +++ b/crates/splice-app/src/editor/agent.rs @@ -104,6 +104,7 @@ fn look_json(look: &splice_core::look::Look) -> Value { "despill": k.despill, })), "mask": look.mask, + "crop": look.crop, }) } @@ -153,6 +154,10 @@ impl Editor { let answer = match cmd { "score" | "scene" | "open" => return self.ctl_made(cmd, request, reply, window, cx), "story" if request["sub"] == json!("shot") => return self.ctl_shot(request, reply, cx), + "story" if matches!(request["sub"].as_str(), Some("cast" | "brief" | "take")) => { + return self.ctl_plan(request, reply, cx); + } + "bible" => return self.ctl_bible(request, reply, cx), "story" | "cue" | "view" => self.ctl_story(cmd, request, cx), "record" => return self.ctl_record(request, reply, cx), "symbol" | "placeholder" | "hold" => return self.ctl_symbol(cmd, request, reply, cx), @@ -593,7 +598,28 @@ impl Editor { Some("high") => Quality::High, _ => Quality::Standard, }; - if let Err(e) = self.start_export(path.clone(), quality, false, cx) { + // Just a stretch: --from/--to, a clip's span, or a beat. + let range = if let Ok(Some(clips)) = self.clips_of(request).map(|c| c.first().copied()) { + self.doc.project.clip(clips).map(|c| (c.start, c.end())) + } else if let Some(beat) = request.get("beat").and_then(Value::as_u64) { + match self.doc.project.story.span(splice_core::story::BeatId(beat)) { + Some(span) => Some(span), + None => return Some(fail(format!("no beat {beat}"))), + } + } else { + match (self.time(request, "from"), self.time(request, "to")) { + (Ok(None), Ok(None)) => None, + (Ok(a), Ok(b)) => { + let (a, b) = (a.unwrap_or(0), b.unwrap_or(self.doc.project.duration())); + if b <= a { + return Some(fail("--to must be after --from")); + } + Some((a, b)) + } + (Err(e), _) | (_, Err(e)) => return Some(fail(e)), + } + }; + if let Err(e) = self.start_export(path.clone(), quality, false, range, cx) { return Some(fail(e)); } if flag("wait") == Some(true) { @@ -989,5 +1015,18 @@ fn apply_look(look: &mut splice_core::look::Look, request: &Value) -> Result<(), mask.feather = number("feather").map_or(mask.feather, |v| v.clamp(0.001, 1.0)); mask.invert = flag("invert").unwrap_or(mask.invert); } + if text("crop") == Some("none") { + look.crop = None; + } + let edges = ["crop_left", "crop_top", "crop_right", "crop_bottom"]; + if edges.iter().any(|k| number(k).is_some()) { + let mut c = look.crop.unwrap_or_default(); + c.left = number("crop_left").unwrap_or(c.left); + c.top = number("crop_top").unwrap_or(c.top); + c.right = number("crop_right").unwrap_or(c.right); + c.bottom = number("crop_bottom").unwrap_or(c.bottom); + let c = c.clamped(); + look.crop = (!c.is_none()).then_some(c); + } Ok(()) } diff --git a/crates/splice-app/src/editor/inspector.rs b/crates/splice-app/src/editor/inspector.rs index a581735..30e59ac 100644 --- a/crates/splice-app/src/editor/inspector.rs +++ b/crates/splice-app/src/editor/inspector.rs @@ -27,6 +27,10 @@ pub enum Field { MaskWidth, MaskHeight, MaskFeather, + CropLeft, + CropRight, + CropTop, + CropBottom, EnterDuration, ExitDuration, } @@ -44,6 +48,7 @@ impl Field { Field::MaskX | Field::MaskY => (0.0, 1.0), Field::MaskWidth | Field::MaskHeight => (0.05, 1.5), Field::MaskFeather => (0.001, 1.0), + Field::CropLeft | Field::CropRight | Field::CropTop | Field::CropBottom => (0.0, 0.95), Field::EnterDuration | Field::ExitDuration => (0.1, 3.0), } } @@ -59,6 +64,7 @@ impl Field { Field::MaskX | Field::MaskY => 0.5, Field::MaskWidth | Field::MaskHeight => 0.8, Field::MaskFeather => Mask::default().feather, + Field::CropLeft | Field::CropRight | Field::CropTop | Field::CropBottom => 0.0, Field::EnterDuration | Field::ExitDuration => 0.5, } } @@ -78,6 +84,10 @@ impl Field { Field::MaskWidth => "Width", Field::MaskHeight => "Height", Field::MaskFeather => "Feather", + Field::CropLeft => "Left", + Field::CropRight => "Right", + Field::CropTop => "Top", + Field::CropBottom => "Bottom", Field::EnterDuration | Field::ExitDuration => "Length", } } @@ -124,6 +134,10 @@ impl Field { Field::MaskWidth => mask.width, Field::MaskHeight => mask.height, Field::MaskFeather => mask.feather, + Field::CropLeft => look.crop.map_or(0.0, |c| c.left), + Field::CropRight => look.crop.map_or(0.0, |c| c.right), + Field::CropTop => look.crop.map_or(0.0, |c| c.top), + Field::CropBottom => look.crop.map_or(0.0, |c| c.bottom), Field::EnterDuration => secs(look.enter.duration) as f32, Field::ExitDuration => secs(look.exit.duration) as f32, } @@ -163,6 +177,17 @@ impl Field { } } } + Field::CropLeft | Field::CropRight | Field::CropTop | Field::CropBottom => { + let mut c = look.crop.unwrap_or_default(); + match self { + Field::CropLeft => c.left = v, + Field::CropRight => c.right = v, + Field::CropTop => c.top = v, + _ => c.bottom = v, + } + let c = c.clamped(); + look.crop = (!c.is_none()).then_some(c); + } Field::EnterDuration => look.enter.duration = splice_core::time::from_secs(v as f64), Field::ExitDuration => look.exit.duration = splice_core::time::from_secs(v as f64), } @@ -765,6 +790,7 @@ impl Editor { sections.push(self.place_section(id, &clip, cx).into_any_element()); sections.push(self.motion_section(id, &clip, cx).into_any_element()); if media_clip { + sections.push(self.crop_section(id, &clip, cx).into_any_element()); sections.push(self.key_section(id, &clip, cx).into_any_element()); } sections.push(self.mask_section(id, &clip, cx).into_any_element()); @@ -1260,6 +1286,41 @@ impl Editor { }) } + /// Cutting edges off a picture: four sliders, and a way back. + fn crop_section(&mut self, id: ClipId, clip: &Clip, cx: &mut Context) -> impl IntoElement + use<> { + let crop = clip.look.crop.filter(|c| !c.is_none()); + let summary = match crop { + None => "Off".to_string(), + Some(c) => { + let (w, h) = (1.0 - c.left - c.right, 1.0 - c.top - c.bottom); + format!("{:.0}% × {:.0}% left", w * 100.0, h * 100.0) + } + }; + let (section, open) = self.fold(id, "Crop", crop.is_some(), summary, cx); + section.when(open, |d| { + d.child(self.slider(id, clip, Field::CropLeft, cx)) + .child(self.slider(id, clip, Field::CropRight, cx)) + .child(self.slider(id, clip, Field::CropTop, cx)) + .child(self.slider(id, clip, Field::CropBottom, cx)) + .child( + div() + .text_size(px(10.5)) + .line_height(px(15.0)) + .text_color(Theme::overlay()) + .child("The picture keeps its size; what remains stays where it was."), + ) + .when(crop.is_some(), |d| { + d.child( + div() + .flex() + .child(self.chip("crop-reset", "Reset crop", false, cx, move |this, cx| { + this.change(id, cx, |c| c.look.crop = None); + })), + ) + }) + }) + } + fn mask_section(&mut self, id: ClipId, clip: &Clip, cx: &mut Context) -> impl IntoElement + use<> { let mask = clip.look.mask; let shapes: Vec = [ diff --git a/crates/splice-app/src/editor/plan.rs b/crates/splice-app/src/editor/plan.rs new file mode 100644 index 0000000..5cc1fe0 --- /dev/null +++ b/crates/splice-app/src/editor/plan.rs @@ -0,0 +1,1640 @@ +//! Planning beyond the beats: the bible (characters, places, things and the +//! look, each with pictures and continuity words), who is cast in each beat, +//! the takes tried for a beat (generated or shot) with where each came from, +//! and briefs that gather all of it for whoever makes a beat's footage. +//! +//! The bible is a page of the Story view, built like the board: cards with +//! pictures you drop in, words you type in place. Cast shows on each beat +//! card as faces; takes as a strip, the one laid in ringed, a pending one +//! breathing until its footage arrives. + +use super::render::{caps, icon, panel, tip}; +use super::story::Target; +use super::*; +use gpui::{Animation, AnimationExt, FontWeight, ObjectFit, img, pulsating_between}; +use splice_core::bible::{Entry, EntryId, Kind}; +use splice_core::story::{Beat, BeatId, Origin, Take, TakeId}; + +/// Which page of the Story view shows. +#[derive(Clone, Copy, Debug, Default, PartialEq, Eq)] +pub enum Page { + #[default] + Beats, + Bible, +} + +/// How many frames at the start of a generated extension repeat what it +/// extends: skipped when such a take is laid in. +const SEAM_FRAMES: i64 = 3; + +/// A pending take this old has probably been lost. +const STALE: u64 = 30 * 60; + +/// Seconds since 1970, now. +pub fn now() -> u64 { + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .map_or(0, |d| d.as_secs()) +} + +/// What footage being imported becomes: a new take of a beat, or a pending +/// take landing. +#[derive(Clone, Debug)] +pub enum Landing { + New { + beat: BeatId, + origin: Origin, + skip: Time, + choose: bool, + }, + Land { + take: TakeId, + origin: Origin, + skip: Time, + choose: bool, + }, +} + +impl Landing { + /// Makes or lands the take with `media`, inside an import's edit. + /// Returns what happened, for the reply. + pub fn land(self, project: &mut Project, media: MediaId) -> Option { + let (take, choose) = match self { + Landing::New { + beat, + origin, + skip, + choose, + } => { + let id = project.add_take(beat, Some(media), origin, now())?; + project.set_take(id, |t| t.skip = skip); + // A beat with nothing laid in takes its first take. + let choose = choose || project.chosen_take(beat).is_none(); + (id, choose) + } + Landing::Land { + take, + origin, + skip, + choose, + } => { + let beat = project.take_beat(take)?; + project.set_take(take, |t| { + t.media = Some(media); + t.skip = skip.max(t.skip); + // What the landing adds to what was known. + let o = &mut t.origin; + for (field, value) in [ + (&mut o.tool, origin.tool), + (&mut o.model, origin.model), + (&mut o.mode, origin.mode), + (&mut o.prompt, origin.prompt), + (&mut o.job, origin.job), + (&mut o.note, origin.note), + ] { + if !value.is_empty() { + *field = value; + } + } + if !origin.references.is_empty() { + o.references = origin.references; + } + }); + (take, choose || project.chosen_take(beat).is_none()) + } + }; + let beat = project.take_beat(take)?; + let clip = if choose { project.choose_take(take) } else { None }; + let (start, end) = project.story.span(beat)?; + let room = end - start; + let length = project.media(media).map_or(0, |m| m.duration); + let skip = project.take(take).map_or(0, |t| t.skip); + let usable = (length - skip).max(0); + Some(serde_json::json!({ + "take": take.0, + "beat": beat.0, + "clip": clip.map(|c| c.0), + "chosen": clip.is_some(), + // A take shorter than its beat leaves a gap; a longer one is cut. + "short_by": (usable < room).then(|| secs(room - usable)), + "trimmed_by": (usable > room).then(|| secs(usable - room)), + })) + } +} + +fn rgb(color: u32) -> gpui::Hsla { + gpui::rgb(color).into() +} + +/// An entry's initials, for a face without a picture. +fn initials(name: &str) -> String { + let letters: String = name + .split_whitespace() + .filter_map(|w| w.chars().next()) + .take(2) + .collect(); + if letters.is_empty() { + "?".into() + } else { + letters.to_uppercase() + } +} + +impl Editor { + // The page switch. + + pub(super) fn page_switch(&self, cx: &mut Context) -> AnyElement { + let count = self.doc.project.bible.entries.len(); + let tab = |id: &'static str, label: String, page: Page, on: bool| { + div() + .id(id) + .h(px(24.0)) + .px(px(10.0)) + .rounded(px(6.0)) + .flex() + .items_center() + .cursor_pointer() + .text_size(px(11.5)) + .font_weight(FontWeight::SEMIBOLD) + .text_color(if on { Theme::text() } else { Theme::overlay() }) + .when(on, |d| d.bg(Theme::surface0())) + .hover(|s| s.text_color(Theme::text())) + .child(label) + .on_click(cx.listener(move |this, _, _, cx| { + this.end_typing(cx); + this.page = page; + cx.notify(); + })) + }; + div() + .p(px(2.0)) + .rounded(px(8.0)) + .bg(Theme::crust()) + .flex() + .gap(px(2.0)) + .child(tab("page-beats", "Beats".into(), Page::Beats, self.page == Page::Beats)) + .child(tab( + "page-bible", + if count == 0 { + "Bible".into() + } else { + format!("Bible · {count}") + }, + Page::Bible, + self.page == Page::Bible, + )) + .into_any_element() + } + + // The bible. + + /// A new entry of `kind`, named and ready to type. + pub(super) fn new_entry(&mut self, kind: Kind, window: &mut Window, cx: &mut Context) { + let n = self.doc.project.bible.of(kind).count() + 1; + let name = match kind { + Kind::Character => format!("Character {n}"), + Kind::Place => format!("Place {n}"), + Kind::Thing => format!("Thing {n}"), + Kind::Look => "The look".to_string(), + }; + let id = self.edit(cx, |p| p.add_entry(kind, &name)); + self.page = Page::Bible; + self.begin_typing(Target::EntryName(id), true, window, cx); + } + + pub(super) fn render_bible(&mut self, cx: &mut Context) -> AnyElement { + let bible = self.doc.project.bible.clone(); + let empty = bible.is_empty(); + let mut sections: Vec = Vec::new(); + // The look first: it is said once, before every shot. + for kind in [Kind::Look, Kind::Character, Kind::Place, Kind::Thing] { + let entries: Vec = bible.of(kind).cloned().collect(); + let cards: Vec = entries.iter().map(|e| self.entry_card(e, cx)).collect(); + // A full card to start a kind; once it has some, a small tile. + let first = entries.is_empty(); + let add = div() + .id(SharedString::from(format!("add-{}", kind.name()))) + .w(px(if first && kind != Kind::Look { 200.0 } else { 180.0 })) + .h(px(if first && kind != Kind::Look { 260.0 } else { 40.0 })) + .rounded(px(10.0)) + .border_1() + .border_dashed() + .border_color(Theme::surface1()) + .flex() + .when(first && kind != Kind::Look, |d| d.flex_col()) + .items_center() + .justify_center() + .gap(px(8.0)) + .cursor_pointer() + .hover(|s| s.border_color(rgb(kind.color())).bg(Theme::lane())) + .child(icon("plus", Theme::subtext()).size(px(if first { 16.0 } else { 12.0 }))) + .child( + div() + .text_size(px(12.0)) + .text_color(Theme::subtext()) + .child(match kind { + Kind::Character => "Add a character", + Kind::Place => "Add a place", + Kind::Thing => "Add a thing", + Kind::Look => "Describe the look", + }), + ) + .on_click(cx.listener(move |this, _, window, cx| this.new_entry(kind, window, cx))); + // The look is one thing for the whole movie. + let show_add = kind != Kind::Look || entries.is_empty(); + sections.push( + div() + .flex() + .flex_col() + .gap(px(10.0)) + .child( + div() + .flex() + .items_center() + .gap(px(8.0)) + .child(div().size(px(8.0)).rounded_full().bg(rgb(kind.color()))) + .child(caps(kind.plural())) + .child( + div() + .text_size(px(10.5)) + .text_color(Theme::overlay()) + .child(match kind { + Kind::Character => "who appears: their face and what never changes", + Kind::Place => "where it happens", + Kind::Thing => "props and products that must look the same", + Kind::Look => "light, colour, lens, grain: said once, for every shot", + }), + ), + ) + .child( + div() + .flex() + .flex_wrap() + .gap(px(14.0)) + .children(cards) + .when(show_add, |d| d.child(add)), + ) + .into_any_element(), + ); + } + panel() + .id("bible") + .flex_1() + .min_w_0() + .h_full() + .flex() + .flex_col() + .child( + div() + .h(px(52.0)) + .flex_none() + .px(px(16.0)) + .flex() + .items_center() + .gap(px(10.0)) + .border_b_1() + .border_color(Theme::edge()) + .child(self.page_switch(cx)) + .child( + div() + .flex_1() + .min_w_0() + .text_size(px(10.5)) + .text_color(Theme::overlay()) + .truncate() + .child(if empty { + "Who and what the movie keeps coming back to, so every shot matches" + } else { + "Cast them into beats from each beat card; briefs carry their pictures and words" + }), + ), + ) + .child( + div() + .id("bible-cards") + .flex_1() + .min_h_0() + .overflow_y_scroll() + .p(px(16.0)) + .flex() + .flex_col() + .gap(px(22.0)) + .children(sections), + ) + .into_any_element() + } + + fn entry_card(&mut self, entry: &Entry, cx: &mut Context) -> AnyElement { + let id = entry.id; + let kind = entry.kind; + let color = kind.color(); + let pictures: Vec = entry.pictures.iter().filter_map(|s| self.still_file(s)).collect(); + let shown = self + .shown_pictures + .get(&id) + .copied() + .filter(|i| *i < pictures.len()) + .unwrap_or(0); + let beats = self.doc.project.cast_in(id).len(); + let face = pictures.get(shown).cloned(); + // A face wants room for a head; a place, the width of a view. The + // look spans the page, its picture beside its words. + let wide = kind == Kind::Look; + let picture = div() + .relative() + .when(wide, |d| d.w(px(220.0)).flex_none()) + .when(!wide, |d| d.w_full()) + .h(px(match kind { + Kind::Character => 180.0, + Kind::Look => 124.0, + _ => 132.0, + })) + .rounded(px(7.0)) + .overflow_hidden() + .bg(Theme::crust()) + .map(|d| match face { + Some(path) => d.child(img(path).size_full().object_fit(ObjectFit::Cover)), + None => d + .flex() + .flex_col() + .items_center() + .justify_center() + .gap(px(6.0)) + .child( + div() + .size(px(44.0)) + .rounded_full() + .bg(rgb(color)) + .flex() + .items_center() + .justify_center() + .text_size(px(16.0)) + .font_weight(FontWeight::BOLD) + .text_color(Theme::ground()) + .map(|d| { + if kind == Kind::Look { + d.child(icon("swatch", Theme::ground()).size(px(20.0))) + } else { + d.child(initials(&entry.name)) + } + }), + ) + .child( + div() + .text_size(px(10.5)) + .text_color(Theme::overlay()) + .child("Drop pictures here"), + ), + }); + // The pictures as a strip, the one on show outlined, with ways to add. + let thumbs: Vec = pictures + .iter() + .enumerate() + .map(|(i, path)| { + let group: SharedString = format!("pic-{}-{i}", id.0).into(); + div() + .id(SharedString::from(format!("pic-{}-{i}", id.0))) + .group(group.clone()) + .relative() + .w(px(40.0)) + .h(px(28.0)) + .flex_none() + .rounded(px(4.0)) + .overflow_hidden() + .border_1() + .border_color(if i == shown { Theme::text() } else { Theme::edge() }) + .cursor_pointer() + .child(img(path.clone()).size_full().object_fit(ObjectFit::Cover)) + .child( + div() + .id(SharedString::from(format!("pic-remove-{}-{i}", id.0))) + .absolute() + .top(px(1.0)) + .right(px(1.0)) + .size(px(13.0)) + .rounded(px(3.0)) + .bg(Theme::scrim()) + .flex() + .items_center() + .justify_center() + .invisible() + .group_hover(group, |s| s.visible()) + .child(icon("close", Theme::text()).size(px(8.0))) + .on_click(cx.listener(move |this, _, _, cx| { + cx.stop_propagation(); + this.typing = None; + this.shown_pictures.remove(&id); + this.edit(cx, |p| { + p.set_entry(id, |e| { + if i < e.pictures.len() { + e.pictures.remove(i); + } + }) + }); + })), + ) + .tooltip(tip( + if i == 0 { + "The face (first picture)" + } else { + "Click to show; drag to the first place with ⌘-click" + }, + None, + )) + .on_click(cx.listener(move |this, e: &ClickEvent, _, cx| { + // ⌘-click makes it the first: the face briefs lead with. + if e.modifiers().platform && i > 0 { + this.edit(cx, |p| { + p.set_entry(id, |e| { + let pic = e.pictures.remove(i); + e.pictures.insert(0, pic); + }) + }); + this.shown_pictures.insert(id, 0); + } else { + this.shown_pictures.insert(id, i); + } + cx.notify(); + })) + .into_any_element() + }) + .collect(); + let tool = |key: String, glyph: &'static str, hint: &'static str| { + div() + .id(SharedString::from(key)) + .w(px(28.0)) + .h(px(28.0)) + .flex_none() + .rounded(px(4.0)) + .border_1() + .border_dashed() + .border_color(Theme::surface1()) + .flex() + .items_center() + .justify_center() + .cursor_pointer() + .hover(|s| s.border_color(Theme::accent()).bg(Theme::surface0())) + .tooltip(tip(hint, None)) + .child(icon(glyph, Theme::subtext()).size(px(11.0))) + }; + let strip = div() + .flex() + .flex_wrap() + .gap(px(4.0)) + .children(thumbs) + .child( + tool( + format!("pic-add-{}", id.0), + "plus", + "Add pictures (or drop them on the card)", + ) + .on_click(cx.listener(move |this, _, _, cx| this.pick_entry_pictures(id, cx))), + ) + .child( + tool(format!("pic-grab-{}", id.0), "film", "Use the viewer's frame") + .on_click(cx.listener(move |this, _, _, cx| this.grab_entry_picture(id, cx))), + ); + let hint = kind.hint(); + // What would make its briefs weak, said where it can be fixed. + let missing: Vec<&str> = [ + (pictures.is_empty(), "No picture yet: briefs have nothing to show"), + (entry.words.trim().is_empty(), "No words yet: say what never changes"), + ] + .iter() + .filter(|(m, _)| *m) + .map(|(_, t)| *t) + .collect(); + let warnings = div() + .flex() + .flex_col() + .gap(px(2.0)) + .children(missing.into_iter().map(|t| { + div() + .flex() + .items_center() + .gap(px(5.0)) + .child(div().size(px(5.0)).flex_none().rounded_full().bg(Theme::accent())) + .child(div().text_size(px(10.0)).text_color(Theme::overlay()).child(t)) + })); + let footer = div() + .flex() + .items_center() + .justify_between() + .child( + div() + .flex() + .items_center() + .gap(px(5.0)) + .child(div().size(px(6.0)).rounded_full().bg(rgb(color))) + .child( + div() + .text_size(px(10.0)) + .text_color(Theme::overlay()) + .child(match (kind, beats) { + (Kind::Look, _) => "every beat".to_string(), + (_, 0) => "in no beat yet".to_string(), + (_, 1) => "in 1 beat".to_string(), + (_, n) => format!("in {n} beats"), + }), + ), + ) + .child( + render::icon_button(SharedString::from(format!("entry-remove-{}", id.0)), "trash", true) + .tooltip(tip("Remove from the bible (and from every beat)", None)) + .on_click(cx.listener(move |this, _, _, cx| { + this.typing = None; + this.edit(cx, |p| p.remove_entry(id)); + })), + ); + let words = div() + .flex() + .flex_col() + .gap(px(8.0)) + .when(wide, |d| d.flex_1().min_w_0()) + .child( + div() + .font_weight(FontWeight::SEMIBOLD) + .child(self.field(Target::EntryName(id), "Name", 13.0, cx)), + ) + .child(self.field(Target::EntryWords(id), hint, 11.5, cx)) + .child(warnings); + let card = div() + .id(("entry", id.0 as usize)) + .when(wide, |d| d.w_full().max_w(px(640.0)).flex_row()) + .when(!wide, |d| d.w(px(200.0)).min_h(px(300.0)).flex_col()) + .p(px(10.0)) + .rounded(px(10.0)) + .bg(Theme::lane()) + .border_1() + .border_color(Theme::edge()) + .flex() + .gap(px(if wide { 14.0 } else { 8.0 })) + .drag_over::(move |s, _, _, _| s.border_color(rgb(color)).bg(Theme::surface0())) + .on_drop(cx.listener(move |this, paths: &ExternalPaths, _, cx| { + cx.stop_propagation(); + this.add_entry_pictures(id, paths.paths().to_vec(), None, cx); + })); + if wide { + card.child(div().flex().flex_col().gap(px(8.0)).child(picture).child(strip)) + .child(words.child(div().flex_1()).child(footer)) + .into_any_element() + } else { + card.child(picture) + .child(strip) + .child(words) + .child(div().flex_1()) + .child(footer) + .into_any_element() + } + } + + /// Adds pictures from files to a bible entry. + pub(super) fn add_entry_pictures( + &mut self, + entry: EntryId, + paths: Vec, + reply: Option>, + cx: &mut Context, + ) { + if paths.is_empty() { + return; + } + self.working += 1; + cx.spawn(async move |this, cx| { + let made = cx + .background_spawn(async move { paths.iter().map(|p| super::shots::make_still(p)).collect::>() }) + .await; + this.update(cx, |editor, cx| { + editor.working -= 1; + let mut added = 0; + let mut errors = Vec::new(); + for result in made { + match result { + Ok(still) => { + editor.embed_still(&still, super::shots::still_path(&still)); + editor.edit(cx, |p| p.set_entry(entry, |e| e.pictures.push(still))); + added += 1; + } + Err(e) => errors.push(e.to_string()), + } + } + if let Some(n) = editor.doc.project.bible.entry(entry).map(|e| e.pictures.len()) + && added > 0 + { + editor.shown_pictures.insert(entry, n - 1); + } + match errors.first() { + Some(e) => editor.say_error(e.clone()), + None => editor.say(format!("Added {added} picture{}", if added == 1 { "" } else { "s" })), + } + if let Some(reply) = reply { + let _ = reply.send( + serde_json::json!({ "ok": errors.is_empty(), "added": added, "errors": errors }).to_string(), + ); + } + cx.notify(); + }) + .ok(); + }) + .detach(); + } + + fn pick_entry_pictures(&mut self, entry: EntryId, cx: &mut Context) { + let paths = cx.prompt_for_paths(PathPromptOptions { + files: true, + directories: false, + multiple: true, + prompt: Some("Add pictures".into()), + }); + cx.spawn(async move |this, cx| { + if let Ok(Ok(Some(paths))) = paths.await { + this.update(cx, |editor, cx| editor.add_entry_pictures(entry, paths, None, cx)) + .ok(); + } + }) + .detach(); + } + + /// The viewer's picture at the playhead becomes one of an entry's pictures. + fn grab_entry_picture(&mut self, entry: EntryId, cx: &mut Context) { + let at = self.player.position; + let path = dirs() + .cache + .join(format!("grab-{}-{}.png", std::process::id(), entry.0)); + let planned = export::plan_still(&self.doc.project, &self.resolver(), at, &path); + let plan = match planned { + Ok(plan) => plan, + Err(e) => return self.say_error(format!("Could not grab the frame: {e}")), + }; + cx.spawn(async move |this, cx| { + let made = cx + .background_spawn(async move { + let out = ffmpeg::export_command()?.args(&plan.args).output()?; + anyhow::ensure!(out.status.success(), "ffmpeg could not draw the frame"); + Ok::<_, anyhow::Error>(path) + }) + .await; + this.update(cx, |editor, cx| match made { + Ok(path) => editor.add_entry_pictures(entry, vec![path], None, cx), + Err(e) => editor.say_error(format!("Could not grab the frame: {e}")), + }) + .ok(); + }) + .detach(); + } + + // Cast, on a beat card. + + /// The faces of who and what is in a beat, and a way to change it. One + /// face carries its name; past five, the rest are counted. + pub(super) fn cast_row(&mut self, beat: &Beat, cx: &mut Context) -> AnyElement { + const SHOWN: usize = 5; + let id = beat.id; + let entries: Vec = beat + .cast + .iter() + .filter_map(|e| self.doc.project.bible.entry(*e).cloned()) + .collect(); + let lone = (entries.len() == 1).then(|| entries[0].name.clone()); + let more = entries.len().saturating_sub(SHOWN); + let faces: Vec = entries + .iter() + .take(SHOWN) + .map(|entry| self.face(entry, 22.0, format!("face-{}-{}", id.0, entry.id.0))) + .collect(); + let has_cast = !faces.is_empty(); + div() + .id(SharedString::from(format!("cast-{}", id.0))) + .flex() + .items_center() + .gap(px(3.0)) + .min_h(px(24.0)) + .rounded(px(6.0)) + .cursor_pointer() + .children(faces) + .when_some(lone, |d, name| { + d.child( + div() + .pl(px(3.0)) + .max_w(px(120.0)) + .truncate() + .text_size(px(11.0)) + .text_color(Theme::subtext()) + .child(name), + ) + }) + .when(more > 0, |d| { + d.child( + div() + .h(px(22.0)) + .px(px(6.0)) + .rounded(px(11.0)) + .bg(Theme::surface0()) + .flex() + .items_center() + .text_size(px(10.0)) + .text_color(Theme::subtext()) + .child(format!("+{more}")), + ) + }) + .child( + div() + .h(px(22.0)) + .px(px(if has_cast { 5.0 } else { 7.0 })) + .rounded(px(11.0)) + .border_1() + .border_dashed() + .border_color(Theme::surface1()) + .flex() + .items_center() + .gap(px(4.0)) + .hover(|s| s.border_color(Theme::accent())) + .child(icon("plus", Theme::overlay()).size(px(10.0))) + .when(!has_cast, |d| { + d.child(div().text_size(px(10.5)).text_color(Theme::overlay()).child("Cast")) + }), + ) + .on_click(cx.listener(move |this, _, window, cx| { + cx.stop_propagation(); + this.end_typing(cx); + this.beat = Some(id); + this.overlay = Overlay::Cast { beat: id, index: 0 }; + window.focus(&this.palette_focus); + cx.notify(); + })) + .into_any_element() + } + + /// An entry's face: its first picture in a ring of its kind's colour, + /// or its initials (the look: a swatch). + fn face(&self, entry: &Entry, size: f32, key: String) -> AnyElement { + let face = entry.pictures.first().and_then(|s| self.still_file(s)); + let words = if entry.words.trim().is_empty() { + entry.name.clone() + } else { + format!("{}: {}", entry.name, entry.words.trim()) + }; + div() + .id(SharedString::from(key)) + .size(px(size)) + .flex_none() + .rounded_full() + .overflow_hidden() + .border_2() + .border_color(rgb(entry.kind.color())) + .bg(Theme::crust()) + .flex() + .items_center() + .justify_center() + .map(|d| match face { + Some(path) => d.child(img(path).size_full().object_fit(ObjectFit::Cover)), + None if entry.kind == Kind::Look => d.child(icon("swatch", Theme::text()).size(px(size * 0.5))), + None => d + .text_size(px(size * 0.38)) + .font_weight(FontWeight::BOLD) + .text_color(Theme::text()) + .child(initials(&entry.name)), + }) + .tooltip(tip(words, None)) + .into_any_element() + } + + /// The bible's castable entries, in order: characters, places, things. + pub(super) fn castable(&self) -> Vec { + [Kind::Character, Kind::Place, Kind::Thing] + .iter() + .flat_map(|k| self.doc.project.bible.of(*k).cloned().collect::>()) + .collect() + } + + /// Row `index` of the cast sheet: an entry to toggle in or out, or one + /// of the last three: a new character, place or thing in this beat. + pub(super) fn cast_pick(&mut self, beat: BeatId, index: usize, window: &mut Window, cx: &mut Context) { + let entries = self.castable(); + match entries.get(index) { + Some(e) => { + let entry = e.id; + self.edit(cx, |p| p.toggle_cast(beat, entry)); + } + None => { + let kind = [Kind::Character, Kind::Place, Kind::Thing][(index - entries.len()).min(2)]; + self.close_overlay(window, cx); + let n = self.doc.project.bible.of(kind).count() + 1; + let name = match kind { + Kind::Character => format!("Character {n}"), + Kind::Place => format!("Place {n}"), + _ => format!("Thing {n}"), + }; + let entry = self.edit(cx, |p| { + let id = p.add_entry(kind, &name); + p.toggle_cast(beat, id); + id + }); + self.page = Page::Bible; + self.begin_typing(Target::EntryName(entry), true, window, cx); + } + } + cx.notify(); + } + + pub(super) fn cast_overlay(&self, beat: BeatId, index: usize, cx: &mut Context) -> AnyElement { + let entries = self.castable(); + let found = self.doc.project.story.beats.iter().find(|b| b.id == beat); + let cast = found.map(|b| b.cast.clone()).unwrap_or_default(); + let title = found.map_or_else(String::new, |b| b.title.clone()); + let mut rows: Vec = Vec::new(); + let mut kind = None; + for (i, e) in entries.iter().enumerate() { + // A small heading where each kind begins. + if kind != Some(e.kind) { + kind = Some(e.kind); + rows.push( + div() + .px(px(10.0)) + .pt(px(if i == 0 { 2.0 } else { 8.0 })) + .pb(px(2.0)) + .child(caps(e.kind.plural())) + .into_any_element(), + ); + } + let on = cast.contains(&e.id); + rows.push( + div() + .id(("cast-row", i)) + .h(px(36.0)) + .px(px(10.0)) + .rounded(px(7.0)) + .flex() + .items_center() + .gap(px(10.0)) + .cursor_pointer() + .when(i == index, |d| d.bg(Theme::lane())) + .hover(|s| s.bg(Theme::lane())) + .child(self.face(e, 24.0, format!("cast-face-{}", e.id.0))) + .child( + div() + .flex_1() + .min_w_0() + .flex() + .flex_col() + .child( + div() + .text_size(px(12.5)) + .text_color(Theme::text()) + .child(e.name.clone()), + ) + .when(!e.words.trim().is_empty(), |d| { + d.child( + div() + .truncate() + .text_size(px(10.5)) + .text_color(Theme::overlay()) + .child(e.words.trim().to_string()), + ) + }), + ) + .child( + div() + .size(px(16.0)) + .flex_none() + .flex() + .items_center() + .justify_center() + .when(on, |d| d.child(icon("check", Theme::accent()).size(px(15.0)))), + ) + .on_click(cx.listener(move |this, _, window, cx| this.cast_pick(beat, i, window, cx))) + .into_any_element(), + ); + } + let base = entries.len(); + for (n, (label, glyph)) in [ + ("New character in this beat", "plus"), + ("New place", "plus"), + ("New thing", "plus"), + ] + .into_iter() + .enumerate() + { + let at = base + n; + rows.push( + div() + .id(("cast-new", n)) + .h(px(30.0)) + .px(px(10.0)) + .rounded(px(7.0)) + .flex() + .items_center() + .gap(px(10.0)) + .cursor_pointer() + .when(index == at, |d| d.bg(Theme::lane())) + .hover(|s| s.bg(Theme::lane())) + .child(icon(glyph, Theme::overlay()).size(px(12.0))) + .child(div().text_size(px(12.0)).text_color(Theme::subtext()).child(label)) + .on_click(cx.listener(move |this, _, window, cx| this.cast_pick(beat, at, window, cx))) + .into_any_element(), + ); + } + div() + .flex() + .flex_col() + .child( + div() + .h(px(44.0)) + .px(px(16.0)) + .flex() + .items_center() + .border_b_1() + .border_color(Theme::rule()) + .child( + div() + .truncate() + .text_size(px(14.0)) + .text_color(Theme::text()) + .child(format!("In “{title}”")), + ), + ) + .when(entries.is_empty(), |d| { + d.child( + div() + .px(px(16.0)) + .pt(px(12.0)) + .text_size(px(11.5)) + .line_height(px(16.0)) + .text_color(Theme::overlay()) + .child("The bible is empty. Start with who is in this beat; give them pictures and words on the Bible page."), + ) + }) + .child( + div() + .id("cast-rows") + .max_h(px(380.0)) + .overflow_y_scroll() + .p(px(6.0)) + .flex() + .flex_col() + .children(rows), + ) + .child( + div() + .h(px(30.0)) + .px(px(14.0)) + .flex() + .items_center() + .border_t_1() + .border_color(Theme::rule()) + .text_size(px(10.5)) + .text_color(Theme::overlay()) + .child("↑↓ move · ⏎ in or out · esc done"), + ) + .into_any_element() + } + + // Takes, on a beat card. + + /// A beat's takes: each a picture with what made it beneath, the one + /// laid in ringed, a pending one breathing with how long it has run. + pub(super) fn takes_strip(&mut self, beat: &Beat, cx: &mut Context) -> Option { + if beat.takes.is_empty() { + return None; + } + let chosen = self.doc.project.chosen_take(beat.id); + let now = now(); + let tiles: Vec = beat + .takes + .iter() + .enumerate() + .map(|(i, take)| { + let tid = take.id; + let on = chosen == Some(tid); + let thumb = take + .media + .and_then(|m| self.doc.project.media(m)) + .and_then(|m| self.library.get(&m.hash)) + .and_then(|a| a.thumb.clone()); + let pending = take.media.is_none(); + let waited = (take.started > 0).then(|| now.saturating_sub(take.started)); + let stale = pending && waited.is_some_and(|w| w > STALE); + let group: SharedString = format!("take-{}", tid.0).into(); + let origin = &take.origin; + // Beneath: the model, shortened; or how long it has run. + let caption = if pending { + match waited { + Some(_) if stale => "lost?".to_string(), + Some(w) => format!("{}:{:02}", w / 60, w % 60), + None => "making".to_string(), + } + } else if !origin.model.is_empty() { + origin.model.replace("seedance_", "S").replace('_', ".") + } else { + format!("Take {}", i + 1) + }; + let about = { + let mut lines = vec![format!( + "Take {}{}", + i + 1, + if stale { + " · no footage after 30 minutes: land it, or remove it" + } else if pending { + " · being made" + } else if on { + " · laid in" + } else { + " · click to lay it in" + } + )]; + let made = [origin.tool.as_str(), origin.model.as_str(), origin.mode.as_str()] + .iter() + .filter(|s| !s.is_empty()) + .copied() + .collect::>() + .join(" · "); + if !made.is_empty() { + lines.push(made); + } + if !origin.prompt.is_empty() { + let p: String = origin.prompt.chars().take(200).collect(); + lines.push(if origin.prompt.chars().count() > 200 { + format!("{p}…") + } else { + p + }); + } + lines.join("\n") + }; + let picture = div() + .id(("take", tid.0 as usize)) + .group(group.clone()) + .relative() + .w(px(64.0)) + .h(px(40.0)) + .rounded(px(5.0)) + .overflow_hidden() + .bg(if pending { Theme::surface0() } else { Theme::crust() }) + .border_2() + .border_color(if on { + Theme::accent() + } else if pending { + Theme::surface2() + } else { + Theme::edge() + }) + .cursor_pointer() + .map(|d| match thumb { + Some(image) => d.child(img(image).size_full().object_fit(ObjectFit::Cover)), + None if pending => d + .flex() + .items_center() + .justify_center() + .child(icon(if stale { "close" } else { "film" }, Theme::subtext()).size(px(13.0))), + None => d + .flex() + .items_center() + .justify_center() + .text_size(px(10.0)) + .text_color(Theme::overlay()) + .child((i + 1).to_string()), + }) + .child( + div() + .id(("take-remove", tid.0 as usize)) + .absolute() + .top(px(2.0)) + .right(px(2.0)) + .size(px(14.0)) + .rounded(px(3.0)) + .bg(Theme::scrim()) + .flex() + .items_center() + .justify_center() + .invisible() + .group_hover(group, |s| s.visible()) + .child(icon("close", Theme::text()).size(px(8.0))) + .on_click(cx.listener(move |this, _, _, cx| { + cx.stop_propagation(); + this.edit(cx, |p| p.remove_take(tid)); + })), + ) + .tooltip(tip(about, None)) + .on_click(cx.listener(move |this, _, _, cx| { + cx.stop_propagation(); + if let Some(clip) = this.edit(cx, |p| p.choose_take(tid)) { + this.selection = vec![clip]; + } + cx.notify(); + })); + // Breathing while it is made (until it looks lost). + let picture = if pending && !stale { + picture + .with_animation( + ("take-pulse", tid.0 as usize), + Animation::new(Duration::from_millis(1600)) + .repeat() + .with_easing(pulsating_between(0.55, 1.0)), + |d, delta| d.opacity(delta), + ) + .into_any_element() + } else { + picture.into_any_element() + }; + div() + .flex() + .flex_col() + .gap(px(2.0)) + .child(picture) + .child( + div() + .w(px(64.0)) + .truncate() + .text_size(px(9.5)) + .font_family(MONO_FONT) + .text_color(if on { Theme::text() } else { Theme::overlay() }) + .child(caption), + ) + .into_any_element() + }) + .collect(); + Some( + div() + .flex() + .flex_col() + .gap(px(4.0)) + .child( + div() + .flex() + .items_center() + .justify_between() + .child(caps("Takes")) + .child( + div() + .text_size(px(10.0)) + .text_color(Theme::overlay()) + .child("click one to lay it in"), + ), + ) + .child(div().flex().flex_wrap().gap(px(5.0)).children(tiles)) + .into_any_element(), + ) + } + + pub(super) fn take_json(&self, take: &Take) -> serde_json::Value { + serde_json::json!({ + "take": take.id.0, + "media": take.media.map(|m| m.0), + "pending": take.media.is_none(), + "started": take.started, + "stale": take.media.is_none() && take.started > 0 && now().saturating_sub(take.started) > STALE, + "skip": secs(take.skip), + "origin": take.origin, + }) + } + + // Briefs. + + /// Writes a beat's brief to `dir` off the main thread (its reference + /// pictures, numbered as the outline names them, and the frames just + /// before, at the start, at the end and just after), then calls `done` + /// with the brief as JSON, files in place. + pub(super) fn write_brief( + &mut self, + beat: BeatId, + dir: PathBuf, + cx: &mut Context, + done: impl FnOnce(&mut Editor, Result, &mut Context) + 'static, + ) { + use serde_json::json; + let Some(brief) = self.doc.project.brief(beat) else { + return done(self, Err("no such beat".into()), cx); + }; + let Some((start, end)) = self.doc.project.story.span(beat) else { + return done(self, Err("no such beat".into()), cx); + }; + // What to copy: from the cache (or the project file) to the folder. + let mut copies: Vec<(Option, PathBuf)> = Vec::new(); + for p in &brief.images { + let n = p.tag.trim_start_matches("@Image"); + let to = dir.join(format!("{n:0>2}-{}.{}", slug(&p.of), p.still.ext)); + copies.push((self.still_file(&p.still), to)); + } + for (i, p) in brief.more_pictures.iter().enumerate() { + let to = dir.join(format!("more-{:02}-{}.{}", i + 1, slug(&p.of), p.still.ext)); + copies.push((self.still_file(&p.still), to)); + } + // The frames around the beat, planned here and drawn in the background. + let frame = self.doc.project.frame(); + let length = self.doc.project.duration(); + let mut stills = Vec::new(); + for (key, t) in [ + ("before", start - frame), + ("first", start), + ("last", end - frame), + ("after", end), + ] { + if t < 0 || t >= length || self.doc.project.visual_stack(t).is_empty() { + continue; + } + let file = dir.join(format!("frame-{key}.png")); + if let Ok(plan) = export::plan_still(&self.doc.project, &self.resolver(), t, &file) { + stills.push((key, plan, file)); + } + } + let mut out = match serde_json::to_value(&brief) { + Ok(v) => v, + Err(e) => return done(self, Err(e.to_string()), cx), + }; + self.working += 1; + cx.spawn(async move |this, cx| { + let written = cx + .background_spawn(async move { + // A fresh folder: nothing left from an earlier brief. + let _ = std::fs::remove_dir_all(&dir); + std::fs::create_dir_all(&dir).map_err(|e| e.to_string())?; + let copied: Vec> = copies + .into_iter() + .map(|(from, to)| from.and_then(|f| std::fs::copy(&f, &to).ok().map(|_| to))) + .collect(); + let mut frames = json!({}); + for (key, plan, file) in stills { + let drawn = ffmpeg::export_command() + .ok() + .and_then(|mut c| c.args(&plan.args).output().ok()) + .is_some_and(|o| o.status.success()); + if drawn { + frames[key] = json!(file); + } + } + Ok::<_, String>((copied, frames, dir)) + }) + .await; + this.update(cx, |editor, cx| { + editor.working -= 1; + let result = written.map(|(copied, frames, dir)| { + let images = out["images"].as_array().map_or(0, Vec::len); + for (i, file) in copied.into_iter().enumerate() { + let slot = if i < images { + &mut out["images"][i] + } else { + &mut out["more_pictures"][i - images] + }; + // Missing (not yet unpacked): said, never renumbered. + slot["file"] = json!(file); + } + out["frames"] = frames; + out["dir"] = json!(dir); + out + }); + done(editor, result, cx); + cx.notify(); + }) + .ok(); + }) + .detach(); + } + + /// Where a beat's brief is written unless told otherwise. + pub(super) fn brief_dir(beat: BeatId) -> PathBuf { + dirs().cache.join("briefs").join(format!("beat-{}", beat.0)) + } + + // ctl. + + /// `splice ctl bible …`. + pub(super) fn ctl_bible( + &mut self, + request: &serde_json::Value, + reply: std::sync::mpsc::Sender, + cx: &mut Context, + ) -> Option { + use crate::ctl::fail; + use serde_json::{Value, json}; + let text = |key: &str| request.get(key).and_then(Value::as_str).map(str::to_string); + let entry_json = |e: &Entry, this: &Self| { + json!({ + "entry": e.id.0, + "kind": e.kind, + "name": e.name, + "words": e.words, + "pictures": e.pictures.iter().filter_map(|s| this.still_file(s)).collect::>(), + "beats": this.doc.project.cast_in(e.id).iter().map(|b| b.0).collect::>(), + }) + }; + let find = |this: &Self| -> Result { + let key = request + .get("entry") + .map(|v| v.as_str().map_or_else(|| v.to_string(), str::to_string)) + .ok_or("name the entry with --entry ID|NAME (`bible read` lists them)")?; + this.doc + .project + .bible + .find(&key) + .map(|e| e.id) + .ok_or(format!("the bible has no entry {key}")) + }; + match request.get("sub").and_then(Value::as_str).unwrap_or("read") { + "read" => Some(json!({ + "ok": true, + "entries": self.doc.project.bible.entries.iter().map(|e| entry_json(e, self)).collect::>(), + })), + "add" => { + let Some(kind) = text("kind").as_deref().and_then(Kind::parse) else { + return Some(fail("give --kind character|place|thing|look")); + }; + let Some(name) = text("name").filter(|n| !n.trim().is_empty()) else { + return Some(fail("give --name NAME")); + }; + if self.doc.project.bible.find(name.trim()).is_some() { + return Some(fail(format!( + "the bible has {} already; names must differ (`bible set` changes it)", + name.trim() + ))); + } + if kind == Kind::Look && self.doc.project.bible.of(Kind::Look).next().is_some() { + return Some(fail("the bible has its look already: change it with `bible set`")); + } + let words = text("words").unwrap_or_default(); + let id = self.edit(cx, |p| { + let id = p.add_entry(kind, name.trim()); + p.set_entry(id, |e| e.words = words); + id + }); + if let Some(image) = text("image") { + self.add_entry_pictures(id, vec![PathBuf::from(image)], Some(reply), cx); + return None; + } + Some(json!({ "ok": true, "entry": id.0 })) + } + "set" => { + let id = match find(self) { + Ok(id) => id, + Err(e) => return Some(fail(e)), + }; + let (name, words) = (text("name"), text("words")); + if name.is_none() && words.is_none() { + return Some(fail("give --name and/or --words")); + } + if let Some(n) = &name + && self.doc.project.bible.find(n).is_some_and(|e| e.id != id) + { + return Some(fail(format!("the bible has {n} already; names must differ"))); + } + self.edit(cx, |p| { + p.set_entry(id, |e| { + if let Some(n) = name { + e.name = n; + } + if let Some(w) = words { + e.words = w; + } + }) + }); + Some(json!({ "ok": true })) + } + "picture" => { + let id = match find(self) { + Ok(id) => id, + Err(e) => return Some(fail(e)), + }; + if let Some(i) = text("remove").and_then(|r| r.trim().parse::().ok()) { + self.edit(cx, |p| { + p.set_entry(id, |e| { + if (i as usize) < e.pictures.len() { + e.pictures.remove(i as usize); + } + }) + }); + return Some(json!({ "ok": true })); + } + let Some(image) = text("image") else { + return Some(fail("give --image FILE to add, or --remove I (0 is the first picture)")); + }; + if !Path::new(&image).exists() { + return Some(fail(format!("{image} does not exist"))); + } + self.add_entry_pictures(id, vec![PathBuf::from(image)], Some(reply), cx); + None + } + "remove" => match find(self) { + Ok(id) => { + self.typing = None; + self.edit(cx, |p| p.remove_entry(id)); + Some(json!({ "ok": true })) + } + Err(e) => Some(fail(e)), + }, + other => Some(fail(format!( + "unknown `bible {other}`: read, add, set, picture or remove" + ))), + } + } + + /// `splice ctl story cast|brief|take …`. + pub(super) fn ctl_plan( + &mut self, + request: &serde_json::Value, + reply: std::sync::mpsc::Sender, + cx: &mut Context, + ) -> Option { + use crate::ctl::fail; + use serde_json::{Value, json}; + let text = |key: &str| request.get(key).and_then(Value::as_str).map(str::to_string); + let flag = |key: &str| request.get(key).and_then(Value::as_bool) == Some(true); + let beat = |this: &Self| -> Result { + let id = request + .get("beat") + .and_then(Value::as_u64) + .map(BeatId) + .or(this.beat) + .ok_or("name the beat with --beat B")?; + this.doc + .project + .story + .index(id) + .map(|_| id) + .ok_or(format!("no beat {}", id.0)) + }; + let entry = |this: &Self, key: &str| -> Result { + this.doc + .project + .bible + .find(key) + .map(|e| e.id) + .ok_or(format!("the bible has no entry {key}")) + }; + let origin = || Origin { + tool: text("tool").unwrap_or_default(), + model: text("model").unwrap_or_default(), + mode: text("mode").unwrap_or_default(), + prompt: text("prompt").unwrap_or_default(), + references: text("refs") + .map(|r| { + r.split(',') + .map(|s| s.trim().to_string()) + .filter(|s| !s.is_empty()) + .collect() + }) + .unwrap_or_default(), + job: text("job").unwrap_or_default(), + note: text("note").unwrap_or_default(), + }; + match request.get("sub").and_then(Value::as_str).unwrap_or("") { + "cast" => { + let b = match beat(self) { + Ok(b) => b, + Err(e) => return Some(fail(e)), + }; + for (key, join) in [("add", true), ("remove", false)] { + if let Some(name) = text(key) { + let id = match entry(self, &name) { + Ok(id) => id, + Err(e) => return Some(fail(e)), + }; + let present = self + .doc + .project + .story + .beats + .iter() + .any(|x| x.id == b && x.cast.contains(&id)); + if present != join { + self.edit(cx, |p| p.toggle_cast(b, id)); + } + } + } + let cast: Vec = self + .doc + .project + .story + .beats + .iter() + .find(|x| x.id == b) + .map(|x| x.cast.clone()) + .unwrap_or_default() + .iter() + .filter_map(|e| self.doc.project.bible.entry(*e)) + .map(|e| json!({ "entry": e.id.0, "name": e.name })) + .collect(); + Some(json!({ "ok": true, "beat": b.0, "cast": cast })) + } + "brief" => { + let b = match beat(self) { + Ok(b) => b, + Err(e) => return Some(fail(e)), + }; + let dir = text("dir").map(PathBuf::from).unwrap_or_else(|| Self::brief_dir(b)); + self.write_brief(b, dir, cx, move |_, result, _| { + let answer = match result { + Ok(brief) => json!({ "ok": true, "brief": brief }), + Err(e) => fail(e), + }; + let _ = reply.send(answer.to_string()); + }); + None + } + "take" => { + let action = text("action").unwrap_or_default(); + let take = || request.get("take").and_then(Value::as_u64).map(TakeId); + match action.as_str() { + // A generation under way: a take with no footage yet. + "pending" => { + let b = match beat(self) { + Ok(b) => b, + Err(e) => return Some(fail(e)), + }; + let id = self.edit(cx, |p| p.add_take(b, None, origin(), now())); + Some(json!({ "ok": true, "take": id.map(|t| t.0), "beat": b.0 })) + } + // Footage for a take: a new take, or a pending one landing. + "add" | "land" => { + let Some(file) = text("video") else { + return Some(fail("give the footage with --video FILE")); + }; + let path = PathBuf::from(&file); + if !path.exists() { + return Some(fail(format!("{file} does not exist"))); + } + // Read it first: nothing is made for footage that cannot come in. + if let Err(e) = ffmpeg::probe(&path) { + return Some(fail(format!("{file} is not footage Splice can read: {e}"))); + } + let skip = if flag("seam") { + SEAM_FRAMES * self.doc.project.frame() + } else { + 0 + }; + let choose = !flag("keep"); + let landing = if action == "land" { + match take().filter(|t| self.doc.project.take(*t).is_some()) { + Some(take) => Landing::Land { + take, + origin: origin(), + skip, + choose, + }, + None => return Some(fail("name the pending take with --take T")), + } + } else { + match beat(self) { + Ok(beat) => Landing::New { + beat, + origin: origin(), + skip, + choose, + }, + Err(e) => return Some(fail(e)), + } + }; + self.take_after_import.insert(path.clone(), landing); + self.import_paths(vec![path], None, Some(reply), cx); + None + } + // A generation that failed or was abandoned. + "fail" => match take().filter(|t| self.doc.project.take(*t).is_some_and(|t| t.media.is_none())) { + Some(t) => { + self.edit(cx, |p| p.remove_take(t)); + Some(json!({ "ok": true })) + } + None => Some(fail("name a pending take with --take T")), + }, + "choose" => match take().filter(|t| self.doc.project.take(*t).is_some()) { + Some(t) => match self.edit(cx, |p| p.choose_take(t)) { + Some(clip) => { + self.selection = vec![clip]; + Some(self.edited_with(json!({ "clip": clip.0 }))) + } + None => Some(fail("that take has no footage yet")), + }, + None => Some(fail("name the take with --take T (`story read` lists them)")), + }, + "remove" => match take().filter(|t| self.doc.project.take(*t).is_some()) { + Some(t) => { + self.edit(cx, |p| p.remove_take(t)); + Some(json!({ "ok": true })) + } + None => Some(fail("name the take with --take T")), + }, + _ => Some(fail("story take pending | add | land | fail | choose | remove")), + } + } + other => Some(fail(format!("unknown `story {other}`"))), + } + } +} + +/// A file-name-safe form of a name. +fn slug(name: &str) -> String { + let s: String = name + .chars() + .map(|c| { + if c.is_alphanumeric() { + c.to_ascii_lowercase() + } else { + '-' + } + }) + .collect(); + let s = s.trim_matches('-').to_string(); + if s.is_empty() { + "picture".into() + } else { + s.chars().take(32).collect() + } +} diff --git a/crates/splice-app/src/editor/render.rs b/crates/splice-app/src/editor/render.rs index af6a290..bd9f89b 100644 --- a/crates/splice-app/src/editor/render.rs +++ b/crates/splice-app/src/editor/render.rs @@ -1658,6 +1658,10 @@ impl Editor { let content = match &self.overlay { Overlay::None => return None, Overlay::Record { .. } => panel().child(self.record_overlay(cx)), + Overlay::Cast { beat, index } => { + let (beat, index) = (*beat, *index); + panel().w(px(380.0)).child(self.cast_overlay(beat, index, cx)) + } Overlay::SymbolMenu { .. } | Overlay::SymbolName { .. } => { panel().child(self.symbol_overlay(cx).unwrap_or_else(|| div().into_any_element())) } diff --git a/crates/splice-app/src/editor/shots.rs b/crates/splice-app/src/editor/shots.rs index 256da9a..40ffa68 100644 --- a/crates/splice-app/src/editor/shots.rs +++ b/crates/splice-app/src/editor/shots.rs @@ -20,7 +20,7 @@ pub fn still_path(still: &Still) -> PathBuf { /// Turns an image (or a frame of a video) into a reference PNG kept in the /// cache by its hash. Blocking. -fn make_still(source: &Path) -> anyhow::Result { +pub(super) fn make_still(source: &Path) -> anyhow::Result { let probe = ffmpeg::probe(source)?; let dir = dirs().cache.join("media"); std::fs::create_dir_all(&dir)?; @@ -132,7 +132,7 @@ impl Editor { } /// Keeps a reference picture in the project file. - fn embed_still(&mut self, still: &Still, path: PathBuf) { + pub(super) fn embed_still(&mut self, still: &Still, path: PathBuf) { let store_path = self.doc.path().to_path_buf(); let (hash, ext) = (still.hash.clone(), still.ext.clone()); std::thread::spawn(move || { diff --git a/crates/splice-app/src/editor/stage.rs b/crates/splice-app/src/editor/stage.rs index 6f2ff13..5872588 100644 --- a/crates/splice-app/src/editor/stage.rs +++ b/crates/splice-app/src/editor/stage.rs @@ -132,10 +132,22 @@ impl Editor { (image, Some(size)) } }; + let natural = export::natural_size(source, rendered, canvas.0, canvas.1); + // A cropped picture is smaller, and what remains stays in place. + let (natural, pose) = match source { + Source::Media(_) => { + let (size, (dx, dy)) = clip.look.cropped(natural); + let mut pose = pose; + pose.x += dx * pose.scale / canvas.0.max(1) as f64; + pose.y += dy * pose.scale / canvas.1.max(1) as f64; + (size, pose) + } + Source::Generated(_) => (natural, pose), + }; layers.push(Layer { clip: clip.id, image, - natural: export::natural_size(source, rendered, canvas.0, canvas.1), + natural, pose, visible: { // A see-through render is picked and outlined by what it draws. diff --git a/crates/splice-app/src/editor/story.rs b/crates/splice-app/src/editor/story.rs index ae925d8..00531da 100644 --- a/crates/splice-app/src/editor/story.rs +++ b/crates/splice-app/src/editor/story.rs @@ -26,6 +26,9 @@ pub enum Target { Capture(BeatId), Cue(CueId), Shot(splice_core::story::ShotId), + /// A bible entry's name, and its continuity words. + EntryName(splice_core::bible::EntryId), + EntryWords(splice_core::bible::EntryId), } /// Typing into a beat or cue, and the project before the typing began. @@ -117,6 +120,32 @@ impl Editor { } } + /// The chosen beat's brief: its outline to the clipboard, and its + /// references written to a folder shown in Finder, ready to attach. + pub(super) fn copy_brief(&mut self, cx: &mut Context) { + let Some(beat) = self.beat.filter(|b| self.doc.project.story.index(*b).is_some()) else { + self.say("Choose a beat to brief"); + return cx.notify(); + }; + let dir = Self::brief_dir(beat); + self.write_brief(beat, dir, cx, |editor, result, cx| match result { + Ok(brief) => { + let outline = brief["outline"].as_str().unwrap_or_default().to_string(); + cx.write_to_clipboard(gpui::ClipboardItem::new_string(outline)); + if let Some(dir) = brief["dir"].as_str() { + cx.reveal_path(Path::new(dir)); + } + let n = brief["images"].as_array().map_or(0, Vec::len); + editor.say(format!( + "Brief for “{}”: the prompt is copied; its {n} reference{} are in the folder, numbered as @Image", + brief["title"].as_str().unwrap_or(""), + if n == 1 { "" } else { "s" } + )); + } + Err(e) => editor.say_error(format!("Could not brief it: {e}")), + }); + } + pub(super) fn copy_shot_list(&mut self, cx: &mut Context) { let list = self.doc.project.shot_list(); if list.trim().is_empty() { @@ -145,6 +174,8 @@ impl Editor { .find(|s| s.id == id)? .note .clone(), + Target::EntryName(id) => self.doc.project.bible.entry(id)?.name.clone(), + Target::EntryWords(id) => self.doc.project.bible.entry(id)?.words.clone(), }) } @@ -156,6 +187,8 @@ impl Editor { Target::Capture(id) => p.set_beat(id, |b| b.capture = text), Target::Cue(id) => p.set_cue(id, |c| c.label = text), Target::Shot(id) => p.set_shot(id, |s| s.note = text), + Target::EntryName(id) => p.set_entry(id, |e| e.name = text), + Target::EntryWords(id) => p.set_entry(id, |e| e.words = text), } } @@ -226,7 +259,10 @@ impl Editor { let trimmed = text[..at].trim_end(); trimmed.rfind(char::is_whitespace).map_or(0, |i| i + 1) }; - let one_line = matches!(target, Target::Title(_) | Target::Cue(_) | Target::Shot(_)); + let one_line = matches!( + target, + Target::Title(_) | Target::Cue(_) | Target::Shot(_) | Target::EntryName(_) + ); match k.key.as_str() { "escape" => { self.end_typing(cx); @@ -237,6 +273,7 @@ impl Editor { self.end_typing(cx); match target { Target::Title(id) => self.begin_typing(Target::Story(id), false, window, cx), + Target::EntryName(id) => self.begin_typing(Target::EntryWords(id), false, window, cx), _ => window.focus(&self.focus), } } @@ -245,6 +282,7 @@ impl Editor { let next = match target { Target::Title(id) => Some(Target::Story(id)), Target::Story(id) => Some(Target::Capture(id)), + Target::EntryName(id) => Some(Target::EntryWords(id)), _ => None, }; match next { @@ -303,7 +341,7 @@ impl Editor { } /// Words that can be typed into: shown with a caret while typing. - fn field( + pub(super) fn field( &mut self, target: Target, placeholder: &'static str, @@ -367,6 +405,9 @@ impl Editor { // The board. pub(super) fn render_board(&mut self, cx: &mut Context) -> AnyElement { + if self.page == super::plan::Page::Bible { + return self.render_bible(cx); + } let story = self.doc.project.story.clone(); let total = story.end(); let cards: Vec = story @@ -483,35 +524,32 @@ impl Editor { .gap(px(10.0)) .border_b_1() .border_color(Theme::edge()) - .child( - div() - .flex_1() - .min_w_0() - .flex() - .flex_col() - .child( - div() - .text_size(px(13.0)) - .font_weight(FontWeight::SEMIBOLD) - .text_color(Theme::text()) - .child("Story"), + .child(self.page_switch(cx)) + .child(div().flex_1().min_w_0().flex().flex_col().child( + div().text_size(px(10.5)).text_color(Theme::overlay()).child(if empty { + "Plan the movie: beats in order, what happens, what to capture".to_string() + } else { + format!( + "{} beat{} · {}", + story.beats.len(), + if story.beats.len() == 1 { "" } else { "s" }, + seconds(total) ) - .child(div().text_size(px(10.5)).text_color(Theme::overlay()).child(if empty { - "Plan the movie: beats in order, what happens, what to capture".to_string() - } else { - format!( - "{} beat{} · {}", - story.beats.len(), - if story.beats.len() == 1 { "" } else { "s" }, - seconds(total) - ) - })), - ) + }), + )) .child( header_button("story-cue", "flag", "Cue") .tooltip(tip("Mark a cue at the playhead", Some("C"))) .on_click(cx.listener(|this, _, window, cx| this.new_cue(window, cx))), ) + .child( + header_button("story-brief", "text", "Brief") + .tooltip(tip( + "The chosen beat's brief: copies a prompt to start from, and shows its references in Finder", + None, + )) + .on_click(cx.listener(|this, _, _, cx| this.copy_brief(cx))), + ) .child( header_button("story-copy", "copy", "Shot list") .tooltip(tip("Copy the story as a shot list", None)) @@ -762,7 +800,9 @@ impl Editor { .child(self.field(Target::Title(id), "Name this beat", 13.0, cx)), ), ) + .child(self.cast_row(beat, cx)) .child(picture) + .when_some(self.takes_strip(beat, cx), |d, takes| d.child(takes)) .child(self.shot_strip(beat, shown.as_ref().map(|s| s.id), cx)) .when_some(shown.as_ref().filter(|_| showing_reference), |d, shot| { d.child( @@ -1027,6 +1067,11 @@ impl Editor { }, "clips": project.beat_clips(beat.id).iter().map(|c| c.0).collect::>(), "shots": beat.shots.iter().map(|s| self.shot_json(s)).collect::>(), + "cast": beat.cast.iter().filter_map(|e| project.bible.entry(*e)) + .map(|e| serde_json::json!({ "entry": e.id.0, "name": e.name, "kind": e.kind })) + .collect::>(), + "takes": beat.takes.iter().map(|t| self.take_json(t)).collect::>(), + "chosen_take": project.chosen_take(beat.id).map(|t| t.0), }) } diff --git a/crates/splice-app/src/icons.rs b/crates/splice-app/src/icons.rs index f1a90f2..ca9eede 100644 --- a/crates/splice-app/src/icons.rs +++ b/crates/splice-app/src/icons.rs @@ -97,6 +97,7 @@ fn icon(name: &str) -> Option<&'static str> { } "flag" => r#""#, "record" => r#""#, + "check" => r#""#, "close" => r#""#, "minus" => r#""#, "chevron" => r#""#, diff --git a/crates/splice-app/src/timeline.rs b/crates/splice-app/src/timeline.rs index 086ab8c..98b396d 100644 --- a/crates/splice-app/src/timeline.rs +++ b/crates/splice-app/src/timeline.rs @@ -567,9 +567,15 @@ fn paint_story( outline(window, b, Theme::text(), 1.5, 4.0); } let label = format!( - "{} {}", + "{} {}{}", n + 1, - if beat.title.is_empty() { "Untitled" } else { &beat.title } + if beat.title.is_empty() { "Untitled" } else { &beat.title }, + // A take still being made shows here as well as on its card. + if beat.takes.iter().any(|t| t.media.is_none()) { + " · generating" + } else { + "" + } ); let room = x1 - x0 - 12.0; if room > 14.0 { diff --git a/crates/splice-core/src/bible.rs b/crates/splice-core/src/bible.rs new file mode 100644 index 0000000..e46aa13 --- /dev/null +++ b/crates/splice-core/src/bible.rs @@ -0,0 +1,222 @@ +//! The bible: who and what the movie keeps coming back to (its characters, +//! places, things, and its look), each with pictures and the words that must +//! stay the same from shot to shot. Beats name their cast from it, so a brief +//! for any beat (to shoot it, or to generate it) carries the same anchors. + +use crate::model::Project; +use crate::story::{BeatId, Still}; +use serde::{Deserialize, Serialize}; + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] +#[serde(transparent)] +pub struct EntryId(pub u64); + +/// What an entry is. +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, Serialize, Deserialize)] +#[serde(rename_all = "lowercase")] +pub enum Kind { + Character, + Place, + Thing, + /// The movie's look: light, colour, lens, grain, mood. It applies to + /// every beat, so it is never cast. + Look, +} + +impl Kind { + pub const ALL: [Kind; 4] = [Kind::Character, Kind::Place, Kind::Thing, Kind::Look]; + + pub fn name(self) -> &'static str { + match self { + Kind::Character => "character", + Kind::Place => "place", + Kind::Thing => "thing", + Kind::Look => "look", + } + } + + pub fn plural(self) -> &'static str { + match self { + Kind::Character => "Characters", + Kind::Place => "Places", + Kind::Thing => "Things", + Kind::Look => "Look", + } + } + + pub fn parse(text: &str) -> Option { + match text.trim().to_lowercase().trim_end_matches('s') { + "character" | "person" | "people" => Some(Kind::Character), + "place" | "location" => Some(Kind::Place), + "thing" | "prop" | "object" => Some(Kind::Thing), + "look" | "style" => Some(Kind::Look), + _ => None, + } + } + + /// 0xRRGGBB, shared with the story's palette. + pub fn color(self) -> u32 { + match self { + Kind::Character => 0xef6a55, + Kind::Place => 0x86cfc0, + Kind::Thing => 0xf2c14e, + Kind::Look => 0xc792ea, + } + } + + /// What its words are for, shown while they are empty. + pub fn hint(self) -> &'static str { + match self { + Kind::Character => "What never changes: face, hair, build, clothes", + Kind::Place => "What never changes: the space, the light, the details", + Kind::Thing => "What never changes: shape, colour, material, marks", + Kind::Look => "Light, colour, lens, grain, mood", + } + } +} + +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +pub struct Entry { + pub id: EntryId, + pub kind: Kind, + pub name: String, + /// The continuity words: what must read the same in every shot. + #[serde(default)] + pub words: String, + /// Reference pictures, the first being its face. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub pictures: Vec, +} + +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +pub struct Bible { + #[serde(default)] + pub entries: Vec, +} + +impl Bible { + pub fn is_empty(&self) -> bool { + self.entries.is_empty() + } + + pub fn entry(&self, id: EntryId) -> Option<&Entry> { + self.entries.iter().find(|e| e.id == id) + } + + pub fn of(&self, kind: Kind) -> impl Iterator { + self.entries.iter().filter(move |e| e.kind == kind) + } + + /// The entry `key` names: its id, or its name in any case. + pub fn find(&self, key: &str) -> Option<&Entry> { + let key = key.trim(); + if let Ok(n) = key.parse::() + && let Some(e) = self.entry(EntryId(n)) + { + return Some(e); + } + let lower = key.to_lowercase(); + self.entries.iter().find(|e| e.name.to_lowercase() == lower) + } +} + +impl Project { + /// Adds an entry to the bible. Returns its id. + pub fn add_entry(&mut self, kind: Kind, name: &str) -> EntryId { + let id = EntryId(self.alloc()); + self.bible.entries.push(Entry { + id, + kind, + name: name.to_string(), + words: String::new(), + pictures: Vec::new(), + }); + id + } + + pub fn set_entry(&mut self, id: EntryId, f: impl FnOnce(&mut Entry)) { + if let Some(entry) = self.bible.entries.iter_mut().find(|e| e.id == id) { + f(entry); + } + } + + /// Takes an entry out of the bible and out of every beat's cast. + pub fn remove_entry(&mut self, id: EntryId) { + self.bible.entries.retain(|e| e.id != id); + for beat in &mut self.story.beats { + beat.cast.retain(|c| *c != id); + } + } + + /// Puts an entry at position `to` among those of its kind. + pub fn move_entry(&mut self, id: EntryId, to: usize) { + let Some(from) = self.bible.entries.iter().position(|e| e.id == id) else { + return; + }; + let entry = self.bible.entries.remove(from); + let kind = entry.kind; + let at = self + .bible + .entries + .iter() + .enumerate() + .filter(|(_, e)| e.kind == kind) + .map(|(i, _)| i) + .nth(to) + .unwrap_or(self.bible.entries.len()); + self.bible.entries.insert(at, entry); + } + + /// Whether `entry` is in `beat`'s cast; it joins or leaves. + pub fn toggle_cast(&mut self, beat: BeatId, entry: EntryId) { + if self.bible.entry(entry).is_none_or(|e| e.kind == Kind::Look) { + return; + } + self.set_beat(beat, |b| { + if let Some(i) = b.cast.iter().position(|c| *c == entry) { + b.cast.remove(i); + } else { + b.cast.push(entry); + } + }); + } + + /// The beats an entry is cast in. + pub fn cast_in(&self, entry: EntryId) -> Vec { + self.story + .beats + .iter() + .filter(|b| b.cast.contains(&entry)) + .map(|b| b.id) + .collect() + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::time::SECOND; + + #[test] + fn entries_are_cast_into_beats_and_leave_cleanly() { + let mut p = Project::default(); + let ada = p.add_entry(Kind::Character, "Ada"); + let look = p.add_entry(Kind::Look, "Warm dusk"); + let beat = p.add_beat(None, "Open", 4 * SECOND); + p.toggle_cast(beat, ada); + p.toggle_cast(beat, look); + assert_eq!(p.story.beats[0].cast, vec![ada], "the look is never cast"); + assert_eq!(p.cast_in(ada), vec![beat]); + assert_eq!(p.bible.find("ada").map(|e| e.id), Some(ada)); + p.remove_entry(ada); + assert!(p.story.beats[0].cast.is_empty()); + } + + #[test] + fn kinds_read_loosely() { + assert_eq!(Kind::parse("People"), Some(Kind::Character)); + assert_eq!(Kind::parse("locations"), Some(Kind::Place)); + assert_eq!(Kind::parse("style"), Some(Kind::Look)); + assert_eq!(Kind::parse("weather"), None); + } +} diff --git a/crates/splice-core/src/brief.rs b/crates/splice-core/src/brief.rs new file mode 100644 index 0000000..d911c36 --- /dev/null +++ b/crates/splice-core/src/brief.rs @@ -0,0 +1,364 @@ +//! Briefs: everything known about one beat, gathered for whoever makes its +//! footage (a crew, or a video model): what happens, what to capture, who and +//! what is in it with their continuity words and pictures, the movie's look, +//! its shape and length, and what comes just before and after. The outline +//! follows how video models are best prompted: references first, each with +//! one job; the look once; the subject anchored in words; one action and one +//! camera move per shot; what must not change. + +use crate::bible::{Entry, Kind}; +use crate::model::{Project, Settings}; +use crate::story::{BeatId, Still}; +use crate::time::secs; +use serde::Serialize; + +#[derive(Clone, Debug, Serialize)] +pub struct Brief { + pub beat: u64, + pub number: usize, + pub title: String, + pub story: String, + pub capture: String, + /// Where it starts on the timeline, and how long it runs (seconds). + pub start: f64, + pub duration: f64, + /// How long to ask a video model for: the beat's length, rounded up into + /// what models make (4 to 30 seconds). Trim the result to `duration`. + pub generate_seconds: u64, + /// The canvas: what the footage should be. + pub width: u32, + pub height: u32, + pub fps: u32, + pub aspect: String, + pub cast: Vec, + pub look: Vec, + /// The beat's reference shots, in order. + pub shots: Vec, + /// The references to send, at most five, each with one job, numbered + /// as the outline names them (@Image1…): the look, faces, places, + /// things, then shots. + pub images: Vec, + /// Pictures that did not make the five: other views of the cast, more shots. + pub more_pictures: Vec, + /// Cues inside the beat: what happens in the sound. + pub cues: Vec, + pub before: Option, + pub after: Option, + /// A prompt to start from, written to how video models read best. + pub outline: String, +} + +/// A bible entry as a brief carries it. +#[derive(Clone, Debug, Serialize)] +pub struct Anchor { + pub entry: u64, + pub kind: Kind, + pub name: String, + pub words: String, + #[serde(skip)] + pub pictures: Vec, +} + +#[derive(Clone, Debug, Serialize)] +pub struct ShotRef { + pub shot: u64, + pub note: String, + #[serde(skip)] + pub picture: Option, +} + +/// A picture a brief sends, and the one job it has. +#[derive(Clone, Debug, Serialize)] +pub struct Picture { + /// "@Image1", in `images`; empty in `more_pictures`. + #[serde(rename = "ref")] + pub tag: String, + /// Who or what it shows. + pub of: String, + /// What the model should take from it. + pub job: String, + #[serde(skip)] + pub still: Still, +} + +/// References that work best: models follow four or five far better than +/// the nine or ten they accept. +pub const MOST_REFERENCES: usize = 5; + +/// The aspect ratio a model names: 16:9, 9:16, 1:1, 4:3, 3:4, 21:9, else W:H. +pub fn aspect(s: Settings) -> String { + let r = s.width as f64 / s.height.max(1) as f64; + let named = [ + ("21:9", 21.0 / 9.0), + ("16:9", 16.0 / 9.0), + ("4:3", 4.0 / 3.0), + ("1:1", 1.0), + ("3:4", 3.0 / 4.0), + ("9:16", 9.0 / 16.0), + ]; + named + .iter() + .find(|(_, v)| (r - v).abs() < 0.02) + .map_or_else(|| format!("{}:{}", s.width, s.height), |(n, _)| n.to_string()) +} + +fn anchor(e: &Entry) -> Anchor { + Anchor { + entry: e.id.0, + kind: e.kind, + name: e.name.clone(), + words: e.words.trim().to_string(), + pictures: e.pictures.clone(), + } +} + +fn job(kind: Kind) -> &'static str { + match kind { + Kind::Look => "colour, light and grain only", + Kind::Character => "face, hair and wardrobe only", + Kind::Place => "the location and its light only", + Kind::Thing => "its shape, colour and marks only", + } +} + +impl Project { + /// The brief for one beat. + pub fn brief(&self, beat: BeatId) -> Option { + let index = self.story.index(beat)?; + let b = &self.story.beats[index]; + let (start, end) = self.story.span(beat)?; + let cast: Vec = b + .cast + .iter() + .filter_map(|id| self.bible.entry(*id)) + .map(anchor) + .collect(); + let look: Vec = self.bible.of(Kind::Look).map(anchor).collect(); + let shots: Vec = b + .shots + .iter() + .map(|s| ShotRef { + shot: s.id.0, + note: s.note.clone(), + picture: s.image.clone(), + }) + .collect(); + // Every candidate picture, best first: the look, faces, places, + // things, shots; then the further pictures of each. + let mut ranked: Vec = Vec::new(); + let mut later: Vec = Vec::new(); + let by_kind = |k: Kind| cast.iter().chain(look.iter()).filter(move |a| a.kind == k); + for kind in [Kind::Look, Kind::Character, Kind::Place, Kind::Thing] { + for a in by_kind(kind) { + for (i, still) in a.pictures.iter().enumerate() { + let picture = Picture { + tag: String::new(), + of: if kind == Kind::Look { + "the look".into() + } else { + a.name.clone() + }, + job: job(kind).into(), + still: still.clone(), + }; + if i == 0 { + ranked.push(picture) + } else { + later.push(picture) + } + } + } + } + for s in &shots { + if let Some(still) = &s.picture { + ranked.push(Picture { + tag: String::new(), + of: if s.note.trim().is_empty() { + "a shot".into() + } else { + s.note.trim().to_string() + }, + job: "the framing only".into(), + still: still.clone(), + }); + } + } + let mut images: Vec = Vec::new(); + for mut p in ranked { + if images.len() < MOST_REFERENCES { + p.tag = format!("@Image{}", images.len() + 1); + images.push(p); + } else { + later.push(p); + } + } + let cues = self + .story + .cues + .iter() + .filter(|c| c.at >= start && c.at < end) + .map(|c| format!("{} at {:.0}s", c.label, secs(c.at - start))) + .collect(); + let title = |i: usize| self.story.beats.get(i).map(|x| x.title.clone()); + let length = end - start; + let mut brief = Brief { + beat: beat.0, + number: index + 1, + title: b.title.clone(), + story: b.story.trim().to_string(), + capture: b.capture.trim().to_string(), + start: secs(start), + duration: secs(length), + generate_seconds: (secs(length).ceil() as u64).clamp(4, 30), + width: self.settings.width, + height: self.settings.height, + fps: self.settings.fps, + aspect: aspect(self.settings), + cast, + look, + shots, + images, + more_pictures: later, + cues, + before: index.checked_sub(1).and_then(title), + after: title(index + 1), + outline: String::new(), + }; + brief.outline = outline(&brief); + Some(brief) + } +} + +fn sentence(text: &str) -> String { + text.trim().trim_end_matches('.').to_string() +} + +/// A prompt skeleton for the beat, in the order video models read best. +fn outline(b: &Brief) -> String { + let mut out = Vec::new(); + // References, each with one job. + if !b.images.is_empty() { + let refs: Vec = b + .images + .iter() + .map(|p| format!("{} is {}: use it for {}.", p.tag, p.of, p.job)) + .collect(); + out.push(refs.join(" ")); + } + // The look, once, before any shot. + let look: Vec = b + .look + .iter() + .map(|l| sentence(&l.words)) + .filter(|w| !w.is_empty()) + .collect(); + if !look.is_empty() { + out.push(format!("Style: {}.", look.join("; "))); + } + // Who and where, anchored in words even when a picture shows them. + for a in &b.cast { + if !a.words.is_empty() { + out.push(format!("{}: {}.", a.name, sentence(&a.words))); + } + } + // The shots: one action each, timed across the length asked for. With + // several reference shots, one timed shot each. + let total = b.generate_seconds as f64; + let noted: Vec<&ShotRef> = b.shots.iter().filter(|s| !s.note.trim().is_empty()).collect(); + let action = sentence(if b.story.is_empty() { &b.title } else { &b.story }); + if noted.len() >= 2 { + let each = total / noted.len() as f64; + out.push(format!("{action}.")); + for (i, s) in noted.iter().enumerate() { + let (a, z) = ((i as f64 * each).round(), ((i + 1) as f64 * each).round()); + out.push(format!("{a:.0}–{z:.0}s: {}.", sentence(&s.note))); + } + } else { + out.push(format!("0–{total:.0}s: {action}.")); + } + if !b.capture.is_empty() { + out.push(format!("Capture: {}.", sentence(&b.capture))); + } + if !b.cues.is_empty() { + out.push(format!("Sound: {}.", b.cues.join("; "))); + } + // What must not change, and what must not appear. + let keep: Vec<&str> = b.cast.iter().map(|a| a.name.as_str()).collect(); + if !keep.is_empty() { + out.push(format!("Keep {} exactly as described throughout.", keep.join(", "))); + } + out.push("No on-screen text, no watermark, no extra characters.".into()); + out.join("\n") +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::time::SECOND; + + fn still(hash: &str) -> Still { + Still { + hash: hash.into(), + ext: "png".into(), + width: 10, + height: 10, + } + } + + #[test] + fn references_are_ranked_capped_and_numbered_once() { + let mut p = Project::default(); + let look = p.add_entry(Kind::Look, "Look"); + p.set_entry(look, |e| e.pictures.push(still("look"))); + let beat = p.add_beat(None, "Busy", 2 * SECOND); + for n in 0..4 { + let c = p.add_entry(Kind::Character, &format!("C{n}")); + p.set_entry(c, |e| { + e.pictures.push(still(&format!("face{n}"))); + e.pictures.push(still(&format!("side{n}"))); + }); + p.toggle_cast(beat, c); + } + p.add_shot(beat, "wide", Some(still("shot"))); + let brief = p.brief(beat).unwrap(); + assert_eq!(brief.images.len(), MOST_REFERENCES); + assert_eq!(brief.images[0].of, "the look", "the look leads"); + assert_eq!(brief.images[1].tag, "@Image2"); + assert!( + brief.more_pictures.iter().any(|p| p.still.hash == "shot"), + "overflow is kept" + ); + assert_eq!(brief.generate_seconds, 4, "models make at least four seconds"); + assert!(brief.outline.starts_with("@Image1 is the look")); + } + + #[test] + fn a_brief_carries_the_cast_the_look_and_an_outline() { + let mut p = Project::default(); + let ada = p.add_entry(Kind::Character, "Ada"); + p.set_entry(ada, |e| e.words = "short silver hair, red coat".into()); + let look = p.add_entry(Kind::Look, "Dusk"); + p.set_entry(look, |e| e.words = "warm low sun, 35mm, soft grain".into()); + let open = p.add_beat(None, "Open", 4 * SECOND); + let walk = p.add_beat(Some(open), "Walk", 6 * SECOND); + p.set_beat(walk, |b| { + b.story = "Ada crosses the bridge".into(); + b.capture = "slow dolly left".into(); + }); + p.toggle_cast(walk, ada); + let brief = p.brief(walk).unwrap(); + assert_eq!((brief.number, brief.duration, brief.aspect.as_str()), (2, 6.0, "16:9")); + assert_eq!(brief.before.as_deref(), Some("Open")); + assert_eq!(brief.cast[0].name, "Ada"); + assert!(brief.outline.contains("Style: warm low sun"), "{}", brief.outline); + assert!(brief.outline.contains("Ada: short silver hair")); + assert!( + brief.outline.contains("0–6s: Ada crosses the bridge."), + "{}", + brief.outline + ); + assert!(brief.outline.contains("Capture: slow dolly left.")); + assert!(brief.outline.contains("Keep Ada exactly")); + assert_eq!(brief.generate_seconds, 6); + } +} diff --git a/crates/splice-core/src/export.rs b/crates/splice-core/src/export.rs index a85ee1b..804f077 100644 --- a/crates/splice-core/src/export.rs +++ b/crates/splice-core/src/export.rs @@ -70,7 +70,36 @@ pub fn plan(project: &Project, resolve: &impl Resolve, output: &Path, quality: Q if project.duration() <= 0 { return Err("the timeline is empty".into()); } - build(project, resolve, output, Some(Out::Movie(quality)), None, false) + build(project, resolve, output, Some(Out::Movie(quality, None)), None, false) +} + +/// Builds the ffmpeg arguments that render just `from` to `to` of `project` +/// to `output`. The movie is drawn up to `to` and the stretch cut from it, +/// so every clip moves exactly as it does there (an entrance under way, a +/// slow zoom), and clips outside it are left out. +pub fn plan_range( + project: &Project, + resolve: &impl Resolve, + output: &Path, + quality: Quality, + from: Time, + to: Time, +) -> Result { + if to <= from { + return Err("the stretch ends before it starts".into()); + } + let part = project.window(from, to); + if part.is_empty() { + return Err("nothing is on the timeline there".into()); + } + build( + &part, + resolve, + output, + Some(Out::Movie(quality, Some((from, to)))), + None, + false, + ) } /// Builds the ffmpeg arguments that render `project` to `output` (a `.mov`) @@ -86,7 +115,8 @@ pub fn plan_layer(project: &Project, resolve: &impl Resolve, output: &Path) -> R /// What a movie export writes. #[derive(Clone, Copy)] enum Out { - Movie(Quality), + /// A movie; with a window, just that stretch of it. + Movie(Quality, Option<(Time, Time)>), Layer, } @@ -143,8 +173,14 @@ fn build( ) -> Result { let layer = see_through; let frame = project.frame(); + let window = match out { + Some(Out::Movie(_, w)) => w, + _ => None, + }; let duration = if still_at.is_some() { frame + } else if let Some((_, to)) = window { + to.max(frame) } else { project.duration().max(frame) }; @@ -240,7 +276,12 @@ fn build( if shows { let look = &clip.look; - let (nw, nh) = natural_size(source, rendered, w, h); + let natural = natural_size(source, rendered, w, h); + // Only media is cropped; titles and colours are drawn to size. + let ((nw, nh), (cx, cy)) = match source { + Source::Media(_) => look.cropped(natural), + Source::Generated(_) => (natural, (0.0, 0.0)), + }; let pose = look.pose_expr(clip.duration); // Local time inside the clip, in each filter's own clock. let local = if offset > 0 { @@ -305,8 +346,16 @@ fn build( }; let animated = pose.x.constant().is_none() || pose.y.constant().is_none() || pose.scale.constant().is_none(); - let x = format!("(W-w)/2+{}*W", pose.x.ffmpeg(&since)); - let y = format!("(H-h)/2+{}*H", pose.y.ffmpeg(&since)); + // A cropped picture's remains stay where they were, scaled with it. + let shift = |by: f64| { + if by.abs() < 1e-6 { + String::new() + } else { + format!("+{by:.3}*{}", pose.scale.ffmpeg(&since)) + } + }; + let x = format!("(W-w)/2+{}*W{}", pose.x.ffmpeg(&since), shift(cx)); + let y = format!("(H-h)/2+{}*H{}", pose.y.ffmpeg(&since), shift(cy)); filters.push(format!( "[{picture}][v{tag}]overlay=x='{x}':y='{y}':{}{}eof_action=pass:\ enable='between(t,{start},{end})'[p{tag}]", @@ -327,17 +376,29 @@ fn build( } } + // Just a stretch: cut from the movie as drawn, then start it at zero. + let (cut, cut_sound) = match window { + Some((a, b)) => ( + format!("trim=start={}:end={},setpts=PTS-STARTPTS,", fmt(a), fmt(b)), + format!(",atrim=start={}:end={},asetpts=PTS-STARTPTS", fmt(a), fmt(b)), + ), + None => (String::new(), String::new()), + }; filters.push(format!( - "[{picture}]format={}[vout]", + "[{picture}]{cut}format={}[vout]", if layer { "yuva444p10le" } else { "yuv420p" } )); if out.is_some() { filters.push(format!( - "[1:a]{}amix=inputs={}:normalize=0:duration=first,{LIMITER}[aout]", + "[1:a]{}amix=inputs={}:normalize=0:duration=first,{LIMITER}{cut_sound}[aout]", sounds.join(""), sounds.len() + 1 )); } + let (duration, d) = match window { + Some((a, b)) => (b - a, fmt(b - a)), + None => (duration, d), + }; args.extend(strings(&["-filter_complex", &filters.join(";")])); let Some(out) = out else { @@ -345,7 +406,7 @@ fn build( args.push(output.to_string_lossy().to_string()); return Ok(Plan { args, duration }); }; - let Out::Movie(quality) = out else { + let Out::Movie(quality, _) = out else { args.extend(strings(&[ "-map", "[vout]", @@ -548,6 +609,27 @@ mod tests { assert!(plan_still(&p, &Fake, 9 * SECOND, Path::new("/o.png")).is_err()); } + #[test] + fn a_stretch_is_cut_from_the_movie_as_drawn() { + let mut p = Project::default(); + let v = video(&mut p, "a", 4); + p.append_clip(v); + let plan = plan_range( + &p, + &Fake, + Path::new("/tmp/part.mp4"), + Quality::Draft, + SECOND, + 3 * SECOND, + ) + .unwrap(); + let graph = graph(&plan); + assert!(graph.contains("trim=start=1.000000:end=3.000000,setpts=PTS-STARTPTS")); + assert!(graph.contains("atrim=start=1.000000:end=3.000000")); + assert_eq!(plan.duration, 2 * SECOND); + assert!(plan_range(&p, &Fake, Path::new("/tmp/x.mp4"), Quality::Draft, 3 * SECOND, SECOND).is_err()); + } + #[test] fn empty_timeline_is_an_error() { assert!(plan(&Project::default(), &Fake, Path::new("/o.mp4"), Quality::Draft).is_err()); diff --git a/crates/splice-core/src/lib.rs b/crates/splice-core/src/lib.rs index 4a7edee..d5d2e16 100644 --- a/crates/splice-core/src/lib.rs +++ b/crates/splice-core/src/lib.rs @@ -2,6 +2,8 @@ //! single-file project format, and export plans. The app crate owns windows, //! decoding and sound. +pub mod bible; +pub mod brief; pub mod export; pub mod history; pub mod look; diff --git a/crates/splice-core/src/look.rs b/crates/splice-core/src/look.rs index 19790a0..094d781 100644 --- a/crates/splice-core/src/look.rs +++ b/crates/splice-core/src/look.rs @@ -24,6 +24,10 @@ pub struct Look { pub key: Option, #[serde(skip_serializing_if = "Option::is_none")] pub mask: Option, + /// Edges cut away. The picture keeps its scale, and what remains stays + /// where it was on the canvas. + #[serde(skip_serializing_if = "Option::is_none")] + pub crop: Option, pub enter: Motion, pub exit: Motion, pub drift: Drift, @@ -38,6 +42,7 @@ impl Default for Look { opacity: 1.0, key: None, mask: None, + crop: None, enter: Motion::default(), exit: Motion::default(), drift: Drift::None, @@ -77,6 +82,83 @@ pub enum Shape { Ellipse, } +/// Edges cut from a picture, each a fraction of its frame (0 keeps the edge). +#[derive(Clone, Copy, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default)] +pub struct Crop { + pub left: f32, + pub top: f32, + pub right: f32, + pub bottom: f32, +} + +impl Crop { + /// At least a twentieth of the picture always remains. + const KEEP: f32 = 0.05; + + /// The same crop, each edge in range and never crossing the other. + pub fn clamped(self) -> Crop { + let edge = |v: f32| { + if v.is_finite() { + v.clamp(0.0, 1.0 - Self::KEEP) + } else { + 0.0 + } + }; + let (mut l, mut r, mut t, mut b) = (edge(self.left), edge(self.right), edge(self.top), edge(self.bottom)); + let over = (l + r) - (1.0 - Self::KEEP); + if over > 0.0 { + let k = (1.0 - Self::KEEP) / (l + r); + l *= k; + r *= k; + } + let over = (t + b) - (1.0 - Self::KEEP); + if over > 0.0 { + let k = (1.0 - Self::KEEP) / (t + b); + t *= k; + b *= k; + } + Crop { + left: l, + top: t, + right: r, + bottom: b, + } + } + + pub fn is_none(&self) -> bool { + self.left <= 0.0 && self.top <= 0.0 && self.right <= 0.0 && self.bottom <= 0.0 + } + + /// The size of what remains of a picture of `natural` size. + pub fn size(&self, natural: (f64, f64)) -> (f64, f64) { + ( + natural.0 * (1.0 - self.left - self.right) as f64, + natural.1 * (1.0 - self.top - self.bottom) as f64, + ) + } + + /// How far the centre of what remains sits from the picture's centre, + /// in the picture's pixels at scale 1: where to draw it so it stays put. + pub fn offset(&self, natural: (f64, f64)) -> (f64, f64) { + ( + natural.0 * (self.left - self.right) as f64 / 2.0, + natural.1 * (self.top - self.bottom) as f64 / 2.0, + ) + } + + /// The ffmpeg filter that cuts it. + pub fn ffmpeg(&self) -> String { + format!( + "crop=w='trunc(iw*{:.5}/2)*2':h='trunc(ih*{:.5}/2)*2':x='iw*{:.5}':y='ih*{:.5}'", + 1.0 - self.left - self.right, + 1.0 - self.top - self.bottom, + self.left, + self.top + ) + } +} + /// Shows only part of the clip. Measured in fractions of the clip's own frame. #[derive(Clone, Copy, Debug, PartialEq, Serialize, Deserialize)] #[serde(default)] @@ -371,6 +453,10 @@ impl Look { /// the viewer's decoder and export. They leave an RGBA frame. pub fn pixel_filters(&self) -> Option { let mut filters = Vec::new(); + // Cut first, so a key and a mask work on what remains. + if let Some(crop) = self.crop.filter(|c| !c.is_none()) { + filters.push(crop.ffmpeg()); + } if let Some(key) = self.key { filters.push(format!( "chromakey=0x{:06x}:{:.4}:{:.4}", @@ -401,6 +487,17 @@ impl Look { } } +impl Look { + /// A picture's size at scale 1 once cropped, and where to shift it so + /// what remains stays in place (canvas pixels). + pub fn cropped(&self, natural: (f64, f64)) -> ((f64, f64), (f64, f64)) { + match self.crop.filter(|c| !c.is_none()) { + Some(c) => (c.size(natural), c.offset(natural)), + None => (natural, (0.0, 0.0)), + } + } +} + impl Mask { /// How much of the clip shows at (u, v), fractions of its frame: 0 to 1. pub fn alpha(&self, u: f32, v: f32) -> f32 { @@ -431,6 +528,32 @@ impl Mask { mod tests { use super::*; + #[test] + fn crops_keep_what_remains_in_place() { + let crop = Crop { + left: 0.25, + top: 0.0, + right: 0.0, + bottom: 0.5, + }; + assert_eq!(crop.size((400.0, 200.0)), (300.0, 100.0)); + // A quarter off the left: what remains sits 50 px right of centre. + assert_eq!(crop.offset((400.0, 200.0)), (50.0, -50.0)); + let wild = Crop { + left: 0.8, + right: 0.8, + top: f32::NAN, + bottom: -1.0, + } + .clamped(); + assert!(wild.left + wild.right <= 0.95 + 1e-6 && wild.top == 0.0 && wild.bottom == 0.0); + let look = Look { + crop: Some(crop), + ..Look::default() + }; + assert!(look.pixel_filters().unwrap().starts_with("crop=")); + } + #[test] fn still_clips_do_not_move() { let look = Look::default(); diff --git a/crates/splice-core/src/model.rs b/crates/splice-core/src/model.rs index d919d3c..d70204c 100644 --- a/crates/splice-core/src/model.rs +++ b/crates/splice-core/src/model.rs @@ -354,6 +354,9 @@ pub struct Project { /// The plan: story beats along the timeline, and cues. #[serde(default, skip_serializing_if = "crate::story::Story::is_empty")] pub story: crate::story::Story, + /// Who and what the movie keeps coming back to, and its look. + #[serde(default, skip_serializing_if = "crate::bible::Bible::is_empty")] + pub bible: crate::bible::Bible, /// In a symbol: the stretch of it (start, end) that a longer or shorter /// use stretches, so what comes before and after (an entrance, an exit) /// keeps its timing. None stretches all of it evenly. @@ -375,6 +378,7 @@ impl Default for Project { hidden: false, }], story: crate::story::Story::default(), + bible: crate::bible::Bible::default(), hold: None, next_id: 2, } @@ -848,6 +852,18 @@ impl Project { copies } + /// The timeline with only what plays between `from` and `to`: clips + /// wholly outside it go, the rest stay as they are (so they move as they + /// do in the whole movie). Exporting a stretch cuts it from this. + pub fn window(&self, from: Time, to: Time) -> Project { + let mut out = self.clone(); + out.story = crate::story::Story::default(); + for track in &mut out.tracks { + track.clips.retain(|c| c.start < to && c.end() > from); + } + out + } + pub fn set_volume(&mut self, ids: &[ClipId], volume: f32) { for &id in ids { if let Some(clip) = self.clip_mut(id) { @@ -1067,6 +1083,17 @@ mod tests { assert!(Project::from_json(&newer).is_err()); } + #[test] + fn a_window_keeps_what_plays_in_it_unchanged() { + let (mut p, m) = project_with(4 * SECOND); + p.append_clip(m); + p.append_clip(m); + let part = p.window(SECOND, 3 * SECOND); + assert_eq!(part.clips().count(), 1); + let c = &part.tracks[0].clips[0]; + assert_eq!((c.start, c.source_in, c.duration), (0, 0, 4 * SECOND)); + } + #[test] fn duplicate_lands_after() { let (mut p, m) = project_with(4 * SECOND); diff --git a/crates/splice-core/src/story.rs b/crates/splice-core/src/story.rs index f8d05c4..78458f3 100644 --- a/crates/splice-core/src/story.rs +++ b/crates/splice-core/src/story.rs @@ -64,6 +64,71 @@ pub struct Beat { /// Reference shots, in order. #[serde(default, skip_serializing_if = "Vec::is_empty")] pub shots: Vec, + /// Who and what from the bible is in it. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub cast: Vec, + /// Footage tried for it (generated, or shot again), one of them laid in. + #[serde(default, skip_serializing_if = "Vec::is_empty")] + pub takes: Vec, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq, Hash, PartialOrd, Ord, Serialize, Deserialize)] +#[serde(transparent)] +pub struct TakeId(pub u64); + +/// A take for a beat: media tried for it, and where it came from. A take +/// whose media has not arrived yet (a generation under way) is pending. +#[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] +pub struct Take { + pub id: TakeId, + #[serde(default, skip_serializing_if = "Option::is_none")] + pub media: Option, + #[serde(default)] + pub origin: Origin, + /// Frames to skip at its start when laid in (a generated extension's + /// first frames repeat what it extends). + #[serde(default, skip_serializing_if = "is_zero")] + pub skip: Time, + /// The clip it laid into the beat, when it is the one chosen. + #[serde(default, skip_serializing_if = "Option::is_none")] + pub clip: Option, + /// When it was asked for (seconds since 1970), to tell a slow generation + /// from a lost one. + #[serde(default, skip_serializing_if = "is_zero_u64")] + pub started: u64, +} + +fn is_zero_u64(n: &u64) -> bool { + *n == 0 +} + +fn is_zero(t: &Time) -> bool { + *t == 0 +} + +/// Where a take came from: enough to make it again, or a variation of it. +#[derive(Clone, Debug, Default, PartialEq, Serialize, Deserialize)] +#[serde(default)] +pub struct Origin { + /// Who made it: "higgsfield", "camera", "recording". + #[serde(skip_serializing_if = "String::is_empty")] + pub tool: String, + /// The model: "seedance_2_5". + #[serde(skip_serializing_if = "String::is_empty")] + pub model: String, + /// How: "t2v", "omni_reference", "video_edit", "video_extension". + #[serde(skip_serializing_if = "String::is_empty")] + pub mode: String, + #[serde(skip_serializing_if = "String::is_empty")] + pub prompt: String, + /// What it was given, each with its role: "@Image1 Ada (face)". + #[serde(skip_serializing_if = "Vec::is_empty")] + pub references: Vec, + /// The job, to look it up again. + #[serde(skip_serializing_if = "String::is_empty")] + pub job: String, + #[serde(skip_serializing_if = "String::is_empty")] + pub note: String, } #[derive(Clone, Debug, PartialEq, Serialize, Deserialize)] @@ -147,6 +212,8 @@ impl Project { duration: duration.max(self.frame()), color, shots: Vec::new(), + cast: Vec::new(), + takes: Vec::new(), }; // Beats after it make room, and their footage moves with them. let shift = beat.duration; @@ -326,6 +393,139 @@ impl Project { Some(clip) } + /// Adds a take to a beat: media, or none yet (pending, made at + /// `started`, seconds since 1970). Returns its id. + pub fn add_take( + &mut self, + beat: BeatId, + media: Option, + origin: Origin, + started: u64, + ) -> Option { + self.story.index(beat)?; + let id = TakeId(self.alloc()); + self.set_beat(beat, |b| { + b.takes.push(Take { + id, + media, + origin, + skip: 0, + clip: None, + started, + }) + }); + Some(id) + } + + /// The beat a take belongs to. + pub fn take_beat(&self, take: TakeId) -> Option { + self.story + .beats + .iter() + .find(|b| b.takes.iter().any(|t| t.id == take)) + .map(|b| b.id) + } + + pub fn take(&self, take: TakeId) -> Option<&Take> { + self.story.beats.iter().flat_map(|b| &b.takes).find(|t| t.id == take) + } + + pub fn set_take(&mut self, take: TakeId, f: impl FnOnce(&mut Take)) { + if let Some(t) = self + .story + .beats + .iter_mut() + .flat_map(|b| &mut b.takes) + .find(|t| t.id == take) + { + f(t); + } + } + + /// The take laid into a beat: the one whose own clip is on the timeline. + pub fn chosen_take(&self, beat: BeatId) -> Option { + let b = self.story.beats.iter().find(|b| b.id == beat)?; + b.takes + .iter() + .rev() + .find(|t| t.clip.is_some_and(|c| self.slot(c).is_some())) + .map(|t| t.id) + } + + /// Lays a take into its beat in place of the beat's other takes: at the + /// beat's start, as long as the beat, skipping its first `skip`. Only + /// clips a take of this beat laid in are replaced, so footage placed by + /// hand (even of the same media) is never touched. + pub fn choose_take(&mut self, take: TakeId) -> Option { + let beat = self.take_beat(take)?; + let chosen = self.take(take)?.clone(); + let media = chosen.media?; + // Already laid in: nothing to do. + if let Some(clip) = chosen.clip.filter(|c| self.slot(*c).is_some()) { + return Some(clip); + } + let theirs: Vec = self + .story + .beats + .iter() + .find(|b| b.id == beat)? + .takes + .iter() + .filter_map(|t| t.clip) + .collect(); + let (start, end) = self.story.span(beat)?; + // Out with the takes laid in before, keeping their track. + let mut track = None; + for (i, t) in self.tracks.iter_mut().enumerate() { + let before = t.clips.len(); + t.clips.retain(|c| !theirs.contains(&c.id)); + if t.clips.len() != before && track.is_none() { + track = Some(i); + } + } + // Placed straight, not through `insert_clip`, so a take's media is + // never forked or bumped onto another clip of the same media. + let timed = self.media(media).is_some_and(|m| m.is_timed()); + let length = self.media(media).map_or(0, |m| m.duration); + let frame = self.frame(); + let room = end - start; + let (source_in, duration) = if timed { + let skip = chosen.skip.min((length - frame).max(0)); + (skip, (length - skip).min(room).max(frame)) + } else { + (0, room) + }; + let id = ClipId(self.alloc()); + let clip = Clip { + id, + media, + start, + source_in, + duration, + volume: 1.0, + look: crate::look::Look::default(), + generator: None, + placeholder: None, + }; + self.place(clip, track.unwrap_or(0)); + self.set_take(take, |t| t.clip = Some(id)); + self.prune_tracks(); + Some(id) + } + + /// Takes a take out of its beat, and the clip it laid in (if any). + pub fn remove_take(&mut self, take: TakeId) { + if let Some(clip) = self.take(take).and_then(|t| t.clip) { + for t in &mut self.tracks { + t.clips.retain(|c| c.id != clip); + } + self.prune_tracks(); + } + for b in &mut self.story.beats { + b.takes.retain(|t| t.id != take); + } + } + /// Adds a reference shot to a beat. Returns its id. pub fn add_shot(&mut self, beat: BeatId, note: &str, image: Option) -> Option { self.story.index(beat)?; @@ -381,6 +581,16 @@ impl Project { /// Every content hash the project needs kept: its media and the story's /// reference pictures. pub fn hashes(&self) -> Vec { + let pictures = self + .bible + .entries + .iter() + .flat_map(|e| &e.pictures) + .map(|s| s.hash.clone()); + self.media_hashes().into_iter().chain(pictures).collect() + } + + fn media_hashes(&self) -> Vec { self.media .iter() .flat_map(|m| { @@ -588,6 +798,27 @@ mod tests { assert!(p.add_shot(BeatId(999), "x", None).is_none()); } + #[test] + fn takes_replace_only_their_own_clips() { + let (mut p, media) = project(); + let open = p.add_beat(None, "Open", 4 * SECOND); + let next = p.add_beat(Some(open), "Next", 4 * SECOND); + // The same media laid by hand into the first beat. + let by_hand = p.fill_beat(open, media).unwrap(); + let a = p.add_take(next, Some(media), Origin::default(), 0).unwrap(); + let b = p.add_take(next, Some(media), Origin::default(), 0).unwrap(); + let first = p.choose_take(a).unwrap(); + assert_eq!(p.chosen_take(next), Some(a)); + let second = p.choose_take(b).unwrap(); + assert!(p.slot(by_hand).is_some(), "footage placed by hand stays"); + assert!(p.slot(first).is_none(), "the beat's other take goes"); + assert_eq!(p.chosen_take(next), Some(b)); + assert_eq!(p.clip(second).unwrap().start, 4 * SECOND); + p.remove_take(b); + assert!(p.slot(second).is_none()); + assert!(p.slot(by_hand).is_some()); + } + #[test] fn old_projects_have_no_story() { let json = serde_json::to_string(&Project::default()).unwrap(); diff --git a/skill/splice/SKILL.md b/skill/splice/SKILL.md index 4376f77..bfc75a0 100644 --- a/skill/splice/SKILL.md +++ b/skill/splice/SKILL.md @@ -116,7 +116,8 @@ splice ctl style --clip 12 --text "Day two" --color gold --italic splice ctl color --color black --at 0 --duration 2 # a title card or background splice ctl color --color "#00000080" --at 10 --duration 4 # a tint over what is under it splice ctl look --clip 7 --scale 0.4 --x 0.28 --y -0.25 # picture in picture, top right -splice ctl look --clip 7 --mask ellipse --feather 0.15 # a soft round crop +splice ctl look --clip 7 --mask ellipse --feather 0.15 # a soft round vignette +splice ctl look --clip 7 --crop-left 0.2 --crop-bottom 0.1 # cut edges away (--crop none) splice ctl look --clip 9 --key green --similarity 0.18 # green screen (blue, #rrggbb too) splice ctl look --clip 3 --drift zoom-in --enter fade --enter-for 1 ``` @@ -333,6 +334,76 @@ splice ctl view story # show the person the boa - Then fill the beats with footage (`story fill`, or record). `coverage` says which beats still need it. +### The bible and cast + +The bible holds who and what the movie keeps coming back to: characters, +places and things, each with pictures (the first is its face) and the words +that must read the same in every shot, plus one **look** for the whole +movie. Beats name their cast from it. + +```sh +splice ctl bible add --kind character --name Ada --words "short silver hair, red wool coat" --image ada.png +splice ctl bible add --kind place --name Bridge --words "iron footbridge, wet cobbles" --image bridge.jpg +splice ctl bible add --kind look --name Look --words "warm dusk light, 35mm, soft grain" +splice ctl bible picture --entry Ada --image ada-profile.png # more pictures; --remove I +splice ctl story cast --beat 27 --add Ada --add Bridge +splice ctl bible read +``` + +### Generating footage (Seedance through Higgsfield) + +Splice does not call video models itself. You do, with the Higgsfield tools, +and Splice keeps the plan, the references and the takes. For each beat: + +1. `splice ctl story brief --beat B` writes the beat's references to a folder + and returns everything about it: + - what happens and what to capture; + - its length, plus `generate_seconds`: the length to ask for, rounded into + the 4–30 s models make. Trim the take back to the beat's length; + - its aspect ratio, its cast's words, the look and its cues; + - the frames just before, at the start, at the end and just after it. + + `images` holds at most five references, each with one job and its `file`, + numbered @Image1… in the order the `outline` uses them: the look, faces, + places, things, then shots. Pictures that did not make the five are in + `more_pictures`. +2. Write the prompt from `outline`, following what works for Seedance: + - give each reference one job ("@Image1 is Ada: face and wardrobe only"), + and use four or five references at most; + - put the style once at the top, from the look; + - anchor the cast in words, even when a picture shows them; + - one action and one camera move per shot, with timed shots ("0–4s …"); + - say what must not change; + - don't describe what a reference already shows. +3. Choose the mode: + - text or references to video for a new shot; + - first and last frames (`frames.before` / `frames.after`) to bridge + neighbouring shots; + - `video_edit` to change footage already in the beat. Export it first + with `splice ctl export clip.mp4 --beat B`; it must be 4–30 s; + - `video_extension` to run a shot on. + Ask before spending credits, and use the beat's length and the movie's + aspect ratio. +4. Record the generation as soon as it starts: `splice ctl story take + pending --beat B --tool higgsfield --model M --mode X --prompt P --job J`. + The beat then shows it as generating, with how long it has run. +5. When the job finishes, download it and run `splice ctl story take land + --take T --video FILE`: + - add `--seam` for an extension, so its first three frames, which repeat + the clip it extends, are skipped; + - the take is laid into the beat unless you pass `--keep`; + - the reply's `take` says whether it was chosen and whether it is + `short_by` or `trimmed_by` against the beat. + + If a generation fails, run `story take fail --take T`. Choosing a take + replaces only the clips other takes of that beat laid in; footage placed + by hand is never touched. +6. Make two or three takes when the person wants a choice. They pick one on + the beat card, or you run `story take choose --take T`. `story read` + lists the takes, each with its prompt, so you can vary the best one. + +Check each landed take with `splice ctl frame` before calling it done. + ### Recording the screen ```sh