From 4470012abaf022831adc761e00880083be09f971 Mon Sep 17 00:00:00 2001 From: Mikkel Garcia Date: Tue, 28 Jul 2026 22:27:01 -0600 Subject: [PATCH] =?UTF-8?q?references:=20local=20media=20nodes=20DO=20run?= =?UTF-8?q?=20headlessly=20=E2=80=94=20say=20what=20each=20one=20needs?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit references/graph-format.md told an agent that resize, vframes, combine, soundtrack, trim and extractaudio are "browser-only media processing", and that the executors load such graphs with a warning and fail fast at run with UnsupportedNodeError. That is false, and SKILL.md line 248 already said the opposite in the same skill. The truth, read off the executors: - nanoodle-js src/local-media.mjs implements a pure-JS path (PNG resize, PCM-WAV trim, MP4CAT lossless mp4 concat) and shells out to ffmpeg/ffprobe on PATH for everything else. - nanoodle-py src/nanoodle/local_media.py has no pure path at all: every one of the 6 node types shells out to ffmpeg. - Neither library marks any node type unsupported. In nanoodle-py, graph.UNSUPPORTED_TYPES evaluates to the empty tuple; nanoodle-js has no equivalent flag. UnsupportedNodeError is raised only for an UNKNOWN node type. A false "not supported" costs an agent a capability it has. The fix states the truth per node and per language, and keeps the up-front-refusal sentence where it belongs: unknown types only. Co-Authored-By: Claude Opus 5 (1M context) --- SKILL.md | 4 +++- references/graph-format.md | 37 +++++++++++++++++++++++++++---------- 2 files changed, 30 insertions(+), 11 deletions(-) diff --git a/SKILL.md b/SKILL.md index dc73c4b..46b9837 100644 --- a/SKILL.md +++ b/SKILL.md @@ -247,7 +247,9 @@ comment otherwise. The Python CLI (PyPI 0.2.0+) runs share URLs the same way. a pure-JS path that matches the browser (lossless mp4 remux, PCM-WAV trim, PNG resize) and falls back to ffmpeg on `PATH` for everything else; the Python CLI needs ffmpeg on `PATH` for all of them. ffmpeg is a soft - dependency — a clear error if it's required and missing, before any paid call. + dependency — a clear error if it's required and missing. That error arrives + when the node runs, so put a media-only graph through once before you wire a + paid node in front of it. - **Media rides inline as base64** (NanoGPT has no upload endpoint). Files over ~4.4 MB (~3.5 MB for transcription) are refused locally before any paid call. - Missing keys, bad inputs, and unknown node types all fail **before** anything diff --git a/references/graph-format.md b/references/graph-format.md index 2c19d05..193ad36 100644 --- a/references/graph-format.md +++ b/references/graph-format.md @@ -64,30 +64,47 @@ Port kinds: text | image | audio | video. | image | Image | — | — | image:image | model, prompt, size, variations, seed, customCivitaiAir | | edit | Edit | — | image, image2..:image | image:image | model, prompt, size, seed | | inpaint | Inpaint | image:image, mask:image | — | image:image | model, prompt, size, seed, brush | -| resize | Resize/crop | image:image | — | image:image | mode(fit\|fill\|exact), width, height (LOCAL, browser-only) | +| resize | Resize/crop | image:image | — | image:image | mode(fit\|fill\|exact), width, height (LOCAL media) | | vision | Vision | image:image | — | text:text | model, q | | tvideo | Text→Video | — | ref1..:image | video:video | model, prompt, duration, aspect, resolution, modelOpts | | ivideo | Image→Video | image:image | endframe:image | video:video | model, prompt, duration, aspect, resolution, modelOpts | | vedit | Video edit | video:video | — | video:video | model, prompt, resolution, modelOpts | -| vframes | Video→frames | video:video | — | frame1..frameN:image | frames(1-12), gap, dir(end\|start) (LOCAL, browser-only) | -| combine | Combine videos | — | clip1..:video | video:video | dedup (LOCAL, browser-only) | -| soundtrack | Soundtrack | video:video, audio:audio | — | video:video | loop (LOCAL, browser-only) | +| vframes | Video→frames | video:video | — | frame1..frameN:image | frames(1-12), gap, dir(end\|start) (LOCAL media) | +| combine | Combine videos | — | clip1..:video | video:video | dedup (LOCAL media) | +| soundtrack | Soundtrack | video:video, audio:audio | — | video:video | loop (LOCAL media) | | lipsync | Avatar/lipsync | image:image, audio:audio | — | video:video | model, prompt, resolution, modelOpts | | music | Music | — | — | audio:audio | model, prompt, lyrics, instrumental, duration, negative_prompt, seed, extraJson | | remix | Remix audio | audio:audio | — | audio:audio | model, prompt, lyrics, duration, extraJson | | tts | Speech | — | — | audio:audio | model, prompt, voice, speed, instructions, extraJson | -| trim | Trim audio | audio:audio | — | audio:audio | start, length (LOCAL, browser-only) | -| extractaudio | Extract audio | video:video | — | audio:audio | start, length (LOCAL, browser-only) | +| trim | Trim audio | audio:audio | — | audio:audio | start, length (LOCAL media) | +| extractaudio | Extract audio | video:video | — | audio:audio | start, length (LOCAL media) | | transcribe | Transcribe | audio:audio | — | text:text | model, language | | comment | Comment | — (note; never runs) | — | — | text, color | Headless executor support: local nodes (text, upload, aupload, vupload, choice, join, comment) run in-process; NanoGPT nodes (llm, image, edit, inpaint, vision, tvideo, ivideo, vedit, lipsync, music, remix, tts, transcribe) call the -API; the rows marked LOCAL/browser-only (resize, vframes, combine, soundtrack, -trim, extractaudio) are browser-only media processing — the executors load such -graphs with a warning and fail fast at run with `UnsupportedNodeError`, before -any network call. +API; the rows marked LOCAL media (resize, vframes, combine, soundtrack, trim, +extractaudio) also run headlessly, on the machine you run them on, and never +spend. Some of them need `ffmpeg` and `ffprobe` on `PATH`: + +| node | Node CLI / nanoodle-js | Python CLI / nanoodle-py | +|---|---|---| +| resize | pure JS for a PNG source, else ffmpeg | ffmpeg | +| trim | pure JS for a PCM WAV source, else ffmpeg | ffmpeg | +| combine | pure JS lossless mp4 remux when every clip is mp4 with matching stream parameters, else ffmpeg | ffmpeg | +| vframes | ffmpeg | ffmpeg | +| soundtrack | ffmpeg | ffmpeg | +| extractaudio | ffmpeg | ffmpeg | + +ffmpeg is a soft dependency, not a package dependency. A node that needs it and +cannot find it stops the run with a clear error naming it. That error arrives +when the node's turn comes, so paid nodes earlier in the graph have already +spent — run a media-only graph once to prove the setup first. + +Only an *unknown* node type is refused up front: the libraries warn at load and +throw `UnsupportedNodeError` at run, before any network call. That is what a +graph saved before 2026-07-22 with the retired `draw` node does. Inpaint note: the browser app composites the mask onto black at the source pixel size; the executor libraries pass your mask through verbatim — supply a