From 265ee5f114708b395766953d76fdf7fa80677a2f Mon Sep 17 00:00:00 2001 From: Spencer Qian Date: Mon, 17 Aug 2026 13:44:51 -0700 Subject: [PATCH 1/2] feat: document stems separation on text-to-music and video-to-music stems=true on text_to_music and video_to_music splits each generated track into four separated instrument tracks (drums/bass/vocals/other), returned as a stems array alongside the clean audio in the task result. Live in prod on REST and the hosted MCP server; free of charge. - text-to-music/SKILL.md, video-to-music/SKILL.md: stems parameter row, a dedicated Stems section (result shape, stream_index lookup, stems_error semantics, +2-6 min timing with a 30 min cap, fixed htdemucs stem names, async-only on REST), and a workflow tip. The video-to-music section also spells out that it splits the GENERATED music, never the source audio - that is preserve_speech's job. - references/api-claims.md: new dated stems facts section; the 'not stems-in-DAW-sense' output claim from 2026-07-28 is now false and is corrected in place. - task-recovery/SKILL.md: the 404 row's 'purely streaming' parenthetical now names the async triggers (wav/variants/stems) instead of implying text_to_music never has a task. - tests/tool_surface.json: stems added to the hosted block for both tools and to must_document, so a skill covering either tool that goes silent about stems fails the build. The local block is deliberately untouched: the published sonilo-mcp (0.17.0) does not accept stems yet, and CI diffs that block against the installed package weekly - the skills scope the claim to REST + hosted MCP accordingly, with a _generated note saying --refresh will pick up the local side when the next sonilo-mcp release ships. Validated: tests/validate.py green offline, and --refresh against a fresh install of the published sonilo-mcp 0.17.0 reports no drift. --- references/api-claims.md | 15 ++++++++++++- task-recovery/SKILL.md | 2 +- tests/tool_surface.json | 15 ++++++++----- text-to-music/SKILL.md | 48 +++++++++++++++++++++++++++++++++++++--- video-to-music/SKILL.md | 36 ++++++++++++++++++++++++++++-- 5 files changed, 104 insertions(+), 12 deletions(-) diff --git a/references/api-claims.md b/references/api-claims.md index 59ce5d8..c9fc10b 100644 --- a/references/api-claims.md +++ b/references/api-claims.md @@ -37,6 +37,19 @@ Added 2026-08-16 with the endpoint itself; verified against the shipped backend - [x] The variation prompts are **narrower than what the upstream produces** by product decision: `negative_prompt`, `thinking`, `structure_source` and the variation title/summary/tags are stripped before the envelope is built and are not recoverable from the task. - [x] ⚠️ **Input differs by MCP server**: the hosted server exposes `video_url` only; local `sonilo-mcp` (0.17.0+) also takes `video_path`. Both SDKs and both CLIs accept a local file or a URL. +## stems (text_to_music + video_to_music) + +Added 2026-08-17, verified against the shipped backend (live on REST `/v1/text-to-music` + `/v1/video-to-music` and on the hosted MCP server's `text_to_music` + `video_to_music`). + +- [x] **Free of charge.** Splits each generated track into four separated instrument tracks — `drums`, `bass`, `vocals`, `other` — delivered as a `stems` array alongside the clean `audio` in the task result. +- [x] REST requires `mode=async` (`stems=true` in stream mode = **400**); MCP is always async, so the param just works there. +- [x] Result entry shape: `{ stream_index, drums, bass, vocals, other }`, each stem `{ url, content_type, file_size }`. **Look entries up by `stream_index`, never by position** — a stream whose separation failed is absent, so `stems` can be shorter than `audio`. +- [x] `stems_error` (string) appears when separation failed wholly/partly or was skipped, and **can appear alongside a partial `stems`**. The generation itself succeeded and the audio URLs are valid — a missing extra, never a failed generation. +- [x] Separation runs after generation: typically **+2–6 min**, gives up after **30 min**. Stems normally follow `output_format`; each stem's `content_type` reports what was delivered. +- [x] On `video_to_music` it splits the **generated** music, never the video's own audio (source speech = `preserve_speech`, unrelated). +- [x] The four stem names are fixed (htdemucs): melodic instruments land in `other`; on instrumental tracks `vocals` is near-silent — correct behavior, not a bug. +- [x] ⚠️ **Surface gap as of 2026-08-17**: the local `sonilo-mcp` package (0.17.0) does not accept `stems` yet, and neither do the SDKs or CLIs — hosted MCP and REST only. Re-verify when the next sonilo-mcp release lands and update the two skills' surface notes plus `tests/tool_surface.json` (`--refresh` picks up the local block). + ## Billing / general - [x] Charged up front at submission; **failed generations auto-refunded**. Caller retries = new charge. **No preview/low-cost mode.** Music + SFX = separate task types, separate per-second rates, separate prepay minute pools. `variants_num` scales v2m cost linearly; N>1 never covered by free trial. @@ -44,7 +57,7 @@ Added 2026-08-16 with the endpoint itself; verified against the shipped backend 1. MCP input = **`video_url` only**, no file upload 2. MCP **always async** — no `mode` param; returns `task_id`, results via `get_generation_task` (named `get_sfx_task` on the local server) 3. MCP `video_to_music` has **no `segments` param** — segmented music via MCP only through section-shaped prompt text (prompt-analysis path) -- [x] Output = **audio files only** (not stems-in-DAW-sense, not video). v2m: m4a default, `output_format=wav` optional (async-only on REST; always on MCP); preserve_speech adds vocals track + mux; ducking adds ducked music URLs. v2sfx: single file, aac default, wav/mp3/flac optional. Video out = separate `/v1/video-to-video-*` endpoints + corresponding MCP tools. +- [x] Output = **audio files only** (not video). v2m: m4a default, `output_format=wav` optional (async-only on REST; always on MCP); preserve_speech adds vocals track + mux; ducking adds ducked music URLs. v2sfx: single file, aac default, wav/mp3/flac optional. Video out = separate `/v1/video-to-video-*` endpoints + corresponding MCP tools. ⚠️ An earlier pass said "not stems-in-DAW-sense" — no longer true: `stems=true` on t2m/v2m (added 2026-08-17, see the stems section below) returns exactly that. - [x] Multi-track input — default ffmpeg stream selection (typically first audio track) for ducking/speech. Wording: "for multi-track videos, the default audio track is used." ## Empirical test (2026-07-29) diff --git a/task-recovery/SKILL.md b/task-recovery/SKILL.md index 8280fa4..bef0c0c 100644 --- a/task-recovery/SKILL.md +++ b/task-recovery/SKILL.md @@ -100,7 +100,7 @@ The MCP tool does one status check and, if `status` is terminal, downloads and s | `processing` | Returns a "still processing, try again later" message. No file saved. Call again after a short wait. | | `succeeded` | Downloads and saves the result — audio for `text_to_sfx`/`video_to_sfx`; a single `.wav` or `.mp4` for `audio_ducking`; a single `.mp4` for `video_to_video_music`/`video_to_video_sfx`/`video_to_video_sound`; a single `.wav` for `video_to_sound`; one `.mp4` per language for `dubbing`; for a `video_to_music(preserve_speech=true)` task, the music stream(s) plus the `vocals` stem plus the `mux`. An `analyze_video` task is the exception: it has no file at all, so the creative brief comes back inline as JSON and nothing is written to disk. | | `failed` | Raises an error including the backend's error code/message and whether the charge was **refunded** — check this before telling the user they've been billed for nothing. | -| task id not found (404) | The id is wrong/typo'd, or belongs to a purely streaming call (`text_to_music`, or `video_to_music` without `preserve_speech`/`ducking`/`output_format="wav"`) — those have no task at all and can't be recovered this way. Nothing to retry. | +| task id not found (404) | The id is wrong/typo'd, or belongs to a purely streaming call (`text_to_music` in its default streaming form — no `wav`, no `variants_num` above 1, no `stems` — or `video_to_music` without `preserve_speech`/`ducking`/`stems`/`output_format="wav"`) — those have no task at all and can't be recovered this way. Nothing to retry. | | any other status (not `processing`/`succeeded`/`failed`) | Raised as an "unexpected task status" error naming the task id. Treat as transient — this is a backend state this tool doesn't otherwise special-case — and call `get_sfx_task(task_id)` again shortly. | ## Workflow Tips diff --git a/tests/tool_surface.json b/tests/tool_surface.json index 1ac2a65..d97aa7d 100644 --- a/tests/tool_surface.json +++ b/tests/tool_surface.json @@ -13,7 +13,8 @@ "_generated": { "local_from": "sonilo-mcp 0.17.0 (PyPI)", "hosted_from": "sonilo-api-dashboard backend/app/mcp_server.py", - "date": "2026-08-16" + "date": "2026-08-17", + "_note": "`stems` is hosted-only for now: the deployed hosted server has it on text_to_music and video_to_music, the published sonilo-mcp 0.17.0 does not yet. When the next sonilo-mcp release ships it, --refresh will pick it up here." }, "hosted": { "get_account_services": [], @@ -32,7 +33,8 @@ "prompt", "duration", "output_format", - "variants_num" + "variants_num", + "stems" ], "video_to_music": [ "video_url", @@ -41,7 +43,8 @@ "output_format", "ducking", "variants_num", - "prompt_influence" + "prompt_influence", + "stems" ], "video_to_sfx": [ "video_url", @@ -411,7 +414,8 @@ "preserve_speech", "ducking", "variants_num", - "prompt_influence" + "prompt_influence", + "stems" ], "video_to_video_music": [ "keep_original_sound", @@ -431,7 +435,8 @@ "variants_num" ], "text_to_music": [ - "variants_num" + "variants_num", + "stems" ], "dubbing": [ "languages", diff --git a/text-to-music/SKILL.md b/text-to-music/SKILL.md index 6c895af..9a99e39 100644 --- a/text-to-music/SKILL.md +++ b/text-to-music/SKILL.md @@ -86,7 +86,7 @@ curl -X POST "https://api.sonilo.com/v1/text-to-music" \ | Tool | Description | |------|-------------| -| `text_to_music(prompt, duration, output_format?, variants_num?, output_directory?)` | Generate music from a text description only — no video. | +| `text_to_music(prompt, duration, output_format?, variants_num?, stems?, output_directory?)` | Generate music from a text description only — no video. | ## Parameters @@ -96,8 +96,49 @@ curl -X POST "https://api.sonilo.com/v1/text-to-music" \ | `duration` | int | — | Required. 1–360 seconds. Unlike the video tools, there is no source to take the length from, so you must set it. | | `variants_num` | int | `1` | 1–10. Generates that many distinct creative directions in one request — different takes, not re-renders of one. **Cost scales linearly with the count, and any value above 1 is never covered by the free trial**, so confirm the number with the user first. Above 1 writes one file per variant and forces the backend's async mode. | | `output_format` | string | `m4a` | `m4a` or `wav`. `wav` triggers the backend's async mode internally — no user-facing "mode" param needed. | +| `stems` | bool | `false` | **Free.** Additionally splits each generated track into four separated instrument tracks — `drums`, `bass`, `vocals`, `other` — returned alongside the untouched full mix. Async-only on REST (`stems=true` without `mode=async` is a `400`). Live on REST and the **hosted** MCP server today; the local `sonilo-mcp` package does not accept it yet. See [Stems](#stems). | | `output_directory` | string | `SONILO_MCP_BASE_PATH` | Absolute, or relative to the base path. | +## Stems + +`stems=true` additionally returns each generated track split into four +separated instrument tracks — `drums`, `bass`, `vocals`, `other` — **free of +charge**. The full mix is untouched; the stems arrive alongside it in the task +result as a `stems` array next to `audio`: + +```json +"stems": [ + { + "stream_index": 0, + "drums": { "url": "…", "content_type": "audio/mp4", "file_size": 2913044 }, + "bass": { "url": "…", "content_type": "audio/mp4", "file_size": 2870211 }, + "vocals": { "url": "…", "content_type": "audio/mp4", "file_size": 2794560 }, + "other": { "url": "…", "content_type": "audio/mp4", "file_size": 3011830 } + } +] +``` + +What matters when you use it: + +- **Async only on REST.** `stems=true` requires `mode=async` (a `400` otherwise): you get a `202` + `task_id` and poll `/v1/tasks/{task_id}`. The MCP tools are always async, so on the hosted server the param just works. +- **Surfaces today: REST and the hosted MCP server.** The local `sonilo-mcp` package does not accept `stems` yet (its next release adds it), and the SDKs and CLIs don't expose it yet either — use the REST call or the hosted tool until they do. +- **Match stems to tracks by `stream_index`, never by array position.** A stream whose separation failed is simply absent, so `stems` can be shorter than `audio`. +- **`stems_error` is not a failed generation.** When separation failed wholly or partly, or was skipped, the task carries a `stems_error` string — possibly *alongside* a partial `stems` array. The generation itself succeeded and every `audio` URL is valid: treat missing stems as a missing extra, never as a reason to retry or refund. +- **Timing:** separation runs after generation finishes — typically another 2–6 min, giving up after 30 min (then `stems_error`). +- **The four stem names are fixed** (htdemucs separation): melodic instruments — piano, synths, guitar, strings — land in `other`, and on instrumental tracks `vocals` is near-silent. That is correct behavior, not a bug. +- **Formats:** stems normally follow `output_format`; trust each stem's `content_type` for what was actually delivered. + +```bash +# REST: submit with stems, then poll the task +curl -X POST "https://api.sonilo.com/v1/text-to-music" \ + -H "Authorization: Bearer $SONILO_API_KEY" \ + --data-urlencode "prompt=A chill lo-fi hip hop beat with jazzy piano chords" \ + --data-urlencode "duration=30" \ + --data-urlencode "mode=async" \ + --data-urlencode "stems=true" +# → {"task_id": "…"} — poll GET /v1/tasks/{task_id} for audio + stems +``` + ## Prompting The prompt is the only input — there is no footage to lean on. Describe genre, @@ -117,12 +158,13 @@ auto-refund, but your own retry is a new charge. - **If the user has a finished video, you are in the wrong skill.** [video-to-music](../video-to-music) syncs to the actual cut instead of producing a generic track of matching length. - **Duration is required here.** Don't guess it — ask if the user hasn't said. - **Several takes in one go:** `variants_num=3` returns three distinct directions for one request instead of three re-rolls. It costs 3×, and it is never free-trial covered — say the price before calling. +- **User wants the track's instruments as separate files** (to remix, re-balance, or drop one)? `stems=true` — it's free, but async-only and not on every surface yet; see [Stems](#stems). - **Content restriction:** prompts cannot reference specific artists, bands, or copyrighted lyrics. ## Recovering a Timed-Out Call -`text_to_music` streams its result in one call unless `output_format="wav"` or -`variants_num` above 1 triggers the backend's async mode. If an async variant +`text_to_music` streams its result in one call unless `output_format="wav"`, +`variants_num` above 1, or `stems=true` triggers the backend's async mode. If an async variant times out, the error message includes a `task_id`; the generation keeps running (and is already charged) on the backend. Call `get_sfx_task(task_id)` — `get_generation_task(task_id)` on the hosted server — to retrieve the result; diff --git a/video-to-music/SKILL.md b/video-to-music/SKILL.md index 32034c9..b5f16be 100644 --- a/video-to-music/SKILL.md +++ b/video-to-music/SKILL.md @@ -107,7 +107,7 @@ curl -X POST "https://api.sonilo.com/v1/video-to-music" \ | Tool | Description | |------|-------------| -| `video_to_music(video_path? \| video_url?, prompt?, preserve_speech?, output_format?, ducking?, variants_num?, prompt_influence?, output_directory?)` | Score a video: matches pacing/motion/emotion, matches the video's duration exactly. Returns audio only (the video itself is not muxed). | +| `video_to_music(video_path? \| video_url?, prompt?, preserve_speech?, output_format?, ducking?, variants_num?, prompt_influence?, stems?, output_directory?)` | Score a video: matches pacing/motion/emotion, matches the video's duration exactly. Returns audio only (the video itself is not muxed). | | `video_to_video_music(video_path? \| video_url?, prompt?, segments?, keep_original_sound?, ducking?, preserve_speech?, variants_num?, prompt_influence?, output_directory?)` | Same scoring, but returns a **new `.mp4`** with the music already muxed in. **By default the source's own audio is dropped** — see `keep_original_sound`. | ## Parameters @@ -122,9 +122,40 @@ curl -X POST "https://api.sonilo.com/v1/video-to-music" \ | `keep_original_sound` | bool | `false` | `video_to_video_music` only. **This is the parameter to reach for when the result sounds wrong.** By default the returned `.mp4` carries the generated music *alone* — the source's dialogue, room tone, and effects are gone. Set `true` to keep the whole source track with the music mixed under it, and add `ducking=true` to dip the music under the voice rather than mixing it flat. `keep_original_sound` supersedes `preserve_speech`. | | `variants_num` | int | `1` | 1–10. Generates that many distinct creative directions in one request — different takes, not re-renders of one. **Cost scales linearly with the count, and any value above 1 is never covered by the free trial**, so confirm the number with the user first. Above 1 writes one file per variant and forces the backend's async mode. | | `prompt_influence` | float \| null | API default `0.5` | 0–1: how strictly the music follows your prompt versus what the video itself suggests. Lower lets the footage lead, higher enforces the brief. **Free**, and it does not change the mode or the number of files — omit it unless the user asks for stricter or looser adherence. | +| `stems` | bool | `false` | `video_to_music` only — `video_to_video_music` has no such param. **Free.** Additionally splits each **generated** track into four separated instrument tracks — `drums`, `bass`, `vocals`, `other` — returned alongside the untouched full mix. It never touches the video's own audio. Async-only on REST (`stems=true` without `mode=async` is a `400`). Live on REST and the **hosted** MCP server today; the local `sonilo-mcp` package does not accept it yet. See [Stems](#stems). | | `output_format` | string | `m4a` | `video_to_music` only — `video_to_video_music` has no such param and always outputs a muxed `.mp4`. `m4a` or `wav`. `wav` (and `preserve_speech`/`ducking`) triggers the backend's async mode internally. | | `output_directory` | string | `SONILO_MCP_BASE_PATH` | Absolute, or relative to the base path. | +## Stems + +`stems=true` on `video_to_music` additionally returns each generated track +split into four separated instrument tracks — `drums`, `bass`, `vocals`, +`other` — **free of charge**. The full mix is untouched; the stems arrive +alongside it in the task result as a `stems` array next to `audio`: + +```json +"stems": [ + { + "stream_index": 0, + "drums": { "url": "…", "content_type": "audio/mp4", "file_size": 2913044 }, + "bass": { "url": "…", "content_type": "audio/mp4", "file_size": 2870211 }, + "vocals": { "url": "…", "content_type": "audio/mp4", "file_size": 2794560 }, + "other": { "url": "…", "content_type": "audio/mp4", "file_size": 3011830 } + } +] +``` + +What matters when you use it: + +- **It splits the GENERATED music, never the video's own audio.** The `vocals` stem is whatever singing the generated score contains — usually near-silent, since scores are mostly instrumental, and that is correct behavior, not a bug. To get the *source* speech isolated, use `preserve_speech` (a different feature) instead. +- **Async only on REST.** `stems=true` requires `mode=async` (a `400` otherwise): you get a `202` + `task_id` and poll `/v1/tasks/{task_id}`. The MCP tools are always async, so on the hosted server the param just works. +- **Surfaces today: REST and the hosted MCP server.** The local `sonilo-mcp` package does not accept `stems` yet (its next release adds it), and the SDKs and CLIs don't expose it yet either — use the REST call or the hosted tool until they do. +- **Match stems to tracks by `stream_index`, never by array position.** A stream whose separation failed is simply absent, so `stems` can be shorter than `audio`. +- **`stems_error` is not a failed generation.** When separation failed wholly or partly, or was skipped, the task carries a `stems_error` string — possibly *alongside* a partial `stems` array. The generation itself succeeded and every `audio` URL is valid: treat missing stems as a missing extra, never as a reason to retry or refund. +- **Timing:** separation runs after generation finishes — typically another 2–6 min, giving up after 30 min (then `stems_error`). +- **The four stem names are fixed** (htdemucs separation): melodic instruments — piano, synths, guitar, strings — land in `other`. +- **Formats:** stems normally follow `output_format`; trust each stem's `content_type` for what was actually delivered. + ## Prompting No prompt is required — the model reads the cut. A short structured brief adds @@ -145,6 +176,7 @@ generate once, iterate on the prompt, not on rerolls. - **`preserve_speech` for talking-head or narrated video:** if the source has dialogue/voiceover the user wants kept, set `preserve_speech=true`. Behavior differs by tool: on `video_to_music` you get the music, the isolated speech stem, *and* a ready-mixed combined file (the mux) — use the mux directly rather than re-mixing yourself. On `video_to_video_music` there's no separate stem or mux file; it just keeps the source speech audible in the single muxed output video. - **Want the video back with the score baked in?** Use `video_to_video_music` instead of `video_to_music` — same inputs, but the output is a new `.mp4`, not just audio. **Warn the user that the source audio is dropped by default**: if their video has dialogue, narration, or effects they expect to hear, pass `keep_original_sound=true` (add `ducking=true` to keep the voice on top), or `preserve_speech=true` for the isolated speech only. A "the music is there but my voiceover vanished" report is always this. - **Several takes in one go:** `variants_num=3` returns three distinct directions for one request instead of three re-rolls. It costs 3×, and it is never free-trial covered — say the price before calling. +- **User wants the score's instruments as separate files** (to remix, re-balance, or drop one)? `stems=true` on `video_to_music` — it's free, but async-only and not on every surface yet, and it splits the generated music, never the source audio; see [Stems](#stems). - **Duration:** always matched to the source video automatically — don't ask for it. - **Need SFX too?** To generate music **and** sound effects for the same video in one balanced, single-charge call, use [video-to-sound](../video-to-sound) rather than calling this and [video-to-sfx](../video-to-sfx) separately. - **Don't know what it should sound like?** Run [video-analysis](../video-analysis) first: one call returns a section plan plus ready-to-use generation prompts read off the footage, which beats guessing a prompt and rerolling. It is a paid call that generates nothing, so use it when the brief is genuinely unclear — not when the user already told you what they want. @@ -152,7 +184,7 @@ generate once, iterate on the prompt, not on rerolls. ## Recovering a Timed-Out Call -`video_to_music` (without any of `preserve_speech`/`ducking`/`output_format="wav"`) streams its result in one call. Any variant that triggers the backend's async mode — and `video_to_video_music`, which is always async — can time out on a very long `TIME_OUT_SECONDS`. If it does, the error message includes a `task_id`; the generation keeps running (and is already charged) on the backend. Call `get_sfx_task(task_id)` — `get_generation_task(task_id)` on the hosted server — to retrieve the result once ready; see the [task-recovery](../task-recovery) skill. +`video_to_music` (without any of `preserve_speech`/`ducking`/`stems`/`output_format="wav"`) streams its result in one call. Any variant that triggers the backend's async mode — and `video_to_video_music`, which is always async — can time out on a very long `TIME_OUT_SECONDS`. If it does, the error message includes a `task_id`; the generation keeps running (and is already charged) on the backend. Call `get_sfx_task(task_id)` — `get_generation_task(task_id)` on the hosted server — to retrieve the result once ready; see the [task-recovery](../task-recovery) skill. ## Output Files From 9b232abe51d3238ae1ace26e024bce926d56f148 Mon Sep 17 00:00:00 2001 From: Spencer Qian Date: Mon, 17 Aug 2026 14:22:00 -0700 Subject: [PATCH 2/2] Close the stems surface gap: every client now ships it sonilo-mcp 0.18.0 (PyPI + registry), npm sonilo 0.16.0 / sonilo-cli 0.15.0, and PyPI sonilo 0.15.0 / sonilo-cli 0.14.0 all released today with stems support. tool_surface.json refreshed against the published 0.18.0; the two skills drop their hosted-only caveats and now state the minimum versions per surface; api-claims records the gap as closed. --- references/api-claims.md | 2 +- tests/tool_surface.json | 14 ++++++++------ text-to-music/SKILL.md | 4 ++-- video-to-music/SKILL.md | 4 ++-- 4 files changed, 13 insertions(+), 11 deletions(-) diff --git a/references/api-claims.md b/references/api-claims.md index c9fc10b..3f5bdce 100644 --- a/references/api-claims.md +++ b/references/api-claims.md @@ -48,7 +48,7 @@ Added 2026-08-17, verified against the shipped backend (live on REST `/v1/text-t - [x] Separation runs after generation: typically **+2–6 min**, gives up after **30 min**. Stems normally follow `output_format`; each stem's `content_type` reports what was delivered. - [x] On `video_to_music` it splits the **generated** music, never the video's own audio (source speech = `preserve_speech`, unrelated). - [x] The four stem names are fixed (htdemucs): melodic instruments land in `other`; on instrumental tracks `vocals` is near-silent — correct behavior, not a bug. -- [x] ⚠️ **Surface gap as of 2026-08-17**: the local `sonilo-mcp` package (0.17.0) does not accept `stems` yet, and neither do the SDKs or CLIs — hosted MCP and REST only. Re-verify when the next sonilo-mcp release lands and update the two skills' surface notes plus `tests/tool_surface.json` (`--refresh` picks up the local block). +- [x] **Surface gap closed 2026-08-17** (same day): sonilo-mcp 0.18.0, npm sonilo 0.16.0 / sonilo-cli 0.15.0, and PyPI sonilo 0.15.0 / sonilo-cli 0.14.0 all ship `stems`; `tests/tool_surface.json` refreshed against the published 0.18.0. Every surface now accepts it. ## Billing / general diff --git a/tests/tool_surface.json b/tests/tool_surface.json index d97aa7d..890a12e 100644 --- a/tests/tool_surface.json +++ b/tests/tool_surface.json @@ -108,6 +108,7 @@ "duration", "output_format", "variants_num", + "stems", "output_directory" ], "video_to_music": [ @@ -119,6 +120,7 @@ "ducking", "variants_num", "prompt_influence", + "stems", "output_directory" ], "text_to_sfx": [ @@ -189,6 +191,12 @@ "task_id", "output_directory" ], + "analyze_video": [ + "video_path", + "video_url", + "prompt", + "variants_num" + ], "audio_ducking": [ "voice_path", "voice_url", @@ -202,12 +210,6 @@ ], "play_audio": [ "input_file_path" - ], - "analyze_video": [ - "video_path", - "video_url", - "prompt", - "variants_num" ] }, "cli": { diff --git a/text-to-music/SKILL.md b/text-to-music/SKILL.md index 9a99e39..807f873 100644 --- a/text-to-music/SKILL.md +++ b/text-to-music/SKILL.md @@ -96,7 +96,7 @@ curl -X POST "https://api.sonilo.com/v1/text-to-music" \ | `duration` | int | — | Required. 1–360 seconds. Unlike the video tools, there is no source to take the length from, so you must set it. | | `variants_num` | int | `1` | 1–10. Generates that many distinct creative directions in one request — different takes, not re-renders of one. **Cost scales linearly with the count, and any value above 1 is never covered by the free trial**, so confirm the number with the user first. Above 1 writes one file per variant and forces the backend's async mode. | | `output_format` | string | `m4a` | `m4a` or `wav`. `wav` triggers the backend's async mode internally — no user-facing "mode" param needed. | -| `stems` | bool | `false` | **Free.** Additionally splits each generated track into four separated instrument tracks — `drums`, `bass`, `vocals`, `other` — returned alongside the untouched full mix. Async-only on REST (`stems=true` without `mode=async` is a `400`). Live on REST and the **hosted** MCP server today; the local `sonilo-mcp` package does not accept it yet. See [Stems](#stems). | +| `stems` | bool | `false` | **Free.** Additionally splits each generated track into four separated instrument tracks — `drums`, `bass`, `vocals`, `other` — returned alongside the untouched full mix. Async-only on REST (`stems=true` without `mode=async` is a `400`). See [Stems](#stems). | | `output_directory` | string | `SONILO_MCP_BASE_PATH` | Absolute, or relative to the base path. | ## Stems @@ -121,7 +121,7 @@ result as a `stems` array next to `audio`: What matters when you use it: - **Async only on REST.** `stems=true` requires `mode=async` (a `400` otherwise): you get a `202` + `task_id` and poll `/v1/tasks/{task_id}`. The MCP tools are always async, so on the hosted server the param just works. -- **Surfaces today: REST and the hosted MCP server.** The local `sonilo-mcp` package does not accept `stems` yet (its next release adds it), and the SDKs and CLIs don't expose it yet either — use the REST call or the hosted tool until they do. +- **Available on every surface** (verified 2026-08-17): REST, the hosted MCP server, the local `sonilo-mcp` package (>= 0.18.0), the SDKs (`sonilo` npm >= 0.16.0, PyPI >= 0.15.0), and the CLIs (`--stems`, npm `sonilo-cli` >= 0.15.0, PyPI `sonilo-cli` >= 0.14.0). - **Match stems to tracks by `stream_index`, never by array position.** A stream whose separation failed is simply absent, so `stems` can be shorter than `audio`. - **`stems_error` is not a failed generation.** When separation failed wholly or partly, or was skipped, the task carries a `stems_error` string — possibly *alongside* a partial `stems` array. The generation itself succeeded and every `audio` URL is valid: treat missing stems as a missing extra, never as a reason to retry or refund. - **Timing:** separation runs after generation finishes — typically another 2–6 min, giving up after 30 min (then `stems_error`). diff --git a/video-to-music/SKILL.md b/video-to-music/SKILL.md index b5f16be..f4df296 100644 --- a/video-to-music/SKILL.md +++ b/video-to-music/SKILL.md @@ -122,7 +122,7 @@ curl -X POST "https://api.sonilo.com/v1/video-to-music" \ | `keep_original_sound` | bool | `false` | `video_to_video_music` only. **This is the parameter to reach for when the result sounds wrong.** By default the returned `.mp4` carries the generated music *alone* — the source's dialogue, room tone, and effects are gone. Set `true` to keep the whole source track with the music mixed under it, and add `ducking=true` to dip the music under the voice rather than mixing it flat. `keep_original_sound` supersedes `preserve_speech`. | | `variants_num` | int | `1` | 1–10. Generates that many distinct creative directions in one request — different takes, not re-renders of one. **Cost scales linearly with the count, and any value above 1 is never covered by the free trial**, so confirm the number with the user first. Above 1 writes one file per variant and forces the backend's async mode. | | `prompt_influence` | float \| null | API default `0.5` | 0–1: how strictly the music follows your prompt versus what the video itself suggests. Lower lets the footage lead, higher enforces the brief. **Free**, and it does not change the mode or the number of files — omit it unless the user asks for stricter or looser adherence. | -| `stems` | bool | `false` | `video_to_music` only — `video_to_video_music` has no such param. **Free.** Additionally splits each **generated** track into four separated instrument tracks — `drums`, `bass`, `vocals`, `other` — returned alongside the untouched full mix. It never touches the video's own audio. Async-only on REST (`stems=true` without `mode=async` is a `400`). Live on REST and the **hosted** MCP server today; the local `sonilo-mcp` package does not accept it yet. See [Stems](#stems). | +| `stems` | bool | `false` | `video_to_music` only — `video_to_video_music` has no such param. **Free.** Additionally splits each **generated** track into four separated instrument tracks — `drums`, `bass`, `vocals`, `other` — returned alongside the untouched full mix. It never touches the video's own audio. Async-only on REST (`stems=true` without `mode=async` is a `400`). See [Stems](#stems). | | `output_format` | string | `m4a` | `video_to_music` only — `video_to_video_music` has no such param and always outputs a muxed `.mp4`. `m4a` or `wav`. `wav` (and `preserve_speech`/`ducking`) triggers the backend's async mode internally. | | `output_directory` | string | `SONILO_MCP_BASE_PATH` | Absolute, or relative to the base path. | @@ -149,7 +149,7 @@ What matters when you use it: - **It splits the GENERATED music, never the video's own audio.** The `vocals` stem is whatever singing the generated score contains — usually near-silent, since scores are mostly instrumental, and that is correct behavior, not a bug. To get the *source* speech isolated, use `preserve_speech` (a different feature) instead. - **Async only on REST.** `stems=true` requires `mode=async` (a `400` otherwise): you get a `202` + `task_id` and poll `/v1/tasks/{task_id}`. The MCP tools are always async, so on the hosted server the param just works. -- **Surfaces today: REST and the hosted MCP server.** The local `sonilo-mcp` package does not accept `stems` yet (its next release adds it), and the SDKs and CLIs don't expose it yet either — use the REST call or the hosted tool until they do. +- **Available on every surface** (verified 2026-08-17): REST, the hosted MCP server, the local `sonilo-mcp` package (>= 0.18.0), the SDKs (`sonilo` npm >= 0.16.0, PyPI >= 0.15.0), and the CLIs (`--stems`, npm `sonilo-cli` >= 0.15.0, PyPI `sonilo-cli` >= 0.14.0). - **Match stems to tracks by `stream_index`, never by array position.** A stream whose separation failed is simply absent, so `stems` can be shorter than `audio`. - **`stems_error` is not a failed generation.** When separation failed wholly or partly, or was skipped, the task carries a `stems_error` string — possibly *alongside* a partial `stems` array. The generation itself succeeded and every `audio` URL is valid: treat missing stems as a missing extra, never as a reason to retry or refund. - **Timing:** separation runs after generation finishes — typically another 2–6 min, giving up after 30 min (then `stems_error`).