Lead/backing vocal split, stems relocation fixes, eager model pre-download

.github/workflows/ci.yml

@@ -26,6 +26,13 @@ jobs:
       image: ghcr.io/astral-sh/uv:python3.12-bookworm-slim
     steps:
       - uses: actions/checkout@v7.0.1
+      # diffq (a mandatory transitive dependency of audio-separator, #275) has
+      # no prebuilt wheel for Python 3.11+ on Linux -- its last release only
+      # ever shipped cp310 wheels -- so uv sync must compile it from source.
+      # Docker and the Linux desktop release build already install
+      # build-essential for the same reason; this container didn't need it
+      # before audio-separator existed.
+      - run: apt-get update && apt-get install -y --no-install-recommends build-essential
       - run: uv sync --frozen --all-extras
       - run: uv run ruff check app/ tests/
       - run: uv run ruff format --check app/ tests/
@@ -37,7 +44,8 @@ jobs:
       image: ghcr.io/astral-sh/uv:python3.12-bookworm-slim
     steps:
       - uses: actions/checkout@v7.0.1
-      - run: apt-get update && apt-get install -y --no-install-recommends ffmpeg
+      # build-essential: see the matching comment in the lint job above.
+      - run: apt-get update && apt-get install -y --no-install-recommends ffmpeg build-essential
       - run: uv sync --frozen --all-extras
       - run: uv run pytest tests/ -q
 

.gitignore

@@ -46,6 +46,7 @@ htmlcov/
 data/
 jobs/
 cache/
+models/
 settings.json
 .run/
 .build

README.md

@@ -53,14 +53,15 @@ StemDeck is free and **does not accept any money, sponsorship, or funding** - no
 
 | Name | What they do | Link |
 |---|---|---|
+| Analog4Lyfe | Analog music gear | [@analog4lyfe](https://www.instagram.com/analog4lyfe) |
 | Dlima Guitars | Custom guitars and basses | [@dlimaguitars](https://www.instagram.com/dlimaguitars) |
-| Lisbon Guitar Works | Guitar building | [dlimaguitars.com](https://dlimaguitars.com) |
+| Empress Effects | Effects pedals | [empresseffects.com](https://empresseffects.com) |
 | Joao Gaspar | Producer/Film Scorer, Touring/Session Musician | [@jay_glaspar](https://www.instagram.com/jay_glaspar) |
 | Kris Luthier | Luthier and Musical Instrument Repair, Lisboa | [@krisluthier](https://www.instagram.com/krisluthier) |
-| Thomann | Online Music Store | [@thomann.music](https://www.instagram.com/thomann.music) |
-| Analog4Lyfe | Analog music gear | [@analog4lyfe](https://www.instagram.com/analog4lyfe) |
-| Empress Effects | Effects pedals | [empresseffects.com](https://empresseffects.com) |
+| Lisbon Guitar Works | Guitar building | [dlimaguitars.com](https://dlimaguitars.com) |
 | More Notes Less Talk | Instruments and gear with personality, recorded raw to tape. No hype, no gatekeeping. | [@morenoteslesstalk](https://www.youtube.com/@morenoteslesstalk) |
+| Seratone | Turns any TV into a studio-grade karaoke stage | [seratone.audio](https://seratone.audio/) |
+| Thomann | Online Music Store | [@thomann.music](https://www.instagram.com/thomann.music) |
 
 
 ---
@@ -153,7 +154,7 @@ Extract the zip anywhere, run `StemDeck.exe`. FFmpeg, the Demucs model, config,
 
 <br>
 
-StemDeck is built on **[Python 3.12](https://python.org)** managed via **[uv](https://github.com/astral-sh/uv)**, with a **[FastAPI](https://fastapi.tiangolo.com)** backend serving REST and Server-Sent Events. Stem separation uses **[Demucs](https://github.com/facebookresearch/demucs)** (`htdemucs_6s`), Meta AI's open-source 6-stem neural network. YouTube audio is fetched via **[yt-dlp](https://github.com/yt-dlp/yt-dlp)**; transcoding and mixing use **[FFmpeg](https://ffmpeg.org)**. BPM detection and key analysis run on **[librosa](https://librosa.org)**; loudness measurement uses **[pyloudnorm](https://github.com/csteinmetz1/pyloudnorm)** (ITU-R BS.1770). The macOS and Windows desktop shells are **[Tauri v2](https://tauri.app)** (Rust/WKWebView on macOS, Rust/WebView2 on Windows). The frontend is vanilla JS with the Web Audio API, no framework and no build step; waveforms are rendered on `<canvas>` using min/max sample rendering.
+StemDeck is built on **[Python 3.12](https://python.org)** managed via **[uv](https://github.com/astral-sh/uv)**, with a **[FastAPI](https://fastapi.tiangolo.com)** backend serving REST and Server-Sent Events. Stem separation uses **[Demucs](https://github.com/facebookresearch/demucs)** (`htdemucs_6s`), Meta AI's open-source 6-stem neural network. The optional on-demand lead/backing vocal split runs the UVR-MDX-NET Karaoke 2 model via **[audio-separator](https://github.com/nomadkaraoke/python-audio-separator)**, trained as part of the **[Ultimate Vocal Remover](https://github.com/Anjok07/ultimatevocalremovergui)** project by Anjok07. YouTube audio is fetched via **[yt-dlp](https://github.com/yt-dlp/yt-dlp)**; transcoding and mixing use **[FFmpeg](https://ffmpeg.org)**. BPM detection and key analysis run on **[librosa](https://librosa.org)**; loudness measurement uses **[pyloudnorm](https://github.com/csteinmetz1/pyloudnorm)** (ITU-R BS.1770). The macOS and Windows desktop shells are **[Tauri v2](https://tauri.app)** (Rust/WKWebView on macOS, Rust/WebView2 on Windows). The frontend is vanilla JS with the Web Audio API, no framework and no build step; waveforms are rendered on `<canvas>` using min/max sample rendering.
 
 *Thanks to the creators and maintainers of all the open-source libraries that make StemDeck possible.*
 

app/api/config.py

@@ -2,11 +2,16 @@
 
 from fastapi import APIRouter
 
-from app.core.config import STEM_NAMES
+from app.core.config import EXTRA_STEM_NAMES, STEM_NAMES
 
 router = APIRouter()
 
 
 @router.get("/config")
 def get_config() -> dict:
-    return {"stem_names": list(STEM_NAMES)}
+    # extra_stem_names (#275) are produced only when a job's on-demand
+    # lead/backing vocal split has run -- kept separate from stem_names so
+    # existing clients that assume "every job produces exactly these stems"
+    # are unaffected; new clients merge it into their lane vocab (see
+    # syncStemNamesFromAPI in static/js/constants.js).
+    return {"stem_names": list(STEM_NAMES), "extra_stem_names": list(EXTRA_STEM_NAMES)}

app/api/jobs.py

@@ -33,7 +33,11 @@
 from app.core.settings import get_max_duration_sec
 from app.core.stems_location import is_relocating
 from app.pipeline import jobqueue
+from app.pipeline.collect import merge_stem_peaks
 from app.pipeline.download import InvalidYouTubeURL, validate_youtube_url
+from app.pipeline.errors import classify_failure
+from app.pipeline.runner import _pipeline_lock
+from app.pipeline.vocal_split import split_vocals
 
 router = APIRouter(tags=["jobs"])
 logger = logging.getLogger("stemdeck.api")
@@ -328,6 +332,63 @@ def cancel_job(job_id: str) -> dict:
     return job.to_state()
 
 
+def _write_vocal_split_error(stems_dir: Path, cause: str, tail: list[str]) -> None:
+    """Best-effort error record for the on-demand vocal split (#275). The job
+    itself stays "done" -- this is diagnostic-only, not the quarantine path
+    (which would delete the job's base stems)."""
+    try:
+        lines = [f"cause: {cause}", "", "--- stderr tail ---", *tail]
+        (stems_dir / "vocal_split_error.txt").write_text("\n".join(lines) + "\n", encoding="utf-8")
+    except OSError:
+        logger.warning("could not write vocal_split_error.txt in %s", stems_dir, exc_info=True)
+
+
+@router.post("/{job_id}/vocal-split")
+async def start_vocal_split(job_id: str) -> Response:
+    """Trigger the on-demand lead/backing vocal split (#275) for a completed
+    job: a second model pass over the existing vocals.wav, producing
+    lead_vocals.wav + backing_vocals.wav. Idempotent once done -- calling
+    again returns 202 with the existing result rather than re-running the
+    (expensive) model."""
+    if not JOB_ID_RE.match(job_id):
+        raise HTTPException(status_code=404, detail="job not found")
+    job = registry_get(job_id)
+    if job is None or job.status != "done":
+        raise HTTPException(status_code=404, detail="job not found")
+    if job.vocal_split == "running":
+        raise HTTPException(status_code=409, detail="vocal split already running")
+    if job.vocal_split == "done":
+        return JSONResponse(_job_state(job), status_code=202)
+
+    stems_dir = (JOBS_DIR / job_id / "stems").resolve()
+    if not stems_dir.is_relative_to(JOBS_DIR.resolve()):
+        raise HTTPException(status_code=404, detail="job not found")
+
+    job.vocal_split = "running"
+    _set(job, stage="Splitting lead/backing vocals...")
+    try:
+        async with _pipeline_lock:
+            new_names = await asyncio.to_thread(split_vocals, job, stems_dir)
+    except Exception as e:
+        cause = classify_failure("\n".join([*(getattr(e, "tail", None) or []), str(e)]))
+        logger.warning("[%s] vocal split failed: %s", job_id, e, exc_info=True)
+        _write_vocal_split_error(stems_dir, cause, getattr(e, "tail", None) or [str(e)])
+        job.vocal_split = "error"
+        _set(job, stage="Done")
+        registry_persist(JOBS_DIR)
+        raise HTTPException(status_code=500, detail="vocal split failed") from e
+
+    existing = {s["name"] for s in job.stems}
+    for name in new_names:
+        if name not in existing:
+            job.stems.append({"name": name, "url": f"/api/jobs/{job_id}/stems/{name}.wav"})
+    merge_stem_peaks(stems_dir, new_names)
+    job.vocal_split = "done"
+    _set(job, stage="Done")
+    registry_persist(JOBS_DIR)
+    return JSONResponse(_job_state(job))
+
+
 _SECTION_ID_RE = re.compile(r"^[a-zA-Z0-9_\-]{1,64}$")
 _COLOR_RE = re.compile(r"^#[0-9a-fA-F]{3,8}$")
 

app/api/stems.py

@@ -20,6 +20,7 @@
 
 from app.core.config import (
     CACHE_DIR,
+    EXTRA_STEM_NAMES,
     JOB_ID_RE,
     JOBS_DIR,
     STEM_NAMES,
@@ -36,17 +37,23 @@
 router = APIRouter(tags=["stems"])
 
 # Stem files served by this endpoint: the 6 demucs stems + two
-# pipeline-produced extras. "original" is the re-encoded source song
-# (added when the user picked a strict subset), "mix" is the ffmpeg
-# amix of the user's selected stems.
-_ALLOWED_NAMES = frozenset(STEM_NAMES) | {"original", "mix"}
+# pipeline-produced extras, plus the on-demand lead/backing vocal split
+# (#275, EXTRA_STEM_NAMES) when a job has requested it. "original" is the
+# re-encoded source song (added when the user picked a strict subset), "mix"
+# is the ffmpeg amix of the user's selected stems.
+_ALLOWED_NAMES = frozenset(STEM_NAMES) | frozenset(EXTRA_STEM_NAMES) | {"original", "mix"}
 
 # Lanes the dynamic mixdown may sum: the 6 stems plus "original" (the complement
-# track shown when the user picked a subset). "mix" is excluded -- it is the
-# static pre-render this endpoint replaces. Gains are linear; the studio caps a
-# lane at 2.0, so this generous bound just rejects abusive values.
-_MIXDOWN_NAMES = frozenset(STEM_NAMES) | {"original"}
+# track shown when the user picked a subset) plus lead/backing vocals when a
+# job has split them. "mix" is excluded -- it is the static pre-render this
+# endpoint replaces. Gains are linear; the studio caps a lane at 2.0, so this
+# generous bound just rejects abusive values.
+_MIXDOWN_NAMES = frozenset(STEM_NAMES) | frozenset(EXTRA_STEM_NAMES) | {"original"}
 _MIXDOWN_MAX_GAIN = 4.0
+# lead_vocals/backing_vocals are a decomposition of vocals, not an independent
+# signal -- summing vocals alongside either would double-count the vocal
+# energy in the mix (#275).
+_VOCAL_DECOMPOSITION_NAMES = frozenset(EXTRA_STEM_NAMES)
 
 # Output encoders by container/extension, shared by the dynamic mixdown and the
 # stems zip. WAV is lossless PCM, FLAC is lossless compressed, MP3 is VBR ~190 kbps,
@@ -304,6 +311,11 @@ def _parse_lane_gains(stems: str, gains: str) -> tuple[list[str], list[float]]:
         raise HTTPException(status_code=422, detail="gain out of range")
     if not set(names) <= _MIXDOWN_NAMES:
         raise HTTPException(status_code=422, detail="unknown stem requested")
+    if "vocals" in names and _VOCAL_DECOMPOSITION_NAMES & set(names):
+        raise HTTPException(
+            status_code=422,
+            detail="vocals cannot be combined with lead_vocals/backing_vocals (same signal)",
+        )
     return names, parsed_gains
 
 
@@ -910,13 +922,19 @@ async def get_all_stems_zip(
         raise HTTPException(status_code=404, detail="job not ready")
 
     # Resolve the requested subset (whitelisted) or fall back to all stems.
+    all_names = (*STEM_NAMES, *EXTRA_STEM_NAMES)
     if stems:
         requested = {s for s in stems.split(",") if s}
-        if not requested <= set(STEM_NAMES):
+        if not requested <= set(all_names):
             raise HTTPException(status_code=422, detail="unknown stem requested")
-        wanted = [name for name in STEM_NAMES if name in requested]
+        if "vocals" in requested and _VOCAL_DECOMPOSITION_NAMES & requested:
+            raise HTTPException(
+                status_code=422,
+                detail="vocals cannot be combined with lead_vocals/backing_vocals (same signal)",
+            )
+        wanted = [name for name in all_names if name in requested]
     else:
-        wanted = list(STEM_NAMES)
+        wanted = list(all_names)
 
     jobs_root = JOBS_DIR.resolve()
     stems_dir = (JOBS_DIR / job_id / "stems").resolve()

app/core/config.py

@@ -52,6 +52,12 @@ def detect_torch_device() -> str:
 ROOT = Path(__file__).resolve().parent.parent.parent
 STATIC_DIR = ROOT / "static"
 STEM_NAMES: tuple[str, ...] = ("vocals", "drums", "bass", "guitar", "piano", "other")
+# Produced only when a job opts into the lead/backing vocal split (best-effort,
+# additive -- see app/pipeline/vocal_split.py). Kept out of STEM_NAMES itself:
+# selected_stems defaults, the "Original" complement-track math, and the
+# GET /api/config contract all assume "the 6 Demucs stems" and must not change
+# just because a job happened to request this extra pass.
+EXTRA_STEM_NAMES: tuple[str, ...] = ("lead_vocals", "backing_vocals")
 JOB_ID_RE = re.compile(r"^[a-f0-9]{12}$")
 
 # Runtime knobs -- env-backed so Docker / desktop packaging / local dev can
@@ -144,6 +150,15 @@ def _stored_jobs_dir() -> Path | None:
 TIMEOUT_FFMPEG = _env_int("STEMDECK_TIMEOUT_FFMPEG", 300)
 TIMEOUT_ANALYZE = _env_int("STEMDECK_TIMEOUT_ANALYZE", 120)
 TIMEOUT_DEMUCS_STALL = _env_int("STEMDECK_TIMEOUT_DEMUCS_STALL", 1800)
+# On-demand lead/backing vocal split (#275). UVR-MDX-NET Karaoke 2 is an
+# officially-distributed UVR-project model (MIT + credit-to-UVR per the
+# audio-separator README) -- the default. STEMDECK_KARAOKE_MODEL lets a
+# deployment swap the checkpoint (e.g. to a roformer model) without a code
+# change; see docs/models.md for the license audit behind this default.
+VOCAL_SPLIT_MODEL = os.environ.get("STEMDECK_KARAOKE_MODEL", "").strip() or "UVR_MDXNET_KARA_2.onnx"
+# A first run downloads the checkpoint (hundreds of MB); generous default so a
+# slow connection isn't mistaken for a stall.
+TIMEOUT_VOCAL_SPLIT = _env_int("STEMDECK_TIMEOUT_VOCAL_SPLIT", 1800)
 # Beat-grid stage decodes the whole drums stem (not the 180 s analyze window),
 # so it gets its own, larger budget.
 TIMEOUT_BEATGRID = _env_int("STEMDECK_TIMEOUT_BEATGRID", 300)

app/core/models.py

@@ -76,6 +76,12 @@ class Job:
     # Wall-clock seconds per pipeline stage ({"download": 12.3, ...}); written
     # to metadata.json and the one-line completion summary in the log.
     stage_timings: dict[str, float] | None = None
+    # On-demand lead/backing vocal split (#275) -- a post-hoc action on an
+    # already-"done" job, not part of the main pipeline. "none" until the user
+    # asks for it; "error" leaves the job's base stems untouched (see
+    # app/pipeline/vocal_split.py) and is recorded in stems/vocal_split_error.txt,
+    # not job.error_detail, since the job itself did not fail.
+    vocal_split: Literal["none", "running", "done", "error"] = "none"
     # Set by POST /api/jobs/{id}/cancel; consumed by pipeline stages.
     # Not surfaced via to_state() -- it's internal control state.
     cancel_requested: bool = False
@@ -126,6 +132,7 @@ def to_state(self) -> dict[str, Any]:
             "compute_device": self.compute_device,
             "gpu_fallback": self.gpu_fallback,
             "stage_timings": self.stage_timings,
+            "vocal_split": self.vocal_split,
             "created_at": self.created_at,
         }
 

app/core/registry.py

@@ -8,7 +8,7 @@
 import uuid
 from pathlib import Path
 
-from app.core.config import DEMUCS_MODEL, JOB_ID_RE, STEM_NAMES
+from app.core.config import DEMUCS_MODEL, EXTRA_STEM_NAMES, JOB_ID_RE, STEM_NAMES
 from app.core.models import Job
 
 logger = logging.getLogger("stemdeck.registry")
@@ -237,7 +237,7 @@ def _recover_done_job(job_dir: Path) -> Job | None:
         return None
     stems = [
         {"name": name, "url": f"/api/jobs/{job_dir.name}/stems/{name}.wav"}
-        for name in ("original", *STEM_NAMES)
+        for name in ("original", *STEM_NAMES, *EXTRA_STEM_NAMES)
         if (stems_dir / f"{name}.wav").is_file()
     ]
     if not stems:
@@ -246,6 +246,12 @@ def _recover_done_job(job_dir: Path) -> Job | None:
     if (stems_dir / "mix.wav").is_file():
         mix_url = f"/api/jobs/{job_dir.name}/stems/mix.wav"
     selected = [stem["name"] for stem in stems if stem["name"] in STEM_NAMES] or list(STEM_NAMES)
+    # A restart between the split finishing and its next registry persist
+    # would otherwise report the job as never split -- derive from disk the
+    # same way `stems` above does, so the recovered library entry doesn't
+    # regress (#275).
+    has_split = all((stems_dir / f"{name}.wav").is_file() for name in EXTRA_STEM_NAMES)
+    vocal_split = "done" if has_split else "none"
     meta_path = job_dir / "metadata.json"
     meta: dict = {}
     if meta_path.is_file():
@@ -287,6 +293,7 @@ def _recover_done_job(job_dir: Path) -> Job | None:
         stem_presence=meta.get("stem_presence"),
         sections=meta.get("sections"),
         tags=meta.get("tags"),
+        vocal_split=vocal_split,
     )
 
 

app/core/settings.py

@@ -84,14 +84,20 @@ def _ensure() -> dict:
     return _state
 
 
-def _save() -> None:
+def _save() -> bool:
+    """Write settings.json. Best-effort for most settings (read-only FS,
+    permissions: the in-memory value still applies for this session, so a
+    caller here does not fail its request over it) -- but the write outcome
+    is still reported back, because one caller (set_jobs_dir) is coupled to
+    something irreversible enough that silently swallowing a failure there
+    would be actively misleading rather than merely inconvenient (#403)."""
     try:
         _SETTINGS_PATH.parent.mkdir(parents=True, exist_ok=True)
         _SETTINGS_PATH.write_text(json.dumps(_ensure()), encoding="utf-8")
+        return True
     except Exception:
-        # Persistence is best-effort (read-only FS, permissions): the in-memory
-        # value still applies for this session, so don't fail the request.
         _log.warning("could not persist settings to %s", _SETTINGS_PATH, exc_info=True)
+        return False
 
 
 def _num(v: object) -> int | None:
@@ -138,16 +144,24 @@ def get_jobs_dir() -> str | None:
         return value if isinstance(value, str) and value.strip() else None
 
 
-def set_jobs_dir(value: str | None) -> str | None:
+def set_jobs_dir(value: str | None) -> tuple[str | None, bool]:
+    """Persist the jobs folder choice. Returns (resolved_value, persisted).
+
+    Unlike every other setting in this module, a failed persist here is not a
+    minor inconvenience: this is called only after move_library() has already
+    physically relocated the user's library (POST /api/settings/stems-location
+    in app/main.py), so quietly keeping the in-memory value "for this session"
+    and reporting success would mean the app comes back to the OLD (now-empty)
+    folder on the very next restart, with the real data sitting at a location
+    nothing points at any more (#403). The caller must check `persisted` and
+    tell the user the truth rather than assume a 200 means the choice stuck."""
     with _LOCK:
         if value is None or not str(value).strip():
             _ensure().pop("jobs_dir", None)
-            _save()
-            return None
+            return None, _save()
         resolved = str(Path(str(value)).expanduser().resolve())
         _ensure()["jobs_dir"] = resolved
-        _save()
-        return resolved
+        return resolved, _save()
 
 
 # ── playlist_max_items ──