Replace NC MuQ/Essentia with CLAP + Short-chunk CNN

[gabrielemidulla]

Summary

• Swap OpenMuQ/MuQ-MuLan-large (CC BY-NC weights) for laion/larger_clap_music (Apache 2.0, 512-d, 48 kHz) — no new packages beyond existing transformers.
• Replace Essentia Discogs/Jamendo heads + essentia-tensorflow (AGPL) with MIT Short-chunk CNN (Jamendo top-50), vendored model definition + downloadable weights.
• Version profile Qdrant collections by tagger identity so MiniLM profile vectors do not mix across tagger eras; audio collections version via CLAP provider name/version (old MuQ collections left intact for rollback).

Eval (Precision@6 / Recall@6 / MRR)

Relevance for calm-ambient frozen from MuQ-era collection tags (777 tracks). Catalog has 1300 ready tracks.

Query	Metric	Before (MuQ)	After (CLAP)
calm-ambient	P@6	1.0	1.0
calm-ambient	R@6	0.0077	0.0077
calm-ambient	MRR	1.0	1.0
absent-concept (opera aria, expect_none)	P@6	0.0 (6 hits)	0.0 (6 hits)
absent-concept	MRR	0.0	0.0

Notes:

• Semantic recall for calm/ambient stays strong (all top-6 in the frozen ambient-like set; MRR 1.0).
• expect_none already failed under MuQ (standout filter still returns 6 tracks). CLAP is not worse on this probe, but also not better — watch false positives; opera FPs after reindex are emotional indie/popfolk, not classical opera.
• Fine-grained genre regression is real: Discogs400 (~400 styles) → Jamendo top-50 genres (~30 unique genre keys in catalog). Coarse labels like Ambient / Popfolk / Pop dominate. Escalation path (not in this PR): train a head on CLAP embeddings.

Artifacts: apps/ml-worker/evals/results_{before,after}.json.

Collections after reindex

Collection	Points	Status
ox1audio_laion-clap-music_laion_larger_clap_music	9780	new
…MiniLM…__short-chunk-cnn-jamendo-top50	1300	new
ox1audio_muq-mulan-large_OpenMuQ_MuQ-MuLan-large	9780	untouched (rollback)
…MiniLM-L6-v2 (old profile)	1300	untouched (rollback)

Qdrant volume snapshot / restore

Snapshot taken before reindex:

docker run --rm -v ox1audio_qdrant_data:/from -v "$PWD/backups":/to alpine \
  tar czf /to/qdrant_data_premigration.tar.gz -C /from .

Archive: backups/qdrant_data_premigration.tar.gz (~60MB, non-empty). Do not delete the original volume.

Restore:

docker compose stop qdrant
docker run --rm -v ox1audio_qdrant_data:/to -v "$PWD/backups":/from alpine \
  sh -c 'rm -rf /to/* /to/.[!.]*; tar xzf /from/qdrant_data_premigration.tar.gz -C /to'
docker compose start qdrant

Code rollback: revert this branch; old MuQ collections remain.

Reindex

uv run python scripts/download_short_chunk_model.py   # ml-worker
# ensure laion/larger_clap_music in HF_HOME volume
docker compose -f compose.yaml -f compose.dev.yaml up -d --build ml-worker backend-worker
# from apps/backend:
uv run python scripts/requeue_analysis.py

Test plan

• Qdrant volume snapshot verified non-empty
• CLAP projection_dim == 512 (refused to proceed otherwise)
• Full catalog reindex (1300/1300 ready, 0 failed)
• New + old collections coexist with expected point counts
• Golden evals before/after recorded
• Spot-check search UI for calm ambient / absent concepts
• Confirm image no longer pulls TensorFlow / essentia

Made with Cursor

[Copilot]

Pull request overview

This PR migrates the ML worker’s audio embedding and rich-tagging pipeline away from non-commercial / copyleft dependencies (MuQ weights + Essentia/TF) to an Apache-licensed CLAP embedder and an MIT-licensed Short-chunk CNN tagger, while versioning Qdrant collections to prevent mixing vectors across “tagger eras” and keeping rollback collections intact.

Changes:

• Replace MuQ audio embeddings with CLAP (laion/larger_clap_music) and enforce 512-d invariants at load/runtime.
• Replace Essentia Discogs/Jamendo tagging with a vendored Short-chunk CNN (Jamendo top-50) + downloadable weights.
• Version profile Qdrant collections by language model + tagger identity; adjust dev Compose env/volumes to preserve built venvs and avoid inheriting host-local URLs.

Reviewed changes

Copilot reviewed 24 out of 26 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
compose.yaml	Updates service comment to reflect CLAP owning the GPU.
compose.dev.yaml	Hard-codes in-compose service URLs and adds a persistent ml-worker venv volume.
apps/ml-worker/uv.lock	Removes Essentia/MuQ-related dependencies; ensures torchaudio is present.
apps/ml-worker/src/ox1audio_ml_worker/vector/store.py	Versions profile collection names by language model + tagger to avoid mixing eras.
apps/ml-worker/src/ox1audio_ml_worker/embedders/provider.py	Implements CLAP text/audio embedding + collection-dimension guards; renames device helpers.
apps/ml-worker/src/ox1audio_ml_worker/config.py	Replaces MuQ/Essentia settings with CLAP model id + tagger weights path.
apps/ml-worker/src/ox1audio_ml_worker/audio/tagging.py	Adds Short-chunk CNN-based rich tagging implementation and provider identity constant.
apps/ml-worker/src/ox1audio_ml_worker/audio/short_chunk/tags.py	Adds Jamendo top-50 tag list required by the checkpoint head.
apps/ml-worker/src/ox1audio_ml_worker/audio/short_chunk/model.py	Vendors the Short-chunk CNN + residual model definition.
apps/ml-worker/src/ox1audio_ml_worker/audio/short_chunk/LICENSE	Adds upstream MIT license text for vendored code.
apps/ml-worker/src/ox1audio_ml_worker/audio/short_chunk/init.py	Exposes the vendored model + tag list as a small internal package.
apps/ml-worker/src/ox1audio_ml_worker/audio/essentia.py	Removes Essentia-based tagging/embedding implementation.
apps/ml-worker/src/ox1audio_ml_worker/audio/analysis.py	Switches pipeline to 48 kHz for CLAP clips and routes rich tagging to Short-chunk CNN.
apps/ml-worker/scripts/download_short_chunk_model.py	Adds downloader for Short-chunk CNN weights.
apps/ml-worker/scripts/download_essentia_models.py	Removes Essentia model downloader.
apps/ml-worker/README.md	Updates architecture docs, licensing table, and setup steps for CLAP + Short-chunk CNN.
apps/ml-worker/pyproject.toml	Drops MuQ/Essentia deps; adds torchaudio dependency.
apps/ml-worker/evals/score_golden.py	Adds script to score golden queries against a running ML worker.
apps/ml-worker/evals/results_before.json	Records baseline eval outputs (MuQ-era).
apps/ml-worker/evals/results_after.json	Records eval outputs after CLAP migration.
apps/ml-worker/evals/relevant_calm_ambient_muq.json	Adds frozen relevance set for before/after comparability.
apps/ml-worker/evals/golden_queries.json	Expands golden query metadata to include frozen relevance ids/tags.
apps/ml-worker/Dockerfile	Reuses CUDA base torch/torchaudio and verifies imports during image build.
apps/ml-worker/config.yaml	Switches default model config to CLAP and documents Short-chunk weights path.
apps/backend/scripts/requeue_analysis.py	Adds helper script to enqueue a full reindex across READY tracks.
.gitignore	Ignores .pth weights artifacts and local backups/ snapshot directory.

Comments suppressed due to low confidence (1)

apps/ml-worker/src/ox1audio_ml_worker/audio/tagging.py:170

• In predict_chunks(), the hop size is divided by n_chunks, which makes chunk start positions much more clustered toward the beginning than intended (and can produce nearly-duplicate windows for longer audio). To spread n_chunks windows across the available range (matching the clip selection logic elsewhere), the denominator should be (n_chunks - 1) so the last window can reach the end of the signal.

    length = int(audio.size)
    n_chunks = min(MAX_CHUNKS, max(1, length // CHUNK_SAMPLES))
    if length == CHUNK_SAMPLES or n_chunks == 1:
        batch = audio[:CHUNK_SAMPLES][None, :]
    else:
        hop = max(1, (length - CHUNK_SAMPLES) // n_chunks)
        rows = [
            audio[i * hop : i * hop + CHUNK_SAMPLES]

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.