feat(enrichment): external audio-features provider as primary energy source (+ Lexicon override, LLM fallback)

[thewrz]

Part of the pool→builder data-contract epic (#539). Provides the energy (+ mood/danceability/valence) columns for the global store + request-time enrichment.

Why

Energy is the one required field no current integration supplies: Beatport/Tidal give bpm/key, MusicBrainz gives genre, but none give energy/danceability, and Spotify's audio-features API (energy/danceability/valence/…) was deprecated on 2024-11-27 (new apps get 403; no official replacement). The chosen strategy is an external audio-features API as the primary, whole-pool-coverage energy source, with Lexicon measured energy as an override where the DJ has it, and LLM-inferred energy as the last-resort fallback.

Scope

• New provider adapter behind a clean interface, config-gated, writing energy (0–10 normalized) + source/freshness into the global tracks row (and thus available at request time and in the setbuilder pool).
• Capture the whole audio-feature payload, not just energy. The chosen providers return danceability + valence (and acousticness/instrumentalness/speechiness/liveness) in the same call as energy, so storing them is zero marginal API cost. Add energy, danceability, valence as the near-term trio and the rest as nullable columns in the same migration — re-fetching the whole catalog later to backfill them would pay the call twice. This feeds future WrzDJSet pass-1 sequencing (energy/mood/danceability-aware), where energy × valence forms the canonical mood quadrant for emotional-arc sequencing. (YAGNI deliberately does not apply — see epic(setbuilder): pool→builder data contract + global enriched-track store #539.)
• Also capture explicit (boolean). Soundcharts returns it in the same payload. Unlike the audio features (which weight ordering), explicit is a hard pool pre-filter enabling a future "clean set / all-ages event" toggle (e.g. family events) — a one-line pool filter later instead of a re-enrichment pass. Free to store now.
• Energy cascade (precedence): external API (baseline) ← Lexicon measured override (research(setbuilder): LexiconDJ as an optional measured-energy source of truth for WrzDJSet (+ library sync) #526, ISRC-gated, local-only) ← falls back to LLM-inferred (WrzDJSet: TrackVibe LLM enrichment + community vibe display (read) #391) when neither is available.
• Match by ISRC (preferred) / normalized signature; cache forever in the global store (one call per unique track).

Vendor decision — REVISED after deep research (2026-06-23); still requires contract/ToS confirmation before pinning

Multi-source, adversarially-verified research (see comment below) reorders the candidates. The recommended FIRST provider is Soundcharts — which WrzDJ already licenses and already uses for genre/bpm/key.

• ★ PRIMARY — Soundcharts (https://soundcharts.com/en/audio-features-api). As of v2.46.1 (Nov 2025) its Audio Features API returns the full Spotify-style descriptor set — energy, danceability, valence, acousticness, instrumentalness, speechiness, liveness, bpm, key/mode, time signature, loudness — on the 0–1 scale, and supports ISRC lookup (GET /song/by-isrc/{isrc} → UUID, then GET /song/{uuid} → features). ISRC-native (our exact key, no audio upload), no new vendor/secret/supply-chain trust decision.
  • Empirically confirmed (2026-06-23) against GET /api/v2.25/song/{uuid} with our existing key (test track: "bad guy", ISRC USUM71900764). Features are nested under object.audio.* and the same call also returns tempo (bpm), key+mode, genres (root/sub), duration, and the echoed isrc — i.e. one ISRC-resolvable call can backfill bpm+key+genre+duration+energy+danceability+valence together (a single-source-consolidation opportunity for feat(enrichment): write the global track store at request time (enrich once, reuse everywhere) #541/feat(setbuilder): pool reads global store + pool→builder contract & build coverage gate #542, not just an energy source). Response headers observed: x-ratelimit-limit: 10000 (reset 31s — generous), x-quota-remaining: 1000 (plan credit allotment — the real ceiling).
  • Gates before pinning: (a) caching + product-use rights — the public website ToS (https://soundcharts.com/en/terms) is NOT the API data license; it is restrictive by default (Art. 5.1 prohibits copying/distributing Soundcharts data without consent) and requires a visible "Source: www.soundcharts.com" attribution (Art. 6.2), and is silent on API caching/TTL. The right to cache per-track features in our DB indefinitely lives in the separate API/commercial agreement tied to the subscription — must check that agreement (or ask Soundcharts directly). Mitigation/lever: WrzDJ mostly consumes features as an internal algorithmic input (the DJ sees a sequenced set, not raw energy=0.43); storing them as a private input and not displaying/redistributing the raw values sidesteps most of the redistribution restriction — only the storage right still needs the agreement's blessing. (b) plan quota capacity — x-quota-remaining was 1000, so a bulk catalog backfill could exhaust the allotment even though steady-state (one cached call per new track) is cheap. Adapter must treat quota-exhaustion → fall back to ReccoBeats, not retry. (Plan-tier inclusion of the Audio Features API is already proven by the live test.) If caching/product-use is disallowed by the API agreement, ReccoBeats becomes primary (explicitly free for commercial use, no attribution; its catch is undocumented Spotify-data provenance, not a restrictive grant).
• FREE FALLBACK — ReccoBeats (https://reccobeats.com). Returns the same nine Spotify features on the identical 0–1 scale; free incl. commercial use, no attribution. Caveats: no ISRC lookup (only ReccoBeats UUID / Spotify ID / audio upload — requires our existing ISRC→Spotify-ID resolve), feature-value provenance undocumented (the claim that values are re-derived vs. stale Spotify values was refuted 1-2), operator "LatteBits" opaque, and its ToS makes the user "responsible for compliance with Spotify's terms" (Spotify-aggregated data → redistribution risk on cached values).
• NOT a numeric drop-in — Cyanite.ai (commercial). Rich mood (13 labels) + valence/arousal + bpm/key, but energy is categorical (variable/medium/high/low) and there is no danceability field; submission is by audio upload, not ISRC.
• DEAD — AcousticBrainz. Frozen to submissions since Jun 2022, site shut down early 2023; only a static dump (29.4M rows, 2022-07-06) survives, keyed by MBID not ISRC, and MetaBrainz itself disowned the data quality (no per-value confidence). Unusable as a live source.
• Core MusicBrainz exposes no audio features (genre + folksonomy tags + relationships only) — confirms our MusicBrainz=genre-only usage is correct. ListenBrainz is listen data, not features.
• Essentia (https://essentia.upf.edu/) — self-hosted, open-source, needs the audio file; "someday / audio-on-hand" only. License (AGPL?) + classifier accuracy unverified — vet separately before relying on it.

Spotify deprecation context — https://developer.spotify.com/blog/2024-11-27-changes-to-the-web-api

Acceptance criteria

• Soundcharts plan tier confirmed to include the Audio Features API, and caching/redistribution ToS confirmed to permit indefinite local caching (else fall back to ReccoBeats as primary). Documented license + provenance + CVE check per security rules; secret via env (never hardcoded); config-gated.
• Provider adapter writes the full feature payload (energy + danceability + valence + acoustic/instr/speech/live) with per-field source + freshness into the global tracks row; unit-tested.
• Energy resolves through the cascade external → Lexicon override → LLM fallback, with source recorded; unit-tested.
• One unique track = one provider call (cached in the global store); coverage stays at/above the enforced gate.

Refs

Feeds the global tracks table (Task 0) + request-time enrichment. Cascade ties to #526 (Lexicon override) and #391 (LLM fallback). See also #527 (enrichment audit).

[thewrz]

Vendor research (deep, adversarially verified) — recommendation revised to Soundcharts-first

Multi-source web research (5 angles, 19 sources, 25 claims adversarially fact-checked — 21 confirmed, 4 refuted) plus a live API test reorders the candidates. The issue body has been updated to match.

TL;DR

PRIMARY → Soundcharts (already licensed by WrzDJ, ISRC-native, full feature set) → FREE FALLBACK → ReccoBeats (via ISRC→Spotify-ID resolve) → LLM-inferred (#391) last resort, with Lexicon measured (#526) as the override tier.

Provider matrix

Provider	energy / danceability / valence	Scale	Lookup key	Cost	ISRC-native	Verdict
Soundcharts	✅ all + acoustic/instr/speech/live + bpm/key/genre/duration	0–1	ISRC → UUID → features	Paid — already in stack	✅	★ First choice
ReccoBeats	✅ all nine Spotify features	0–1 (identical)	UUID / Spotify ID / audio upload	Free (commercial OK, no attribution)	❌ needs ISRC→Spotify-ID	Best free fallback; provenance opaque
Cyanite.ai	⚠️ mood labels + valence/arousal; energy categorical, no danceability	mixed	audio upload	Paid	❌	Poor numeric drop-in
AcousticBrainz	historical only	—	MBID	free dump	❌	Dead (frozen Jun 2022; maintainers disowned quality)
Essentia (self-host)	✅ via classifiers	model-dep.	needs audio file	free (license unverified)	n/a	Last-resort, audio-on-hand
Core MusicBrainz	❌	—	MBID	free	—	Genre/tags only (confirms current usage)

Empirical confirmation (live test, 2026-06-23)

GET /api/v2.25/song/{uuid} on "bad guy" (ISRC USUM71900764) returned, nested under object.audio: energy 0.43, danceability 0.7, valence 0.56, acousticness 0.33, instrumentalness 0.13, liveness 0.1, speechiness 0.38, loudness -10.97, tempo 135.13, key 7, mode 1, timeSignature 4 — plus top-level genres (root/sub), duration (194s), explicit, and the echoed isrc. So one ISRC-resolvable Soundcharts call can backfill bpm + key + genre + duration + energy + danceability + valence + explicit together (a single-source-consolidation opportunity for #541/#542).

Load-bearing caveats (must close before pinning)

1. Caching/redistribution ToS unconfirmed. The "Soundcharts endorses caching in your own DB" claim was refuted 0-3 — confirm the contract permits indefinite local caching, since the global-tracks store assumes cache-one-call-forever.
2. Plan quota capacity. Observed x-quota-remaining: 1000 (with x-ratelimit-limit: 10000/31s). Steady-state is cheap, but a bulk backfill could exhaust the allotment → adapter must treat quota-exhaustion as "fall back to ReccoBeats", not retry.
3. Feature-value provenance is undocumented for both Soundcharts ("derived from streaming platform analysis") and ReccoBeats (re-derived-vs-stale-Spotify claim refuted 1-2) — treat values as approximate; per-field source + freshness (already in epic(setbuilder): pool→builder data contract + global enriched-track store #539) is the right mitigation.

MusicBrainz / siblings (answers the secondary question)

Core MusicBrainz exposes no audio features (genre + folksonomy tags + relationships only — our genre-only usage is correct). AcousticBrainz is dead (frozen Jun 2022, site down early 2023; static dump keyed by MBID not ISRC; MetaBrainz disowned its quality — no per-value confidence). ListenBrainz is listen data, not features.

Scope decision confirmed

Adding mood/danceability/valence (+ explicit) columns now is right: they ride free in the same call as energy; not storing them means re-traversing the catalog later. explicit is a hard pool pre-filter (clean/all-ages sets); energy×valence is the mood quadrant for future pass-1 emotional-arc sequencing.

Sources

• Soundcharts Audio Features API — https://soundcharts.com/en/audio-features-api ; metadata reference — https://developers.soundcharts.com/api/reference/song/get-song-metadata
• ReccoBeats docs (audio-features, resource-id, ToS) — https://reccobeats.com/docs/
• Cyanite V6 classifier — https://api-docs.cyanite.ai/docs/audio-analysis-v6-classifier/
• AcousticBrainz shutdown — https://blog.metabrainz.org/2022/02/16/acousticbrainz-making-a-hard-decision-to-end-the-project/ ; https://acousticbrainz.org/download
• MusicBrainz API — https://musicbrainz.org/doc/MusicBrainz_API
• Essentia — https://essentia.upf.edu/

🤖 Research by Claude Opus 4.8 via a fan-out → adversarial-verify → synthesize workflow.

[thewrz]

Batching verification — Soundcharts has NO batch endpoint (cross-validated)

Checked the live API docs + an independent search. Every Soundcharts song endpoint is single-resource (one ISRC or one UUID per call). The 19 song endpoints (get-song-by-isrc, get-song-metadata, get-song-by-platform-id, get-ids, …) are all singular; the only multi-song endpoint is get-songs, which is filter-based discovery (genre/bpm/key — already used by discover_songs), not lookup-by-ISRC-list. Bulk delivery exists only as the separate Enterprise data dump & data feed (S3/GCS/ClickHouse, ≥$2,000/mo).

Sources: developers.soundcharts.com/documentation/reference/song/get-song-by-isrc · /song/summary · soundcharts.com/en/isrc-api · soundcharts.com/en/music-metadata-api

Implication for the quota strategy (free legacy tier = 1000 calls/mo)

Soundcharts can't batch, so the lever is "fewer tracks ever called," not "fewer calls per track." Two scenarios:

• Steady-state (request-time enrichment, feat(enrichment): write the global track store at request time (enrich once, reuse everywhere) #541): inherently one-track-at-a-time as guests submit — batching buys nothing. Use Soundcharts single-call ISRC + cache forever. The built-in single-call optimization (skip the 2nd call when by-ISRC already carries audio) halves per-track cost. 1000/mo free ≈ 1000 new unique tracks/mo — fine for incremental.
• Bulk backfill (one-time, existing catalog): the only case where volume blows the quota. Route it through a batch-capable provider — ReccoBeats (GET /v1/track?ids=…, free, via ISRC→Spotify-ID), or a single paid Soundcharts month ($250 = 500k calls), or the enterprise dump.

Workflow requirements (#541 owns these)

1. Dedup against the global tracks store before any call — never re-enrich a known ISRC (biggest saver; this is the whole point of epic(setbuilder): pool→builder data contract + global enriched-track store #539's "enrich once per unique track").
2. Cache forever + single-call optimization (adapter already does the latter).
3. Quota-exhaustion → fall back to ReccoBeats; ReccoBeats is also the batch backfill path.

NOT doing: a get_song_features_by_isrcs() that loops single Soundcharts calls — it implies a saving that doesn't exist and burns quota identically. A real batch interface belongs in the #541 workflow layer (dedup + ReccoBeats batch), not the Soundcharts adapter.