Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 5 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -89,4 +89,8 @@ backend/app/priors/grooves/*.json
# Raw MIDI datasets must never be committed (song-transcription corpora are
# copyrighted; keep the repo data-free).
*.mid
*.midi
*.midi

# Source WAVs downloaded by scripts/build_velocity_samples.py. Only the converted
# MP3s under frontend/public/samples/ are committed; the bulky source cache is not.
.sample-cache/
30 changes: 30 additions & 0 deletions DATA_LICENSES.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,6 +60,36 @@ artifacts derived from permissively-licensed (CC-BY, public-domain) sources, and
set bundled at `frontend/public/samples/piano/` (the Tone.js-hosted subset) is
derived from this work. https://archive.org/details/SalamanderGrandPianoV3

## Bundled instrument samples (`frontend/public/samples/`)

These are the audio one-shots the app plays. Samples are **data**, licensed
separately from the GPL-3.0 code. **Everything shipped now has a confirmed license:**

| Set | Source | License | Notes |
|---|---|---|---|
| **Piano** (`piano/`) | Salamander Grand Piano (Alexander Holm) | **CC-BY 3.0** | Attributed above. Clean to ship. |
| **Vibraphone** (`melodic/vibraphone/`) | **VCSL** — Versilian Community Sample Library | **CC0 / public domain** | Velocity-layered (soft/hard mallets) via `scripts/build_velocity_samples.py`. No attribution required; kept here as courtesy. https://github.com/sgossner/VCSL |

Every other instrument is **synthesized** in the app (no shipped samples), so there is
nothing else to license. A **Samples / Synth toggle** in the transport bar switches
between the sampled voices above and full synthesis.

> **Removed for licensing (2026-07-23).** The following sets were deleted because their
> redistribution rights were **not confirmed** — see `docs/LICENSE_AUDIT.md`:
> - **MusyngKite** (all `bass/` + 7 `melodic/` voices: electric pianos, clavinet,
> accordion, drawbar organ, nylon guitar, string ensemble). The redistributor labels
> it CC-BY-SA 3.0, but the soundfont's original author says it is "free to use but
> **not meant to be redistributed**… or used for commercial purposes." Those voices
> now synthesize.
> - **Tone.js drum samples** (`drums/`). No LICENSE on the source repo; and the app
> already synthesizes its drum kit, so the samples were unused. Deleted.
>
> **Adding samples back cleanly:** use `scripts/build_velocity_samples.py` with a
> confirmed **CC0 / public-domain** (VCSL) or **CC-BY** (with attribution) source only.
> Note that University of Iowa MIS, despite being widely called "free," carries **no
> explicit license grant**, so it is *not* used here. VCSL is acoustic/orchestral and
> does not cover the electric/synth voices, which is why they remain synthesized.

## Note on the GPL and data

The GPL-3.0 covers this project's source code. It does **not** attempt to license
Expand Down
106 changes: 106 additions & 0 deletions docs/LICENSE_AUDIT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# License compliance audit — 2026-07-23

A full pass over everything GenreGrid ships, to confirm we (a) respect third-party
licenses and (b) correctly assert our own (GPL-3.0). Companion to
[`DATA_LICENSES.md`](../DATA_LICENSES.md), which covers mined corpora and priors.

**Scope:** bundled audio samples, frontend (npm) and backend (pip) dependencies,
fonts/icons, and our own license declaration + source headers.

---

## Verdict

| Area | Status |
|---|---|
| Our license declaration (GPL-3.0) | ✅ after fixing one metadata mismatch |
| Frontend npm dependencies | ✅ all GPL-3.0-compatible |
| Backend pip dependencies | ✅ all GPL-3.0-compatible |
| Fonts | ✅ none bundled (system font stacks) |
| Icons | ✅ first-party |
| Piano samples | ✅ Salamander, CC-BY 3.0 (attributed) |
| Vibraphone samples | ✅ VCSL, CC0 |
| **Drum samples** | ✅ unconfirmed license + **unused** — deleted |
| **Bass + melodic samples** | ✅ MusyngKite removed — those voices now synthesize |

**Result: the app ships no asset with unconfirmed redistribution rights.** Remaining
sampled voices are piano (CC-BY 3.0) and vibraphone (CC0); everything else synthesizes,
toggleable via the transport bar's Samples / Synth control.

---

## 1. Our own license (GPL-3.0)

- `LICENSE` at repo root is the full GPL-3.0 text. ✅
- Every frontend `src/**/*.{ts,vue}` and backend `backend/app/**/*.py` file carries the
GPL header granting "version 3 … or (at your option) any later version." Only empty
`__init__.py` package markers lack it (no copyrightable content). ✅
- **Mismatch (fixed):** `frontend/package.json` declared `"license": "GPL-3.0-only"`, but
the source headers grant **or-later**. Corrected to `GPL-3.0-or-later` so the metadata
matches the actual grant. The backend has no package metadata license field to set.

## 2. Frontend dependencies (npm)

49 production dependencies, by license: MIT (39), ISC (3), BSD-2-Clause (1),
BSD-3-Clause (1), Apache-2.0 (1), 0BSD (1), BlueOak-1.0.0 (1), Python-2.0 (1). All are
permissive and **GPL-3.0-compatible**. The only `UNLICENSED` entry is our own root
package (`genregrid-frontend`), not a third party.

Dev-only dependencies additionally include a few MPL-2.0 packages (build tooling); MPL-2.0
is GPL-compatible and, regardless, dev tooling is **not distributed** in the packaged app.

_Re-check: `cd frontend && npx license-checker-rseidelsohn --production --summary`._

## 3. Backend dependencies (pip)

fastapi, starlette, httpx (BSD-3), pydantic, mido, uvicorn, python-multipart, idna,
pytest, pytest-asyncio (MIT/BSD). All permissive and **GPL-3.0-compatible**.

## 4. Fonts & icons

- **Fonts:** none bundled — the UI uses system font stacks, so nothing to license. ✅
- **Icons:** `frontend/public/icon.{png,ico}` are first-party GenreGrid artwork. ✅

## 5. Bundled audio samples

Samples are **data**, licensed separately from the GPL code. Detail in
[`DATA_LICENSES.md`](../DATA_LICENSES.md → "Bundled instrument samples").

- ✅ **Piano** — Salamander Grand Piano, **CC-BY 3.0**, attributed.
- ✅ **Vibraphone** — **VCSL, CC0**, velocity-layered (migrated 2026-07-23).
- ✅ **Drums** (`samples/drums/`, 8 kits) — from Tone.js `audio` repo, which has **no
LICENSE file** (unconfirmed rights on vintage drum-machine samples). The app plays a
**synthesized** kit (`makeSynthKit`); `getDrumKit`/the sampled kits had **no callers**.
**Deleted** (dead weight — zero sound impact), along with the dead code.
- ✅ **Bass** (6 sets) **+ Melodic** (7 sets: electric pianos, clavinet, accordion, drawbar
organ, nylon guitar, string ensemble) — from gleitz `midi-js-soundfonts` → **MusyngKite**.
The redistributor labels it CC-BY-SA 3.0, but the soundfont's original author states it is
"free to use but **not meant to be redistributed** … or used for commercial purposes" —
so our redistribution rights are **not confirmed**. **Deleted; those voices now synthesize.**

### How the MusyngKite removal was done

Goal: ship **no** MusyngKite-derived audio, achieved as follows:

1. Deleted all MusyngKite sample sets. Sampled voices are gated behind confirmed-license
allowlists (`SAMPLED_VOICES`, `SAMPLED_BASS_VOICES`); voices not on them synthesize.
2. Added a **Samples / Synth toggle** (`sampleMode`, persisted) so users can A/B the sampled
voices against full synthesis — and the seam a future "bring your own samples" feature uses.
3. Removed `scripts/download_samples.py` (it only fetched the now-banned sources).

**Sources considered and rejected for re-sourcing:**
- **University of Iowa MIS** — widely called "free," but the site carries **no explicit
license grant** (same ambiguity as MusyngKite), so it is *not* used.
- **Freesound** — many CC0 samples exist, but the API requires auth unavailable in the
automation environment; a manual, per-sample CC0 curation pass remains a future option.

Confirmed-clean, direct-download sources that *were* usable: **VCSL (CC0)** and **Salamander
(CC-BY)** — which only cover piano + vibraphone among GenreGrid's voices.

---

## Maintenance

Re-run this audit when adding any dependency, sample set, font, or icon. Never bundle audio
without a confirmed CC0 / public-domain / CC-BY (with attribution) license. Keep the repo
free of any asset whose redistribution rights are unconfirmed.
156 changes: 156 additions & 0 deletions docs/custom-instruments-design.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# Design: User-uploaded custom instruments

**Status (2026-07-23): MVP IMPLEMENTED (needs desktop runtime testing).** Design for
letting users add their own samples, map them to notes/velocities, and pick which
instrument plays each part. Fleshes out the Phase 2 roadmap item "Custom soundfont /
SF2 upload + per-part instrument picker."

**Shipped (T1 + T2 tier):** pure mapping core (`customInstruments.ts`, unit-tested),
library store (`useCustomInstruments.ts`), Electron storage IPC + preload (list/save/
remove/read under `userData/instruments/`, played back as blob: URLs — no custom
scheme, per the Linux Web-Audio note below), engine integration (every part resolves a
custom instrument first in `useMidiPlayer.ts`), and the Instruments panel UI + a 🎹
transport-bar button. **Not yet done:** velocity-layer preview, SF2/SFZ import (T4),
auto pitch-detection, per-style-override UI, web/OPFS storage. **Untested at runtime**
(automation can't launch Electron / play audio) — needs a desktop pass: import a file,
assign it to a part, play, and delete.

## Why now

The July 2026 license pass removed the MusyngKite bass/melodic samples, so 13 voices
now synthesize (see [`LICENSE_AUDIT.md`](LICENSE_AUDIT.md)). Letting users bring their
own samples both restores instrument realism **and sidesteps the licensing problem** —
user-supplied audio is the user's to use, nothing we ship. It also builds directly on
the two seams already in place: the `LayeredSampler` **manifest format** and the
**registry** (`voiceFor(styleId, part)`) that resolves each part to a voice id.

## Core concept

> A **Custom Instrument** is just a `LayeredSampler` manifest + its audio files, given a
> voice id, that the user can select for any part.

**Every part/voice is user-samplable — no exceptions.** Chords, melody, arpeggio, bass,
pads, and counter-melody can each be driven by a user instrument, including the voices
that currently synthesize. There is no "these are sampled, those are synth-only" split at
the user level: the per-part picker offers a user instrument for *any* part, and a user
instrument can replace a built-in voice entirely. (Custom drum *kits* use a pitch→one-shot
mapping rather than a chromatic range, so they get a dedicated editor in a later phase, but
they are still in scope as user-uploadable.)

Nothing new in the audio engine — `LayeredSampler` already plays a `velocity.json`
manifest (note → file(s), velocity layers, round-robins). A custom instrument is that
same structure, sourced from user files instead of `/samples/`. The registry's job
("what voice does this part use?") gains a third answer: built-in sample set, synth
family, **or a user instrument**.

---

## Key decisions (with options + recommendation)

### 1. Upload format & note mapping — *how do files become a playable instrument?*

The hard part of "easy and manageable" is note/velocity mapping. Recommend a **tiered**
approach so the easy case is trivial and quality scales up, all producing one manifest:

| Tier | User does | We do | For |
|---|---|---|---|
| **T1 — one shot** | Drop **one** audio file | Map it across the keyboard, pitch-shifted (single-zone `Tone.Sampler`) | Casual: "make an instrument from this sound" |
| **T2 — named files** | Drop files named by note (`C4.wav`, `A#3.wav`, `Fs3…`) | Parse note from filename → multi-zone manifest | Better realism, minimal effort |
| **T3 — velocity/RR** | Files with velocity/RR suffix or in `soft/`,`hard/` subfolders (our pipeline's convention) | Build layered manifest | Full dynamics — matches `build_velocity_samples.py` output |
| **T4 — soundfont** | Upload `.sf2` / `.sfz` | Parse zones/velocity splits → manifest (+ decode samples) | Import existing libraries |

- **Recommendation:** ship **T1 + T2** as the MVP (covers "easy and manageable" and
reuses the manifest 1:1), then **T3** (already produced by our own pipeline), then
**T4** last (SF2/SFZ parsing is the heaviest — needs a parser + sample decode; SFZ is
text-simple, SF2 is binary and benefits from a lib like `soundfont2`).
- A drag-drop **zip or folder** import wraps T2/T3 (one gesture, many files).
- **Optional polish:** auto pitch-detection (detect fundamental) to relax the filename
requirement in T1/T2 — nice, not required for MVP.

### 2. Assignment model — *which instrument belongs to which part?*

- **Option A — Instrument-first (recommended):** the user creates a named, reusable
instrument ("My Rhodes") in a library, then assigns it to parts via a **per-part
instrument picker**. One instrument, usable across styles/parts.
- **Option B — Part-first:** the user opens a part ("Bass") and uploads for it directly.
Simpler mental model but not reusable; leads to re-uploading the same sound per style.
- **Recommendation: A.** It matches the registry (voices are reusable identities) and
makes the per-part picker a pure selection surface. The picker for each part
(chords / melody / arpeggio / bass / pads / counter-melody) offers: **Synth · Built-in
sample · [user instruments…]**, extending today's `sampleMode` toggle from a global
2-way switch into a per-part source choice.

### 3. Storage & serving — *where do the files live and how does Tone load them?*

Uploaded audio is **user data**, never bundled. `Tone.Sampler` needs a fetchable URL.

- **Electron (primary):** store under `app.getPath('userData')/instruments/<id>/` with a
`manifest.json` per instrument + a top-level `index.json`. Serve to the renderer via a
**custom protocol** (`protocol.handle('gginstr', …)`) → stable, cacheable URLs
(`gginstr://<id>/<file>`) that `LayeredSampler` fetches like `/samples/…`. Cleaner than
IPC-read → object URLs, and respects `contextIsolation` + CSP (add the scheme to CSP).
- **Web build (fallback):** **OPFS** (Origin Private File System) for the bytes + object
URLs, or IndexedDB blobs. Same manifest shape.
- **Recommendation:** custom protocol in Electron; OPFS in the browser. Both hand
`LayeredSampler` a URL, so the engine is storage-agnostic.

### 4. Persistence of assignments

A small user settings object (persisted like `sampleMode`): a default per-part
instrument map, with optional per-style overrides. Resolution order at play time:
**per-style user override → global user default → registry `voiceFor` → synth fallback.**

---

## Data model

```ts
interface CustomInstrument {
id: string // uuid; also the voice id and storage folder
name: string // "My Rhodes"
kind: 'melodic' | 'bass' | 'drums'
manifest: LayeredSamplerManifest // note → file(s), velocity layers, RR
createdAt: number
}
// index.json: CustomInstrument[] (without the audio bytes)

interface InstrumentAssignments {
defaults: Partial<Record<PlayerPart, string>> // part → instrument id
perStyle?: Record<string /*styleId*/, Partial<Record<PlayerPart, string>>>
}
```

## Integration points (all already exist)

- **Engine:** `LayeredSampler` (`layeredSampler.ts`) — unchanged; loads a user manifest
by base URL. `loadLayeredSampler` already fetches `velocity.json`; a user instrument
supplies the same shape.
- **Resolution:** extend the `voiceFor(...)` consumers in `useMidiPlayer.ts` to first
consult `InstrumentAssignments`, then fall through to the registry, then synth.
- **Mode:** generalize `sampleMode` (`useMidiPlayer.ts`) — 'synth' | 'samples' stays as
the global default; the per-part picker overrides it per part.
- **UI:** a new **Instruments** drawer/panel (library CRUD + upload + mapping preview)
and a compact per-part picker on `PartCard.vue` / the transport.
- **Electron:** custom protocol registration + an IPC surface for import/list/delete
(reuse the `save-temp-file` basename-hardening pattern for uploaded filenames).

## Phased delivery

1. **MVP** — Instruments library (list/add/delete) + T1 (one-shot) & T2 (note-named)
import + Electron protocol storage + per-part picker wired into playback. Piano/
vibraphone remain the built-in sampled voices; users fill the rest.
2. **Velocity layers (T3)** + a light mapping preview (mini keyboard, hear each zone).
3. **SF2/SFZ import (T4)** — start with SFZ (text), then SF2 via a parser.
4. **Polish** — auto pitch-detection, per-style overrides UI, export/share an instrument
as a zip (round-trips with `build_velocity_samples.py` output).

## Open questions / risks

- **Formats to accept:** wav/mp3/ogg/flac? (Web Audio decodes all; storage size varies —
maybe transcode-to-ogg on import to bound footprint.)
- **CSP:** the custom scheme must be allowed for media without weakening the existing
localhost-only policy.
- **Large libraries:** memory/CPU if a user loads many big multisamples — lazy-load per
style, cap total decoded buffers.
- **Drums:** custom drum *kits* map pitch→one-shot, not a chromatic range — likely a
separate, later editor from the melodic flow.
Loading
Loading