📺 Watch the full tutorial on YouTube

🎵 Audio Music AI SheetSage2 Transcriber

SheetSage2 is a lightweight AI music foundation model that converts raw audio recordings (.mp3, .wav) directly into editable lead sheets, multi-track MIDI files, and ABC musical notation.

Step 1: Input	Step 2: AI Action	Step 3: Result
Raw Audio Track Load any MP3 or WAV recording	SheetSage2 Model Extract pitch, beat grid & chords	Score & MIDI Export Get editable ABC score & MIDI file

---

⚡ Key Highlights

• 📦 Tiny Model Footprint: Only 229 MB in size, running smoothly on CPU or local GPUs with under 4GB VRAM.
• 🎼 Multi-Format Output: Generates printable lead sheets (score.abc), multi-track MIDI (transcription.mid), and timed JSON event annotations.
• 🏆 Top Benchmark Accuracy: Achieves 92.9% downbeat precision and 90.08% chord recognition score.
• 🚀 Hugging Face Native: Built on MERT-v2 FullSong acoustic features with 3-line Python pipeline execution.

---

🛠️ Model Download & Quickstart Setup

Copy and paste this single PowerShell command to install all dependencies and download the model checkpoint directly:

pip install torch torchaudio transformers soundfile mir_eval mido pretty_midi huggingface_hub; hf download m-a-p/SheetSage2 --local-dir models/SheetSage2

---

💻 Python Usage Example

import torch
from transformers import AutoModel

# Load SheetSage2 model locally or from Hugging Face
model = AutoModel.from_pretrained(
    "m-a-p/SheetSage2", trust_remote_code=True
).eval().to("cuda" if torch.cuda.is_available() else "cpu")

# Transcribe audio file into MIDI and ABC score
result = model.transcribe("inputs/piano_sample.wav", output_dir="outputs")

---

🎯 5 Practical Use Cases

1. 🎸 Song Transcriptions for Musicians: Instantly convert recorded acoustic jams into printed sheet music.
2. 🎹 DAW Remixing & Beat Making: Import extracted multi-track MIDI files directly into FL Studio or Ableton.
3. 🎤 AI Vocal Cover Creation: Extract clean lead sheets to feed melody lines into YuE2 music generators.
4. 📚 Music Education & Analysis: Analyze song chord progressions, key signatures, and measure structures.
5. 🔍 Audio Archiving & Cataloging: Generate searchable text annotations (events.json) for large audio libraries.

---

🔮 5 Future Enhancements

1. 🎛️ Live Real-Time Audio Transcription: Stream microphone audio directly into live scrolling sheet music.
2. 🎻 Orchestral Instrument Separation: Expanded multi-stave separation for complex classical ensemble scores.
3. 📱 Mobile & Edge Export: Quantized ONNX weights for fast execution on tablets and mobile devices.
4. 🎼 Guitar Tablature Converter: Automatic conversion from ABC notation to 6-string guitar tabs.
5. 🌐 WebGPU Web Interface: Browser-based drag-and-drop transcription tool powered by local WebGPU.

---

📁 Repository Structure

SheetSage2/
├── main.py             # Main production transcription pipeline script
└── README.md           # Project documentation and guide

---

🏷️ Keywords & Tags

SheetSage2 AI Music Transcription Audio to MIDI Lead Sheet Generator MERT-v2 ABC Notation Python Music AI Sheet Music Transcriber AI Chord Recognition Local AI Model