Repo rename: This project will live at github.com/Purchio/piano-coach. See RENAME.md for the one-time GitHub Settings step.
Learn to play your baby grand piano using your iPhone or iPad. Piano Coach listens through the microphone, shows which keys to press on an on-screen keyboard, and guides you through songs one note at a time — including simplified Muse piano intros.
- Microphone note detection — hears which key you press and confirms you're on the right note
- Piano calibration (Phase A) — one-time audio walkthrough: play white keys then black keys; saved permanently on your device
- Camera keyboard map (Phase B) — photo + tap your keyboard edges; overlays show which real key to play
- Guided practice mode — highlights the next key, shows finger numbers, advances on correct input
- Song library — Twinkle Twinkle, Ode to Joy, plus simplified Muse piano intros (Madness, Space Dementia, Bliss, New Born)
- Installable PWA — add to iPhone/iPad home screen for a native-like experience
Before your first practice session:
- Tap Calibrate piano on the home screen
- Audio (Step 1): Play each white key left to right, then each black key. The mic records your piano's exact pitch for each key.
- Camera (Step 2): Capture a photo looking down at the keys, tap the leftmost and rightmost keys, then the front edge. Select which notes they are.
- Tap Save — calibration persists in your browser's local storage on that iPad/iPhone.
After calibration, note detection uses your piano's tuning instead of generic pitch tables, and the camera view shows glowing dots on your real keys during practice.
https://purchio.github.io/excursion/
Open in Safari, then:
- Tap Share → Add to Home Screen
- Tap Calibrate piano (top right)
- Pick a song → Enable microphone & start → tap Allow
The app listens to your piano — it does not play song audio through the iPad.
Note:
piano-coach.vercel.appis a different app (MIDI library). This repo deploys to the GitHub Pages URL above.
npm install
npm run dev:localOpen the local URL on your iPhone or iPad (same Wi‑Fi network) for testing changes before deploy.
- Calibrate your piano first (one-time, ~2 minutes)
- Place your iPad or iPhone where the microphone can hear the piano (on the music desk works well)
- Choose a song — start with Twinkle Twinkle if you're brand new
- Tap Start practice
- Play each highlighted note; the app turns green when you're correct
- Enable the camera (📷) to see glowing dots on your real keys (after camera calibration)
- React 19 + TypeScript + Vite
- Web Audio API for real-time pitch detection (autocorrelation)
- MediaDevices API for microphone and rear camera
- PWA via vite-plugin-pwa
- Microphone-based detection works best in a quiet room, close to the piano; calibration improves accuracy significantly
- Camera map uses geometric key layout from your taps — very accurate for overhead views, less so if the angle is steep
- Song catalog is hand-authored simplified arrangements, not full transcriptions
- iOS requires HTTPS for mic/camera in production (localhost works for dev)
Our app (excursion) verifies you on a real piano via mic + camera. The other piano-coach.vercel.app app excels at MIDI playback, falling notes, and a large song library. Go-forward plan:
| Feature | Priority | Notes |
|---|---|---|
| Falling notes (Synthesia-style) | Nice-to-have | Blue/red lanes per hand, notes scroll toward keys |
| Finger on every note (1–5 per hand) | Nice-to-have | “Mavis Beacon for piano” — thumb vs pinky on each note |
| MIDI / MusicXML import | High | Unlock Sky Full of Stars, full library, user uploads |
| Song audio playback | Nice-to-have | Hear the song while you play along (speakers/headphones) |
| Mic verification during scroll mode | Core differentiator | Other app shows notes; we confirm you hit them |
Completed:
- Audio calibration wizard
- Camera keyboard map with key overlays
- Per-note finger hints (practice mode)
- Falling notes view with L/R hand lanes
- Finger labels on all falling notes (both hands)
- MIDI import (Load MIDI/MusicXML)
- Playback audio + scroll sync
- Larger built-in song catalog
MIT