Multiverse Hands 🖐️🎨

English · 简体中文

Turn your hands into a Spider-Verse multi-style comic filter machine — right in your browser.

Pure frontend · no backend · no upload · no login · 100% offline after clone

• 📷 Two-hand tracking with MediaPipe Hand Landmarker
• 🖼️ Ten comic-book shaders rendered in a single WebGL fragment program
• ✂️ Thumb-and-index "viewfinder" gesture → pinch both hands together to freeze a panel and cycle to the next filter
• 📸 One-click PNG export · 🧹 One-click clear all frozen panels
• 🌐 MediaPipe bundle + wasm + model are all vendored locally — no CDN requests at runtime

▶ Try the live demo  ·  (Drop a demo GIF here once you've played with it.)

---

🚀 Run locally

# Option A — Python's built-in server (no deps)
python3 -m http.server 5178 --bind 127.0.0.1

# Option B — Node
npx http-server . -a 127.0.0.1 -p 5178 -c-1
# or: npm run serve

Open http://127.0.0.1:5178/ in Chrome or Edge and allow camera access.

⚠️ Camera access requires localhost, 127.0.0.1, or HTTPS. You cannot open file:// and expect the camera to work.

🧪 Test

npm test

31 node --test cases covering hand geometry, effect selection, frame-mode state machine, and static WebGL renderer checks.

---

🎮 How to play

Mode 1 — Three comic zones between your fingers

Show both hands to the camera with fingers spread. Same-name fingertips on the left and right hand are paired up and used to build three quadrilateral zones:

Zone	Default filter
Thumb ↔ Index	🌈 Web comic
Index ↔ Middle	🎌 Manga screentone
Middle ↔ Pinky	🇺🇸 American pop

The three dropdowns in the bottom bar let you swap any zone's filter live.

Mode 2 — Thumb-index viewfinder + freeze-frame

Make a "finger-gun / OK" pose with both hands at the same time — only your thumb and index finger are extended, the other three fingers are curled in. The three zones collapse into a single live rectangle that follows the midpoints of each hand's thumb-and-index, filled with the current live filter.

To freeze a panel: pinch thumb and index together on both hands simultaneously.

1. Click. The current live rectangle is cloned into a frozen panel that stays put on the canvas.
2. The live filter automatically advances to the next one in the list.
3. Release, move your hands, pinch again → another frozen panel, another filter.
4. Pinch-debounced: one pinch = one panel. You must release before the next capture registers.

The filter cycle (loops forever): Poster Heat → Noir Ink → Manga Screentone → Anime Cel → American Pop → Web Comic → Riso Misprint → Blueprint Ink → Newspaper Halftone → Glitch Print.

Toolbar

Button / Shortcut	Effect
📸 Save PNG · S	Download the current composited canvas as a PNG
🧹 Clear frozen · C	Remove all frozen panels in one click
Debug wireframe	Overlay the detected quadrilateral outlines — useful for tuning your pose

---

🧭 Project layout

multiverse-hands/
├── index.html                     # Main page
├── src/
│   ├── app.js                     # Main loop, MediaPipe loader, UI wiring
│   ├── hand-geometry.js           # Hand normalization, three-zone quads, thumb-index frame
│   ├── effect-selection.js        # 10 filter definitions + default zone bindings
│   ├── frame-mode-state.js        # Frozen-frame state machine (pinch debounce + live cycle)
│   ├── webgl-renderer.js          # WebGL renderer with 10 shader branches
│   └── styles.css                 # Dark glass-morphism UI
├── vendor/tasks-vision/           # @mediapipe/tasks-vision 0.10.14 (bundle + wasm)
├── models/hand_landmarker.task    # HandLandmarker model (~7.5 MB, local)
└── test/                          # node --test unit tests (31 cases)

Load pipeline

src/app.js#loadHandTracker is defensive on purpose — the original upstream would silently hang if the CDN was slow or blocked:

1. Probe each candidate URL with fetch (8 s timeout) before dynamic import.
2. Multi-source fallback for bundle / wasm / model: local vendor → jsDelivr → unpkg → googleapis.
3. Per-step timeouts (20 s bundle · 20 s wasm · 30 s model). No silent hangs.
4. GPU → CPU delegate retry if the browser can't create the GPU inference graph.

Result on a fresh cache: Loading → Show both hands in ~6 s over 127.0.0.1 alone.

---

🙏 Credits · Derived from

This project is derived from SharkSkipper/SpiderVerse (originally named Finger Magic Lite). The upstream provides the hand geometry, frame-mode state machine, WebGL 10-shader renderer, and test suite.

Notable changes in this fork:

• Fully offline runtime — @mediapipe/tasks-vision@0.10.14 bundle + wasm + hand_landmarker.task are vendored under vendor/ and models/. The load pipeline is rewritten with probing, timeouts, multi-source fallback, and GPU→CPU retry.
• Localized UI — status prompts, region labels, and tool buttons are in Chinese; English README is here.
• New features — 📸 PNG export, 🧹 clear-frozen action, ⌨️ S/C shortcuts, 💬 on-canvas hint toasts, bottom-bar live effect + frozen-count indicators.
• Brand refresh — new title, dark glass-morphism control bar, footer attribution.

Thanks to the MediaPipe team for the HandLandmarker task, and to Spider-Man: Across the Spider-Verse for the visual inspiration.

---

📄 License

Released under MIT. Do whatever you want — please keep the copyright notice and attribution.