H2H-Plugin

Filippo Paris · Francesco Moretti · Giorgio Mattina · Luca Trapella

Table of Contents

1. Overview
2. Key Features
3. How to Install
4. Getting Started
5. Visual Preview
6. License & Usage Terms

Overview

H2H-Plugin is a hands-free, gesture-driven audio performance system that leverages the Leap Motion Controller to put full expressive power at your fingertips—literally. Using intuitive hand motions, you can seamlessly switch between four built-in synthesizers (Vangelis, AcidBass, EvolvingPad, Pluck) with your left hand, and sculpt their sound parameters in real time by moving along the X, Y, Z axes and rolling your wrist. Meanwhile, your right hand controls an effects rack featuring Low/High-Pass filtering, Delay, Distortion, and Flanger/Tremolo. Simply clench your fist to “freeze” all parameter values at their current settings—a perfect way to lock in a sound and improvise rhythm or melody without accidental tweaks. H2H-Plugin transforms gestural performance into a fully immersive, instrument-like experience.

Key Features

• Gesture-Based Synth Selection
  Switch instantly between four distinctive synth engines—Vangelis, AcidBass (TB-303 style), EvolvingPad, and Pluck—by extending 1–4 fingers on your left hand.

• Real-Time Parameter Mapping
  Map the X, Y, Z positions and roll of each hand to up to four assignable parameters per synth or effect. Sculpt filters, envelopes, LFO rates, drive, reverb mix, arpeggio patterns, and more—live, with no mouse or keyboard.

• Dynamic Effects Rack
  Use your right hand to choose one of four high-quality effects (Low/High-Pass, Delay, Distortion, Flanger/Tremolo) and manipulate effect parameters in the same intuitive X/Y/Z/roll fashion.

• Instant Freeze Control
  Make a fist to lock (“freeze”) all current parameter values. This lets you hold complex sound states while you solo or jam, then unclench to resume gestural control.

• Seamless MIDI Integration
  Under the hood, synth voices respond to MIDI Note On/Off messages—so you can play with any standard MIDI keyboard or pad controller while still enjoying hands-free gestural tweaking.

• Modular & Extensible
  Built in SuperCollider with a Python/OSC front end driving the Leap Motion, the plugin’s modular design makes it easy to add new synths, effects, or custom gesture mappings.

How to Install

There are two modes in which you can use the system:

• Through the Leap Motion Sensor
• With a regular mouse/trackpad

In any case you wil need a MIDI controller.

If you have the Leap Motion controller, you will have to run an additional python script, which will send the data from the controller to the Processing instance responsible of the GUI rendering.

If you don't have the controller, you can still interact with the system. In any case, follow these installation steps and ignore the last one if you don't wish to use the Leap Motion controller:

1. SuperCollider

1. Download and install SuperCollider:
   https://supercollider.github.io/download
2. Open the SuperCollider IDE and load main_supercollider.scd.
3. Install the VSTPlugin quark in the SC IDE post window:

   Quarks.install("VSTPlugin");

4. Restart the IDE, then in the editor evaluate: s.reboot;
5. Evaluate the entire file to load all SynthDefs, OSC handlers, MIDI setup and the VST host patch.

2. Processing + oscP5

1. Download and install Processing: https://processing.org/download

2. Open main_processing.pde in the Processing IDE.

3. In the menu go to Sketch → Import Library → Add Library…, search for oscP5, and click Install.

4. Run the sketch. You should see OSC messages printed from your Leap Motion Controller.

3. JUCE Audio Plugin (“H2HEffects”)

1. Pull the VST3
   After cloning , unzip VST Precompiled-Plugin and place the correct .vst3 folder (according to your OS) in your DAW’s standard VST3 folder:

   • macOS: ~/Library/Audio/Plug-Ins/VST3/
   • Windows: C:\Program Files\Common Files\VST3\
   • Linux: ~/.vst3/

2. Verify plugin is visible
   In SuperCollider’s post window, run:

   VSTPlugin.search;
   

3. Open the plugin editor After running the previous code, open the editor by running:

   ~h2hController.editor;  // pops up the plugin GUI
   

4. Leap Motion Controller Setup

To use H2H-Plugin, you'll need a Leap Motion Controller. Here's how to acquire and set it up:

1. Purchase the Leap Motion Controller: Visit the official Ultraleap website to purchase the Leap Motion Controller: Ultraleap Store

2. Install the Ultraleap Hand Tracking Software: After receiving your device, download and install the appropriate software for your operating system from the same page.

3. Connect the Device: Plug the Leap Motion Controller into a USB port on your computer. Ensure it's placed on a flat surface facing upward for optimal tracking.

4. Download Python 3.8 or later: Download a recent version of Python, on MacOs, you must use Python 3.8 exactly.

5. Create a virtual environment with the correct python version:

6. Load the repository "leapc-python-bindings": Once inside this repository directory type this in the terminal

   git submodule update --init --recursive
   

7. Install the dependencies

   You need to have the Leap SDK installed in the default location. If this is not the case for you, you can use an environment variable to define the installation location. Define the environment variable LEAPSDK_INSTALL_LOCATION to the path of the LeapSDK folder, if you have installed to a custom location or moved it somewhere else.

   Example: export LEAPSDK_INSTALL_LOCATION="C:\Program Files\CustomDir\Ultraleap\LeapSDK"

   By default, this path is the following for each operating system:

   • Windows: C:/Program Files/Ultraleap/LeapSDK
   • Linux x64: /usr/lib/ultraleap-hand-tracking-service
   • Linux ARM: /opt/ultraleap/LeapSDK

   Once you have the SDK run the following in this repository's directory:

   pip install -r leapc-python-bindings/requirements.txt
   pip install -e leapc-python-bindings/leapc-python-api

   You can read the documentation of the leapc-python-bindings submodule by clicking on this link

8. Install python-osc package

pip install python-osc

You should now be able to run main_python.py either with a Python extension in your IDE or in the terminal:

python3 main_python.py

You now have four hands-free synths (Vangelis, AcidBass, EvolvingPad, Pluck) and four live-controlled effects (Low/High Pass, Delay, Distortion, Flanger/Tremolo) at your fingertips!

Getting Started

1. Prepare your gear

   • Plug in a MIDI keyboard (or controller) to your computer.
   • Connect the Leap Motion USB sensor and mount it facing your hands.

2. Launch the software

   0. Python
      • If you wish to use the controller, open Ultraleap and run the python script in main_python.py
   1. DAW or Host
      • Load the H2HEffects VST3 into your favorite host (Ableton Live, Reaper, Logic, etc.).
      • Verify the plugin GUI pops up and your Leap-driven controls are visible.
   2. Processing
      • Open main_processing.pde, install oscP5 if prompted, then hit ▶ to start sending OSC to SC.
   3. SuperCollider
      • Open main_supercollider.scd and run the whole file.
      • You should see SynthDefs compile and OSC handlers print “Left Hand” / “Right Hand” messages.

Play & perform

• Left hand (1–4 fingers extended) → select one of the four synths.
• Left hand axes & roll → morph the selected synth’s parameters in real time.
• Right hand → choose and tweak one of four effects (filter, delay, distortion, flanger/tremolo).
• Make a fist with either hand to freeze its current parameter values.
• Meanwhile, play notes on your MIDI keyboard to trigger and express your chosen synth.

You can perform solo (one person on keys + Leap) or duet (keyboardist + gestural sound designer). Enjoy the hands-free, highly expressive interface of H2H-Plugin!

Visual Preview

Leap Motion Controller

GUI

License & Usage Terms

H2H-Plugin © 2025 All Rights Reserved.

No part of this project may be reproduced or used without permission.