Skip to content

Home

Jump to bottom
Dominik I. Braun edited this page Jan 14, 2025 · 8 revisions

Virtual Hand Interface

Developed by Dominik I. Braun.

Hand configurations

The hands are provided by the HTC VIVE SDK, HTC Corporation, New Taipei, Taiwan. They consist of 26 configurable bones (controllable in bold):

  1. Palm
  2. Wrist
  3. Thumb Root: Carpometacarpal (CMC)
  4. Thumb joint1: Metacarpophalangeal (MCP)
  5. Thumb joint2: Interphalangeal (IP)
  6. Thumb tip
  7. Index Root
  8. Index joint1: Metacarpophalangeal (MCP)
  9. Index joint2: Proximal interphalangeal (PIP)
  10. Index joint3: Distal interphalangeal (DIP)
  11. Index tip
  12. Middle root
  13. Middle joint1: Metacarpophalangeal (MCP)
  14. Middle joint2: Proximal interphalangeal (PIP)
  15. Middle joint3: Distal interphalangeal (DIP)
  16. Middle tip
  17. Ring root
  18. Ring joint1: Metacarpophalangeal (MCP)
  19. Ring joint2: Proximal interphalangeal (PIP)
  20. Ring joint3: Distal interphalangeal (DIP)
  21. Ring tip
  22. Pinky root
  23. Pinky joint1: Metacarpophalangeal (MCP)
  24. Pinky joint2: Proximal interphalangeal (PIP)
  25. Pinky joint3: Distal interphalangeal (DIP)
  26. Pinky tip

In the virtual hand interface, all moveable joints are configured for their respective range of motion:

  • x: Extension/Flexion
  • y: Supination/Pronation
  • z: Abduction/Adduction

List on predefined control movements:

  • Rest
  • Thumb flexion
  • Index flexion
  • Middle flexion
  • Ring flexion
  • Pinky flexion
  • Power grasp
  • Pinch
  • Tripod pinch

Communication Interface

The communication with the virtual hand interface is based on the WiFi User Diagramm Protocol (UDP) on the "localhost" IP address. The transmitted bytes should be encoded in UTF-8.

  • Virtual hand interface client port: 1236
  • External client port: 1233
  • External client port (predicted hand states): 1234

To validate the successful communication, the external client can send a "status request" to the virtual hand interface.

  • status request: "status"

The virtual hand interface responds then with a "status response".

  • status response: "active"

Input

The input to the virtual hand interface should be decoded in UTF-8 and the input values should be in a string list format. The list should hold at least 9 values that control the predicted hand (PH) with optional 9 values for the control hand (CH):

  • UTF-8 Encoded:
    "[PH1], [PH2], [PH3], [PH4], [PH5], [PH6], [PH7], [PH8], [PH9]" +
    "[CH1], [CH2], [CH3], [CH4], [CH5], [CH6], [CH7], [CH8], [CH9]" (Optional)

The individual elements describe the following:

  • XH1: Thumb Extension [-1] - Flexion [1]
  • XH2: Thumb Abduction [-1] - Adduction [1]
  • XH3: Index Extension [-1] - Flexion [1]
  • XH4: Middle Extension [-1] - Flexion [1]
  • XH5: Ring Extension [-1] - Flexion [1]
  • XH6: Pinky Extension [-1] - Flexion [1]
  • NOT DEFINED -> Wrist Extension [-1] - Flexion [1] planned
  • NOT DEFINED -> Wrist Abduction [-1] - Adduction [1] planned
  • NOT DEFINED -> Wrist Supination [-1] - Pronation [1] planned

If the "Use Control Stream" toggle is not activated, the optional values are ignored!

Note: Currently only the control in flexion direction [0...1] works properly.

Examples:

  1. Fist: "[0.67], [1], [1], [1], [1], [1], [0], [0], [0]"
  2. Pinch: "[0.45], [1], [0.6], [0], [0], [0], [0], [0], [0]"
  3. Tripod Pinch: "[0.55], [1], [0.65], [0.65], [0], [0], [0], [0], [0]"

Output

Besides the status response, the virtual hand interface sends the current state of the control hand (on port 1233) and predicted hand (on port 1234) with a frequency of 60 Hz: "[CH1], [CH2], [CH3], [CH4], [CH5], [CH6], [CH7], [CH8], [CH9]"

Settings

Control Hand Settings

  • "Speed": Describes the frequency (in Hz) of a full kinematic cycle (e.g., flexion - extension cycle)
  • "Holding time": Describes the duration (in s) of the holding phase at the extremas of the kinematic cycle (e.g., holding duration of the flexed state).
  • "Resting time": Describes the duration (in s) of the resting phase
  • "Use Control Stream": If toggled, allows the control of both control and predicted hand from the external UDP connection. For this, the control hand inputs should be sent concatenated to the predicted hand inputs (as described in Input).

Predicted Hand Settings

  • "Select Prediction Method"
    • "Classification": The Virtual Hand Interface interpolates from the previous Input to the current Input with the provided "Transition Speed". This allows a smooth transition between two different states (e.g., rest to power grasp)
    • "Regression": The Virtual Hand Interface sets the hand position directly from the Input.
  • "Transition Speed": Given speed with which it interpolates between different inputs. Only applicable for the "Classification" method.

Visibility Settings

  • "Hide Control Hand": Hides the control hand from the viewport.
  • "Hide Predicted Hand": Hides the predicted hand from the viewport.

Contact

  • Dominik I. Braun - dome.braun@fau.de
    Chair of Neuromuscular Physiology and Neural Interfacing | n-squared lab
    Department Artificial Intelligence in Biomedical Engineering | AIBE
    Friedrich-Alexander-Universität Erlangen-Nürnberg | FAU
  • Alessandro Del Vecchio - alessandro.del.vecchio@fau.de
    Chair of Neuromuscular Physiology and Neural Interfacing | n-squared lab
    Department Artificial Intelligence in Biomedical Engineering | AIBE
    Friedrich-Alexander-Universität Erlangen-Nürnberg | FAU

Clone this wiki locally