Real-time 3D pose tracking and visualization using MediaPipe BlazePose. Captures left arm joint positions (shoulder, elbow, wrist) with live 3D visualization and saves data to Parquet format.
- Real-time 3D Pose Tracking: Uses MediaPipe BlazePose for accurate 3D pose estimation
- Dual Visualization:
- 2D video feed with pose overlay (OpenCV)
- Real-time 3D spatial viewer (Matplotlib - macOS stable)
- macOS Optimized: Uses matplotlib for 3D visualization - no display issues
- Optimized Performance: Configurable frame rate, efficient processing pipeline
- Data Persistence: Saves joint positions to Parquet format with Snappy compression
- Interactive Playback: Replay and analyze captured motion with playback controls
- Color-coded Joints:
- Left Shoulder: Red
- Left Elbow: Green
- Left Wrist: Blue
- Conda (Miniconda or Anaconda)
- Webcam (for live capture)
- macOS, Linux, or Windows
-
Clone the repository:
git clone https://github.com/danielesiegel/3d-pose-tracker.git cd 3d-pose-tracker -
Create conda environment:
conda env create -f environment.yml
-
Activate environment:
conda activate pose-tracker
- Python: 3.10 (optimal compatibility)
- Key Dependencies:
- MediaPipe 0.10.9 (BlazePose model)
- OpenCV 4.8.1 (video capture)
- Matplotlib 3.7+ (3D visualization for capture - macOS stable)
- Open3D 0.18.0+ (3D visualization for viewer)
- Pandas 2.0+ + PyArrow 12.0+ (Parquet I/O)
- Click 8.1.7 (CLI framework)
Start capturing pose data from your webcam:
python src/capture.py --frames 30Options:
--frames,-f: Frames per second to capture (default: 30)--output,-o: Output directory for saved data (default:output/)--no-3d: Disable 3D visualization (useful for display issues on macOS)
Controls:
- Press
qto quit and save data
Examples:
# Capture at 60 FPS
python src/capture.py --frames 60
# Save to custom directory
python src/capture.py --frames 30 --output data/
# Disable 3D viewer (if experiencing display issues)
python src/capture.py --frames 30 --no-3dReplay and analyze saved pose data:
python src/viewer.py output/pose_data_20231129_153045.parquetOptions:
--speed,-s: Playback speed multiplier (default: 1.0)--loop,-l: Loop playback continuously--static: Show static trajectory view
Interactive Controls:
SPACE: Play/PauseR: Reset to beginningMouse: Rotate viewESC: Exit
Examples:
# Play at 2x speed
python src/viewer.py output/pose_data.parquet --speed 2.0
# Loop continuously
python src/viewer.py output/pose_data.parquet --loop
# Static trajectory view
python src/viewer.py output/pose_data.parquet --static3d-pose-tracker/
├── src/
│ ├── capture.py # Main capture script
│ └── viewer.py # Playback viewer
├── output/ # Saved Parquet files (auto-created)
├── data/ # Optional data directory
├── environment.yml # Conda environment specification
├── .gitignore # Git ignore rules
└── README.md # This file
Captured data is saved in Parquet format with the following schema:
| Column | Type | Description |
|---|---|---|
timestamp |
string | ISO 8601 timestamp |
frame |
int64 | Frame number |
shoulder_x |
float64 | Left shoulder X coordinate |
shoulder_y |
float64 | Left shoulder Y coordinate |
shoulder_z |
float64 | Left shoulder Z coordinate |
elbow_x |
float64 | Left elbow X coordinate |
elbow_y |
float64 | Left elbow Y coordinate |
elbow_z |
float64 | Left elbow Z coordinate |
wrist_x |
float64 | Left wrist X coordinate |
wrist_y |
float64 | Left wrist Y coordinate |
wrist_z |
float64 | Left wrist Z coordinate |
- MediaPipe uses a right-handed coordinate system
- Origin is at the center of the detected pose
- Units are in meters relative to hip center
- Z-axis points toward the camera
For researchers and developers looking to analyze human motion data, we provide output/datademo.parquet as a sample dataset.
What it contains: Real 3D kinematic data capturing human left arm motion over time. Each row represents a temporal snapshot (frame) of the spatial configuration of three skeletal keypoints (shoulder, elbow, wrist) as they move through 3D space.
Data represents: The geometric trajectory and spatial evolution of left arm joint positions during natural human movement, sampled at 30 Hz. This biomechanical time-series data encodes:
- Position vectors (x, y, z) for each joint in world coordinates (meters)
- Temporal sequence of arm configurations during motion
- Kinematic state suitable for calculating velocity, acceleration, and joint angles
- 3D spatial relationships between shoulder, elbow, and wrist throughout the motion
Use cases:
- Motion analysis and biomechanics research
- Gesture recognition algorithm development
- Kinematic modeling and animation
- Motor control pattern analysis
- Machine learning training data for pose prediction
# Quick start with the sample data
import pandas as pd
# Load sample motion data
df = pd.read_parquet('output/datademo.parquet')
# Each frame contains 3D positions of left arm joints
# Calculate joint angles, velocities, or train ML models
print(f"Captured {len(df)} frames of motion")
print(f"Duration: {len(df) / 30:.2f} seconds at 30 FPS")- Model Complexity: 2 (heavy model for best 3D accuracy)
- Landmark Format: 33 3D landmarks (COCO format)
- Left Arm Indices:
- Shoulder: 11
- Elbow: 13
- Wrist: 15
- Efficient frame processing with configurable FPS
- Non-blocking 3D visualization using threading
- Parquet with Snappy compression for fast I/O
- Minimal memory footprint
- 2D View: OpenCV window (640x480)
- 3D View (Capture): Matplotlib 3D renderer (macOS stable)
- Interactive rotation with mouse
- Auto-scaling view limits
- Color-coded joints and bones
- 3D View (Viewer): Open3D renderer for playback
- High-performance trajectory visualization
- Playback controls
If webcam doesn't open:
# Check available cameras (macOS/Linux)
ls /dev/video*
# Try different camera index in capture.py
cap = cv2.VideoCapture(1) # Change from 0 to 1If experiencing any 3D display issues:
# Run without 3D visualization
python src/capture.py --frames 30 --no-3dThe capture script now uses matplotlib which is stable on macOS. The --no-3d flag is available if you want to disable the 3D viewer entirely.
If experiencing lag:
- Reduce frame rate:
--frames 15 - Close other applications
- Ensure good lighting for better tracking
- Try disabling 3D view:
--no-3d
If packages are missing:
conda activate pose-tracker
conda env update -f environment.yml- Official Docs: https://google.github.io/mediapipe/solutions/pose
- BlazePose Paper: BlazePose: On-device Real-time Body Pose tracking
- Landmark Guide: https://google.github.io/mediapipe/solutions/pose#pose-landmark-model-blazepose-ghum-3d
- Documentation: http://www.open3d.org/docs/
- Visualization: http://www.open3d.org/docs/latest/tutorial/visualization/visualization.html
- Format Spec: https://parquet.apache.org/docs/
- Python Integration: https://arrow.apache.org/docs/python/parquet.html
MIT License - feel free to use and modify for your projects.
Contributions welcome! Please open issues or pull requests.
Potential additions:
- Full body tracking (all 33 landmarks)
- Multiple person support
- Export to additional formats (CSV, JSON, HDF5)
- Real-time metrics (joint angles, velocity)
- Web-based viewer using Three.js
- Video file input support
- GPU acceleration for processing
- Google MediaPipe team for BlazePose
- Open3D contributors
- Apache Arrow/Parquet communities
Disclaimer: This project was created in part with AI-assisted code and is still a work in progress.