Gait recognition ToF camera

General software:

⚙️ Technical Summary

1. Gait Parameters

Two primary kinematic parameters were extracted for gait recognition using the Intel RealSense D455 camera:

• Waist rotation angle relative to the camera frame.
• Hip joint angle, defined by the leg’s motion with respect to the hip.

These measurements were computed using 3D point-cloud data generated by the RealSense D455 Time-of-Flight (ToF) camera. The frontal-view configuration of the system allowed continuous monitoring of the subject’s hip movement, crucial for analyzing gait patterns in patients with mobility impairments.

2. Model Development

The model was developed through several iterative stages, each addressing specific challenges related to accuracy, real-time performance, and environmental adaptability:

• Initial approach: Simple RGB image cropping to identify the waist region and compute average point-cloud distances. This required strict subject positioning and was unsuitable for dynamic environments.
• Marker-based approach: Visual markers placed on the waist, tracked via RGB input. Although conceptually sound, this method was computationally slow and unreliable for real-time use.
• Pose Estimation approach: Tested MATLAB’s built-in OpenPose and custom-trained neural networks for joint detection. These models lacked precision in close-range frontal views and were highly sensitive to lighting and occlusion.
• Final approach: Point-cloud-only processing. The waist region is dynamically segmented from depth data by filtering relevant 3D regions, removing background noise, and computing hip joint coordinates directly from the spatial point distribution. This method provided the best trade-off between accuracy, speed, and robustness.

3. RealSense Configuration and MATLAB Implementation

The project was implemented entirely in MATLAB using the RealSense SDK for sensor communication and 3D processing. The workflow included:

• Pipeline setup: Establishing bidirectional communication between the RealSense D455 and MATLAB to acquire RGB and depth frames.
• Point-cloud transformation: Converting depth data into 3D coordinates, applying spatial transformations to align the camera’s coordinate frame with the ground.
• ROI filtering: Selecting the waist region dynamically through findPointsInROI and averaging points to extract body keypoints.
• Visualization: Implementing real-time plotting with pcplayer and animatedline to monitor both spatial and angular changes over time.

Two main implementations were created:

• waistEstimator_automatic: an automated function for recursive waist region segmentation and analysis.
• depth_view.m: an App Designer interface enabling manual control of the ROI and real-time visual feedback for both 3D and time-domain data.

---

🧩 Performance Evaluation

Static Performance

Static tests were conducted with a mannequin positioned at distances between 20 cm and 70 cm from the camera. For each configuration, 1000 frames were analyzed to assess accuracy and repeatability.

Key findings:

• The distance measurement of the waist from the camera was highly accurate, with minimal deviation across frames.
• Waist angle estimation showed variability at larger distances due to reduced point density and sensor noise.
• The system performed best within the 30–50 cm range, providing reliable measurements for clinical gait tracking applications.

Real-Time Performance

The real-time prototype was evaluated with the camera mounted on a moving cart, simulating its intended clinical use. Tests were carried out both with the cart pushed and pulled by different subjects.

Observed outcomes:

• The Y Position (depth distance) remained stable during motion, confirming that the system can maintain tracking consistency even when the subject moves irregularly.
• The Angle Position (waist rotation) oscillated periodically, accurately reflecting leg alternation during walking cycles.
• Right and left hip joint angles displayed coherent periodic patterns matching gait symmetry expectations.

Additional tests showed that the algorithm:

• Was robust to moderate clothing variations and environmental clutter.
• Effectively filtered out background objects through dynamic ROI selection.
• Maintained real-time performance with minimal latency in visualization and computation.

Summary of Performance Characteristics

Parameter	Static Accuracy	Dynamic Robustness	Sensitivity
Waist Distance	High	Stable	Low to clothing
Waist Angle	Moderate	Coherent patterns	Sensitive to noise at >60 cm
Real-Time Tracking	N/A	Smooth, low latency	Robust to environment
Processing Speed	~30 FPS	~30 FPS	Stable under normal lighting

---

Conclusion:
The combination of Intel RealSense D455 depth sensing and MATLAB-based point-cloud analysis enables a practical and efficient gait monitoring system suitable for rehabilitation and movement analysis. The final implementation achieved reliable real-time operation and sufficient precision for clinical and research use cases.