The VI^2-Tracking Dataset contains synchronized visual and inertial measurements from a four-camera VR headset and two tracked controllers, together with the recorded headset pose and motion-capture ground truth for both controllers. This public release contains nine two-controller sequences, named D01-D09.
Status: The accompanying paper, Visual-Inertial and Inertial-Object Tracking: Analysis and Algorithm, has been accepted by the IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS) 2026. The current dataset release is
v1.0.0.
The data were collected indoors with a Pimax Crystal Light headset. The IMU reported in the paper is a TDK-42686P sampled at 1000 Hz, and the reference controller trajectories were recorded with an FZMotion motion-capture system.
Only raw sensor streams, calibration needed to interpret them, controller geometry, the recorded headset pose, and authoritative motion-capture trajectories are included. Device serial numbers, acquisition-date metadata in calibration/configuration files, videos, temporary files, debug logs, maps, and algorithm-generated trajectories have been removed.
If you use this dataset in an academic publication, please cite the paper in Citation.
The accepted manuscript is available at paper/IROS2026_VI2_Tracking.pdf.
The dataset is distributed as nine sequence archives in the
dataset-v1.0.0 GitHub Release. Each archive is self-contained and can be downloaded independently:
VI2-Tracking-D01-v1.0.0.zip
VI2-Tracking-D02-v1.0.0.zip
...
VI2-Tracking-D09-v1.0.0.zip
Download SHA256SUMS and verify every archive before extraction. manifest.json also records the archive size, SHA-256 digest, file count, and uncompressed byte count for each sequence. Extract the selected archives into a common parent directory to reconstruct the D01-D09 layout.
VI2-Tracking-Dataset/
|-- README.md
|-- LICENSE-DATASET
|-- D01/
| |-- config/
| | |-- left_controller.json
| | `-- right_controller.json
| |-- device_calibration.yaml
| |-- images/{0,1,2,3}/
| |-- ir_images/{0,1,2,3}/
| |-- imu_hmd.txt
| |-- imu_left.txt
| |-- imu_right.txt
| |-- hmd_pose.txt
| `-- ground_truth/
| |-- left_controller.txt
| `-- right_controller.txt
|-- D02/
| `-- ...
`-- D09/
`-- ...
Each sequence is self-contained. The headset calibration is identical in all nine sequences, while controller configuration is retained per sequence because the two hands are calibrated independently and one sequence contains a controller-specific geometry revision.
The public identifiers and trajectory statistics below are those reported in the IROS 2026 paper. Every sequence contains both a left and a right controller.
| Sequence | Left length (m) | Left duration (s) | Right length (m) | Right duration (s) |
|---|---|---|---|---|
| D01 | 57.004 | 79.875 | 76.284 | 79.875 |
| D02 | 74.708 | 70.725 | 89.661 | 70.725 |
| D03 | 23.728 | 70.200 | 24.881 | 70.200 |
| D04 | 28.424 | 73.550 | 25.735 | 73.550 |
| D05 | 37.122 | 87.950 | 37.558 | 87.725 |
| D06 | 47.708 | 34.450 | 45.550 | 34.450 |
| D07 | 111.162 | 46.000 | 95.275 | 46.000 |
| D08 | 60.643 | 49.383 | 68.028 | 48.533 |
| D09 | 32.946 | 44.283 | 38.675 | 44.100 |
The sequence IDs are intentionally independent of recording dates and device serial numbers.
images/<camera_id>/ and ir_images/<camera_id>/ contain grayscale JPEG frames from cameras 0-3. ir_images is the infrared image collection used for controller LED tracking; images is the companion recorded image collection.
Each filename has the form:
<timestamp_ns>.jpg
timestamp_ns is a zero-padded integer sensor timestamp in nanoseconds. Parse it as an integer and sort numerically. Within either image collection, matching filenames in camera directories 0-3 belong to the same four-camera capture.
The release intentionally contains no sequences.txt. Consumers should enumerate image filenames and merge image, IMU, and pose streams by timestamp.
The three IMU files use one sample per line:
timestamp_ns gyro_x gyro_y gyro_z accel_x accel_y accel_z
imu_hmd.txt: headset IMU.imu_left.txt: left-controller IMU.imu_right.txt: right-controller IMU.- Angular velocity is in radians per second.
- Linear acceleration is in metres per second squared.
- Vectors are expressed in the native IMU frame of the corresponding device.
The first field is an integer nanosecond timestamp in the dataset sensor-clock domain; it is not a Unix timestamp.
hmd_pose.txt uses one pose per line:
timestamp_ns tx ty tz qx qy qz qw
Translation is in metres and quaternion coefficients use Hamilton order (x, y, z, w). The file gives the headset IMU/body pose in the dataset tracking frame and is the headset-pose input used with the controller observations.
Controller ground truth uses the TUM pose format:
timestamp_s tx ty tz qx qy qz qw
ground_truth/left_controller.txt: left-controller body pose.ground_truth/right_controller.txt: right-controller body pose.
The first column is time in seconds, translation is in metres, and quaternion coefficients use order (x, y, z, w). Each trajectory describes the corresponding controller body/device frame in the motion-capture world frame. These body trajectories are the authoritative references for evaluation; historical tracker outputs and raw marker-frame intermediates are not part of this release.
To associate sensor data with ground truth, convert nanosecond timestamps to seconds and use an application-appropriate timestamp tolerance.
Each device_calibration.yaml uses a EuRoC-style homogeneous-matrix representation and contains only the four camera models and their transforms to the headset IMU/body frame.
For every cam0-cam3 entry:
resolutionis[width, height];intrinsicsis[fx, fy, cx, cy];distortion_coefficientsis[k1, k2, k3, k4, p1, p2];T_BSis a row-major homogeneous transform from camera sensor frameSto headset body/IMU frameB.
The transform convention is:
p_B = T_BS * p_S
The camera model is pinhole with the project's six-coefficient KB8-plus-tangential distortion model. Do not interpret the six values as EuRoC's four-coefficient equidistant model merely because the YAML matrix layout is EuRoC-style.
config/left_controller.json and config/right_controller.json contain only the geometry used by the tracking algorithm:
{
"hand": "Left",
"imu_to_device": {
"rotation": [[r00, r01, r02], [r10, r11, r12], [r20, r21, r22]],
"translation_m": [tx, ty, tz]
},
"leds": [
{
"id": 0,
"position_m": [x, y, z],
"normal": [nx, ny, nz]
}
]
}The transform convention is:
p_device = R_imu_to_device * p_imu + t_imu_to_device
LED positions and unit normals are expressed in the controller/device frame. Positions are in metres and normals are dimensionless. Each controller has 15 LED entries with IDs 0-14.
The two hands must be treated independently; do not create one hand by mirroring the other. Always use the left and right configuration shipped with the sequence. In particular, the right-controller geometry in D08 contains a sequence-specific point value and must not be replaced by another sequence's configuration.
This release uses the cleaned JSON and YAML schemas above and intentionally omits sequences.txt. A legacy loader that requires serial-number subdirectories, device_calibration.xml, or sequences.txt must be updated to enumerate timestamped files and read these public schemas.
The VI^2-Tracking Dataset is licensed under the
Creative Commons Attribution-NonCommercial 4.0 International License (CC BY-NC 4.0). Commercial use is prohibited. See LICENSE-DATASET for the complete license text.
You must provide appropriate attribution, link to the license, and indicate whether changes were made. The license does not remove any citation obligation stated below.
The paper PDF is provided separately and is not covered by the dataset license.
If you use the VI^2-Tracking Dataset, please cite:
@inproceedings{chen2026visual,
title = {Visual-Inertial and Inertial-Object Tracking: Analysis and Algorithm},
author = {Chen, Chang and Xia, Ruican and Li, Xiangfu and Huang, Guoquan},
booktitle = {IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS)},
year = {2026}
}