PCXVQA: Person-Centric Cross-Video Question Answering Benchmark

PCXVQA targets cross-video reasoning over synchronized multi-view streams, encompassing eight fine-grained subtasks that assess multi-view synchronization, cross-camera trajectory alignment, and statistical reasoning capabilities in realistic multi-camera surveillance scenarios.

Data Source

PCXVQA is built upon the MultiSensor-Home dataset, which provides synchronized multi-view recordings from multiple cameras distributed across household areas (e.g., kitchen, living room, balcony). The benchmark contains:

• 1,955 synchronized multi-view videos
• 3,383 annotated question-answer pairs

Note: This repository only provides QA annotations. The video data must be downloaded separately.

Download the MultiSensor-Home videos from Hugging Face:

• Dataset page: https://huggingface.co/datasets/thanhhff/MultiSensor-Home1/

A simple way to download the dataset:

# Make sure hf CLI is installed:
# pip install -U "huggingface_hub[cli]"

hf download thanhhff/MultiSensor-Home1 --repo-type=dataset --local-dir dataset

Task Taxonomy

PCXVQA is organized into three major categories with eight subcategories:

Perception (PER)

• PER-Ins (Instantaneous Grounding): localize a person in space at a specific time using multiple views.
• PER-Act (Action Classification): recognize fine-grained actions with cross-view evidence.

Temporal (TMP)

• TMP-Bnd (Boundary Localization): align start/end times of actions across cameras.
• TMP-Seq (Sequential Reasoning): infer ordered action sequences across rooms.

Aggregation (AGG)

• AGG-Loc (Location Statistics): aggregate location frequencies/durations.
• AGG-Dur (Duration Measurement): compute total time spent in a state/action.
• AGG-Sta (State Inference): infer higher-level states (e.g., working vs. leisure).
• AGG-Dom (Dominant Activity): identify the most frequent/important activity.

Files

• fullset.json: full QA set, answer is null for closed evaluation.
• subset.json: public subset (500 QA pairs, answers visible).
• README.md: this document.

subset.json is produced by stratified sampling by subcategory from the full set with real answers (fixed random seed: 2026).

Citation

If you use PCXVQA, please cite our paper and the original MultiSensor-Home dataset.

VideoMAPS: This work is currently under review. The official citation will be updated upon acceptance.

MultiSensor-Home:

@inproceedings{nguyen2025multisensor,
  author    = {Trung Thanh Nguyen and Yasutomo Kawanishi and Vijay John and Takahiro Komamizu and Ichiro Ide},
  title     = {MultiSensor-Home: A Wide-area Multi-modal Multi-view Dataset for Action Recognition and Transformer-based Sensor Fusion},
  booktitle = {Proceedings of the 19th IEEE International Conference on Automatic Face and Gesture Recognition},
  year      = {2025},
  note      = {Best Student Paper Award}
}

Usage and License

• The QA annotations in this repository are released for research use.
• Please follow the license and usage terms of the original MultiSensor-Home dataset for the video files.