LitSpaR is a multimodal large language model (MLLM) that performs 3D spatial reasoning exclusively from point clouds. With only 580M parameters, it achieves competitive performance on VSI-Bench while keeping general QA abilities and supporting semantic object insertion for AR/VR interactions.
- Point-cloud-only input — a Point Transformer V3 encoder + a two-layer MLP adapter + an LLM backbone, no images, video, or camera calibration required.
- Grounding-then-reasoning — first localizes the objects of interest, then reasons about their geometric relationships.
- Two-stage curriculum training — large-scale synthetic pre-training (SpatialLM-based), then real-world fine-tuning (ScanNet, ARKitScenes, ScanNet++).
LitSpaR is evaluated on an adapted VSI-Bench where videos are replaced by point clouds (6 of 8 tasks). Models marked with * take video as input; LitSpaR uses point clouds only.
| Model | Obj. Cnt. | Abs. Dist. | Obj. Size | Room Size | Rel. Dist. | Rel. Dir. (e./m./h.) |
|---|---|---|---|---|---|---|
| GPT-4o* | 46.2 | 5.3 | 43.8 | 38.2 | 37.0 | 41.3 |
| Gemini-1.5 Pro* | 45.4 | 56.2 | 30.9 | 64.1 | 43.6 | 51.3 |
| LLaVA-NeXT-Video-72B* | 48.9 | 22.8 | 57.4 | 35.3 | 42.4 | 36.7 |
| GS-Reasoner (pred. depth)* | 69.1 | 61.9 | 70.0 | 65.7 | 65.4 | 88.9 |
| GS-Reasoner (GT depth)* | 70.9 | 73.6 | 77.8 | 81.8 | 70.6 | 90.5 |
| LitSpaR (w/o synthetic) | 55.5 | 38.8 | 62.5 | 84.9 | 35.4 | 48.6 |
| LitSpaR (ours) | 71.6 | 68.9 | 78.3 | 84.8 | 57.5 | 86.8 (89.9 / 85.4 / 85.3) |
LitSpaR retains general question-answering ability on point cloud scenes: (GPT is LitSpaR)
LitSpaR places virtual objects in 3D scenes from natural-language instructions, enabling a plane-detection-free interaction for AR/VR:
 |
 |
| Put it under the dining table | Put it on the foot of the bed |
@misc{han2026litspar,
title = {LitSpaR: A Lightweight Spatial Reasoning Model for Indoor Scene Understanding},
author = {Han, Hui and Jiang, Zaifan and Wei, Chao},
year = {2026}
}Hui Han — clearhanhui@outlook.com