Awesome-spatial-visual-reasoning-MLLMs

Repository for awesome spatial/visual reasoning MLLMs. (focus more on embodied applications)

Image

1. [arxiv 2506] VGR: Visual Grounded Reasoning [Datasets]
2. [arxiv 2505] Tool-Star: Empowering LLM-Brained Multi-Tool Reasoner via Reinforcement Learning [Code]
3. [arxiv 2505] Active-o3 : Empowering Multimodal Large Language Models with Active Perception via GRPO [Code]
4. [arxiv 2505] One RL to See Them All: Visual Triple Unified Reinforcement Learning [Code] [Dataset]
5. [arxiv 2505] DeepEyes: Incentivizing “Thinking with Images” via Reinforcement Learning [Code] [Dataset]
6. [arxiv 2505] R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO [Code]
7. [arxiv 2505] Think or Not? Selective Reasoning via Reinforcement Learning for Vision-Language Models [Code] [Datasets]
8. [arxiv 2505] Pixel Reasoner: Incentivizing Pixel-Space Reasoning with Curiosity-Driven Reinforcement Learning [Datasets] [Code]
9. [arxiv 2505] VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction [Code]
10. [arxiv 2025] G1: Bootstrapping Perception and Reasoning Abilities of Vision-Language Model via Reinforcement Learning [Code]
11. [arxiv 2505] SophiaVL-R1: Reinforcing MLLMs Reasoning with Thinking Reward [Code] [Dataset]
12. [arxiv 2505] VisionReasoner: Unified Visual Perception and Reasoning via Reinforcement Learning [Code] [Dataset]
13. [arxiv 2505] GuardReasoner-VL: Safeguarding VLMs via Reinforced Reasoning [Code]
14. [arxiv 2505] Multi-SpatialMLLM Multi-Frame Spatial Understanding with Multi-Modal Large Language Models [Code]
15. [arxiv 2505] GRIT: Teaching MLLMs to Think with Images [Code]
16. [arxiv 2505] Visual Agentic Reinforcement Fine-Tuning [Code] [Dataset]
17. [arxiv 2505] OpenThinkIMG: Learning to Think with Images via Visual Tool Reinforcement Learning [Code] [Dataset]
18. [arxiv 2504] Skywork R1V2: Multimodal Hybrid Reinforcement Learning for Reasoning [Code]
19. [arxiv 2504] SRPO: A Cross-Domain Implementation of Large-Scale Reinforcement Learning on LLM [HuggingFace]
20. [arxiv 2504] Perception-R1: Pioneering Perception Policy with Reinforcement Learning [Code] [Dataset]
21. [arxiv 2504] SoTA with Less: MCTS-Guided Sample Selection for Data-Efficient Visual Reasoning Self-Improvement [Code] [Datasets]
22. [arxiv 2504] VLM-R1: A stable and generalizable R1-style Large Vision-Language Model [Code] [Dataset]
23. [arxiv 2504] VLAA-Thinking: SFT or RL? An Early Investigation into Training R1-Like Reasoning Large Vision-Language Models [Code] [Dataset]
24. [arxiv 2504] MAYE: Rethinking RL Scaling for Vision Language Models: A Transparent, From-Scratch Framework and Comprehensive Evaluation Scheme [Code] [Dataset]
25. [arxiv 2504] R1-SGG: Compile Scene Graphs with Reinforcement Learning [Code]
26. [arxiv 2504] NoisyRollout: Reinforcing Visual Reasoning with Data Augmentation [Code] [Datasets]
27. [arxiv 2504] VL-Rethinker: Incentivizing Self-Reflection of Vision-Language Models with Reinforcement Learning [Code] [Dataset]
28. [arxiv 2503] Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning [Code] [Dataset]
29. [arxiv 2503] MM-Eureka: Exploring Visual Aha Moment with Rule-based Large-scale Reinforcement Learning [Code] [Dataset]
30. [arxiv 2503] Visual-RFT: Visual Reinforcement Fine-Tuning [Code] [Datasets]
31. [arxiv 2503] OpenVLThinker: An Early Exploration to Vision-Language Reasoning via Iterative Self-Improvement [Code]
32. [arxiv 2503] Think or Not Think: A Study of Explicit Thinking in Rule-Based Visual Reinforcement Fine-Tuning [Code] [Datasets]
33. [arxiv 2503] R1-VL: Learning to Reason with Multimodal Large Language Models via Step-wise Group Relative Policy Optimization [Code]
34. [arxiv 2503] Skywork R1V: Pioneering Multimodal Reasoning with Chain-of-Thought [Code]
35. [arxiv 2503] R1-Onevision: Advancing Generalized Multimodal Reasoning through Cross-Modal Formalization [Code] [Dataset]
36. [arxiv 2503, CVPR'25] CoSpace: Benchmarking Continuous Space Perception Ability for Vision-Language Models [Code]
37. [arxiv 2503]VisualPRM: An Effective Process Reward Model for Multimodal Reasoning [Code] [Dataset]
38. [arxiv 2503] LMM-R1: Empowering 3B LMMs with Strong Reasoning Abilities Through Two-Stage Rule-Based RL [Code]
39. [arxiv 2503] Curr-ReFT: Boosting the Generalization and Reasoning of Vision Language Models with Curriculum Reinforcement Learning [Code] [Dataset]
40. [arxiv 2503] Vision-R1: Incentivizing Reasoning Capability in Multimodal Large Language Models [Code]
41. [arxiv 2502, NAACL'25 findings] Mitigating Hallucinations in Multimodal Spatial Relations through Constraint-Aware Prompting [Code]
42. [arxiv 2502] VADAR: Visual Agentic AI for Spatial Reasoning with a Dynamic API [Code] [Dataset]
43. [arxiv 2502] Introducing Visual Perception Token into Multimodal Large Language Model [Code]
44. [arxiv 2501] Mulberry: Empowering MLLM with o1-like Reasoning and Reflection via Collective Monte Carlo Tree Search [Code]
45. [arxiv 2501] Virgo: A Preliminary Exploration on Reproducing o1-like MLLM [Code]
46. [arxiv 2412, CVPR'25] Perception Tokens Enhance Visual Reasoning in Multimodal Language Models [Code]
47. [arxiv 2411] Insight-V: Exploring Long-Chain Visual Reasoning with Multimodal Large Language Models [Code]
48. [arxiv 2410] Euclid: Supercharging Multimodal LLMs with Synthetic High-Fidelity Visual Descriptions [Code] [Dataset]
49. [arxiv 2410, ICLR'25 Oral] Do Vision-Language Models Represent Space and How? Evaluating Spatial Frame of Reference Under Ambiguities [Code] [Dataset]
50. [arxiv 2406, ICRA'25] SpatialBot: Precise Spatial Understanding with Vision Language Models [Code] [Dataset]
51. [arxiv 2406, NIPS'24] SpatialRGPT: Grounded Spatial Reasoning in Vision Language Models [Code] [Dataset]
52. [NIPS'24] Right this way: Can VLMs Guide Us to See More to Answer Questions? [Code] [Dataset]
53. [arxiv 2405, NAACL'25 main] VoCoT: Unleashing Visually Grounded Multi-Step Reasoning in Large Multi-Modal Models [Code] [Dataset]
54. [arxiv 2403] Chain-of-Spot: Interactive Reasoning Improves Large Vision-Language Models [Code]
55. [arxiv 2401,CVPR'24 main] SpatialVLM: Endowing Vision-Language Models with Spatial Reasoning Capabilities [Code]
56. [ACM MM'24] LLaVA-VSD: Large Language-and-Vision Assistant for Visual Spatial Description [Code] [Dataset]

Video

1. [arxiv 2505] Time-R1: Towards Comprehensive Temporal Reasoning in LLMs [Code] [Dataset]
2. [arxiv 2505] SpaceR: Reinforcing MLLMs in Video Spatial Reasoning [Code] [Dataset]
3. [arxiv 2504] TinyLLaVA-Video-R1: Towards Smaller LMMs for Video Reasoning [Code]
4. [arxiv 2504] VideoChat-R1: Enhancing Spatio-Temporal Perception via Reinforcement Fine-Tuning [Code]
5. [arxiv 2504] Spatial-R1: Enhancing MLLMs in Video Spatial Reasoning [Code]
6. [arxiv 2504] R1-Zero-VSI: Improved Visual-Spatial Reasoning via R1-Zero-Like Training [Code]
7. [arxiv 2503] SEED-Bench-R1: Exploring the Effect of Reinforcement Learning on Video Understanding: Insights from SEED-Bench-R1 [Code] [Dataset]
8. [arxiv 2503] Video-R1: Reinforcing Video Reasoning in MLLMs [Code] [Dataset]
9. [arxiv 2503] TimeZero: Temporal Video Grounding with Reasoning-Guided LVLM [Code]
10. [arxiv 2412, CVPR'25] Video-3D LLM: Learning Position-Aware Video Representation for 3D Scene Understanding [Code] [Dataset]

Embodied

1. [arxiv 2506] Fast-in-Slow: A Dual-System Foundation Model Unifying Fast Manipulation within Slow Reasoning [Code]
2. [arxiv 2506] Visual Embodied Brain: Let Multimodal Large Language Models See, Think, and Control in Spaces [Code will be released soon]
3. [arxiv 2505] Learning 3D Persistent Embodied World Models
4. [arxiv 2505] ChatVLA-2: Vision-Language-Action Model with Open-World Embodied Reasoning from Pretrained Knowledge [Code will be released soon]
5. [arxiv 2505] Agentic Robot: A Brain-Inspired Framework for Vision-Language-Action Models in Embodied Agents [Code will be released soon]
6. [arxiv 2505] InSpire: Vision-Language-Action Models with Intrinsic Spatial Reasoning [Code]
7. [arxiv 2505] From Grounding to Manipulation: Case Studies of Foundation Model Integration in Embodied Robotic Systems [Code will be released soon]
8. [arxiv 2505] OneTwoVLA: A Unified Vision-Language-Action Model with Adaptive Reasoning [Code] [Dataset]
9. [arxiv 2505] VLA-RL: Towards Masterful and General Robotic Manipulation with Scalable Reinforcement Learning [Code]
10. [arxiv 2505] AgentThink: A Unified Framework for Tool-Augmented Chain-of-Thought Reasoning in Vision-Language Models for Autonomous Driving
11. [arxiv 2505] SSR: Enhancing Depth Perception in Vision-Language Models via Rationale-Guided Spatial Reasoning [Code] [Dataset]
12. [arxiv 2504, CVPR'25] InteractVLM: 3D Interaction Reasoning from 2D Foundational Models [Code] [Dataset]
13. [arxiv 2504] Embodied-R: Collaborative Framework for Activating Embodied Spatial Reasoning in Foundation Models via Reinforcement Learning [Code]
14. [arxiv 2504, CVPR'25] RoboGround: Robotic Manipulation with Grounded Vision-Language Priors [Code] [Dataset]
15. [arxiv 2504] SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning [Code]
16. [arxiv 2504] RoboAct-CLIP: Video-Driven Pre-training of Atomic Action Understanding for Robotics
17. [arxiv 2503] PhysVLM: Enabling Visual Language Models to Understand Robotic Physical Reachability [Code] [Dataset]
18. [arxiv 2503, CVPR'25] CoT-VLA: Visual Chain-of-Thought Reasoning for Vision-Language-Action Models [Code will be releasd soon]
19. [arxiv 2503] Embodied-Reasoner: Synergizing Visual Search, Reasoning, and Action for Embodied Interactive Tasks [Code] [Dataset]
20. [arxiv 2503] LEGO-Puzzles: How Good Are MLLMs at Multi-Step Spatial Reasoning? [Code] [Dataset]
21. [arxiv 2503] MetaSpatial: Reinforcing 3D Spatial Reasoning in VLMs for the Metaverse [Code] [Dataset]
22. [arxiv 2503] Cosmos-Reason1: From Physical Common Sense To Embodied Reasoning [Code] [Dataset]
23. [arxiv 2502, CVPR'25] RoboBrain: A Unified Brain Model for Robotic Manipulation from Abstract to Concrete [Code]
24. [arxiv 2502, ICRA'25] A Real-to-Sim-to-Real Approach to Robotic Manipulation with VLM-Generated Iterative Keypoint Rewards [Code]
25. [ICRA'25] UAD: Unsupervised Affordance Distillation for Generalization in Robotic Manipulation [Code]
26. [arxiv 2502] ConRFT: A Reinforced Fine-tuning Method for VLA Models via Consistency Policy [Code]
27. [arxiv 2502] SoFar: Language-Grounded Orientation Bridges Spatial Reasoning and Object Manipulation [Code]
28. [arxiv 2502] Video2Policy: Scaling up Manipulation Tasks in Simulation through Internet Videos
29. [arxiv 2502, ICLR'25] Predicate Hierarchies Improve Few-Shot State Classification [Code]
30. [arxiv 2501, RSS'25] SpatialVLA: Exploring Spatial Representations for Visual-Language-Action Models [Code]
31. [ICLR'25] Vision Language Models are In-Context Value Learners [Code will be released soon]
32. [CVPR'24] ManipLLM: Embodied Multimodal Large Language Model for Object-Centric Robotic Manipulation [Code]
33. [arxiv 2412] RoboMatrix: A Skill-centric Hierarchical Framework for Scalable Robot Task Planning and Execution in Open-World [Code] [Dataset]
34. [arxiv 2412, RSS'25] NaVILA: Legged Robot Vision-Language-Action Model for Navigation [Code]
35. [arxiv 2412] Enhancing Multi-Robot Semantic Navigation Through Multimodal Chain-of-Thought Score Collaboration [Code]
36. [arxiv 2412] Code-as-Monitor: Constraint-aware Visual Programming for Reactive and Proactive Robotic Failure Detection [Code will be released soon]
37. [arxiv 2412, CVPR'25] 3D-Mem: 3D Scene Memory for Embodied Exploration and Reasoning [Code]
38. [CoRL'24 LEAP workshop] SkillWrapper: Autonomously Learning Interpretable Skill Abstractions with Foundation Models [Code]
39. [arxiv 2411, CVPR'25 Oral] RoboSpatial: Teaching Spatial Understanding to 2D and 3D Vision-Language Models for Robotics [Code] [Dataset]
40. [arxiv 2410,CoRL'24] Scaling Robot Policy Learning via Zero-Shot Labeling with Foundation Models [Code] [Dataset]
41. [arxiv 2410, ICLR'25] AHA: A Vision-Language-Model for Detecting and Reasoning over Failures in Robotic Manipulation [Code]
42. [arxiv 2409, RAL'25] MotIF: Motion Instruction Fine-tuning [Code]
43. [CVPR'25 Highlight] OmniManip: Towards General Robotic Manipulation via Object-Centric Interaction Primitives as Spatial Constraints [Code will be released soon]
44. [arxiv 2407, CoRL'24] Robotic Control via Embodied Chain-of-Thought Reasoning [Code]
45. [arxiv 2406, CoRL'24] RoboPoint: A Vision-Language Model for Spatial Affordance Prediction for Robotics [Code] [Dataset]
46. [arxiv 2406, EMNLP'24 main] TopViewRS: Vision-Language Models as Top-View Spatial Reasoners [Code] [Dataset]
47. [arxiv 2406, ICLR'25] LLaRA: Supercharging Robot Learning Data for Vision-Language Policy [Code]
48. [arxiv 2405] ReKep: Spatio-Temporal Reasoning of Relational Keypoint Constraints for Robotic Manipulation [Code]
49. [arxiv 2403, NIPS'24] SpatialPIN: Enhancing Spatial Reasoning Capabilities of Vision-Language Models through Prompting and Interacting 3D Priors [Code]
50. [arxiv 2402, COLING'24 main] Scaffolding Coordinates to Promote Vision-Language Coordination in Large Multi-Modal Models [Code]
51. [arxiv 2305, NIPS'23 Spotlight] EmbodiedGPT: Vision-Language Pre-Training via Embodied Chain of Thought [Code]
52. [CoRL'24] ThinkGrasp: A Vision-Language System for Strategic Part Grasping in Clutter [Code]
53. [RAL'24] GPT-4V(ision) for Robotics: Multimodal Task Planning From Human Demonstration
54. [NIPS'24 Spotlight] ICAL: VLM Agents Generate Their Own Memories: Distilling Experience into Embodied Programs of Thought [Code]
55. SpatialCoT: Advancing Spatial Reasoning through Coordinate Alignment and Chain-of-Thought for Embodied Task Planning
56. Memory-Driven Multimodal Chain of Thought for Embodied Long-Horizon Task Planning
57. [arxiv 2306,CoRL'23] REFLECT: Summarizing Robot Experiences for Failure Explanation and Correction [Code] [Dataset]

Audio/Omini

1. [arxiv 2505] Omni-R1 (ZJU): Reinforcement Learning for Omnimodal Reasoning via Two-System Collaboration [Code][]
2. [arxiv 2505] Omni-R1 (MIT): Do You Really Need Audio to Fine-Tune Your Audio LLM?
3. [arxiv 2505] EchoInk-R1: Exploring Audio-Visual Reasoning in Multimodal LLMs via Reinforcement Learning [Dataset] [Code]
4. [arxiv 2504] SARI: Structured Audio Reasoning via Curriculum-Guided Reinforcement Learning
5. [arxiv 2503] R1-AQA: Reinforcement Learning Outperforms Supervised Fine-Tuning: A Case Study on Audio Question Answering [Code]
6. [arxiv 2503] Audio-Reasoner: Improving Reasoning Capability in Large Audio Language Models [Code]
7. [arxiv 2503] R1-Omni: Explainable Omni-Multimodal Emotion Recognition with Reinforcement Learning [Code]
8. [arxiv 2402, ICML'24] BAT: Learning to Reason about Spatial Sounds with Large Language Models [Code] [Datasets]

Math

1. [arxiv 2505] URSA: Understanding and Verifying Chain-of-thought Reasoning in Multimodal Mathematics [Code] [Datasets]
2. [arxiv 2403, COLM'24] How Far Are We from Intelligent Visual Deductive Reasoning? [Code]

Web/GUI

1. [arxiv 2505] Web-Shepherd: Advancing PRMs for Reinforcing Web Agents [Code] [Datasets]
2. [arxiv 2505] ARPO: End-to-End Policy Optimization for GUI Agents with Experience Replay [Code]
3. [arxiv 2505] GUI-G1: Understanding R1-Zero-Like Training for Visual Grounding in GUI Agents [Code]
4. [arxiv 2504] WebThinker: Empowering Large Reasoning Models with Deep Research Capability [Code] [Dataset]
5. [arxiv 2504] InfiGUI-R1: Advancing Multimodal GUI Agents from Reactive Actors to Deliberative Reasoners [Code]
6. [arxiv 2504] GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents [Dataset] [Code]

Medical

1. [arxiv 2503] Med-R1: Reinforcement Learning for Generalizable Medical Reasoning in Vision-Language Models [Code]
2. [arxiv 2502] MedVLM-R1: Incentivizing Medical Reasoning Capability of Vision-Language Models (VLMs) via Reinforcement Learning [Code]
3. [EMNLP'24 main] MedCoT: Medical Chain of Thought via Hierarchical Expert [Code]

Grounding

1. [arxiv 2505] UniVG-R1: Reasoning Guided Universal Visual Grounding with Reinforcement Learning [Code] [Dataset]
2. [arxiv 2504] CrowdVLM-R1: Expanding R1 Ability to Vision Language Model for Crowd Counting using Fuzzy Group Relative Policy Reward [Code] [Dataset]
3. [arxiv 2503] Integrating Chain-of-Thought for Multimodal Alignment: A Study on 3D Vision-Language Learning [Dataset]
4. [arxiv 2502, CVPR'25 Highlight] Towards Zero-Shot Anomaly Detection and Reasoning with Multimodal Large Language Models [Code]
5. [arxiv 2503] Seg-Zero: Reasoning-Chain Guided Segmentation via Cognitive Reinforcement [Code]
6. [arxiv 2403, ECCV'24] DetToolChain: A New Prompting Paradigm to Unleash Detection Ability of MLLM [Code]
7. [arxiv 2403, ECCV'24] Agent3D-Zero: An Agent for Zero-shot 3D Understanding

Multimodal Reward Model

1. [arxiv 2505] Skywork-VL Reward: An Effective Reward Model for Multimodal Understanding and Reasoning [Code]
2. [arxiv 2505] UnifiedReward-Think: Unified Multimodal Chain-of-Thought Reward Model through Reinforcement Fine-Tuning [Code] [Datasets]
3. [arxiv 2505] R1-Reward: Training Multimodal Reward Model Through Stable Reinforcement Learning [Code] [Dataset]

Evaluation for reasoning ability

1. [arxiv 2505] ManiTaskGen: A Comprehensive Task Generator for Benchmarking and Improving Vision-Language Agents on Embodied Decision-Making
2. [arxiv 2505] Robo2VLM: Visual Question Answering from Large-Scale In-the-Wild Robot Manipulation Datasets [Dataset]
3. [arxiv 2505] RoboFAC: A Comprehensive Framework for Robotic Failure Analysis and Correction [Code]
4. [arxiv 2505] SpatialScore: Towards Unified Evaluation for Multimodal Spatial Understanding [Code]
5. [arxiv 2505] EgoDex: Learning Dexterous Manipulation from Large-Scale Egocentric Video [Dataset]
6. [arxiv 2505] PhyX: Does Your Model Have the "Wits" for Physical Reasoning? [Code] [Dataset]
7. [arxiv 2505] ReasonMap: Can MLLMs Guide Me Home? A Benchmark Study on Fine-Grained Visual Reasoning from Transit Maps [Code] [Dataset]
8. [arxiv 2504] VisuLogic: A Benchmark for Evaluating Visual Reasoning in Multi-modal Large Language Models [Dataset] [Code]
9. [arxiv 2504] Video-MMLU: A Massive Multi-Discipline Lecture Understanding Benchmark [Dataset] [Code]
10. [arxiv 2504] VCR-Bench: A Comprehensive Evaluation Framework for Video Chain-of-Thought Reasoning [Dataset] [Code]
11. [arxiv 2504] MDK12-Bench: A Multi-Discipline Benchmark for Evaluating Reasoning in Multimodal Large Language Models [Code]
12. [arxiv 2504] Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs [Dataset] [Code]
13. [arxiv 2503] Mind the Gap: Benchmarking Spatial Reasoning in Vision-Language Models [Code]
14. [arxiv 2503] STI-Bench: Are MLLMs Ready for Precise Spatial-Temporal World Understanding? [Dataset] [Code]
15. [arxiv 2503] Open3DVQA: A Benchmark for Comprehensive Spatial Reasoning with Multimodal Large Language Model in Open Space [Code]
16. [arxiv 2503] Gemini Robotics: Bringing AI into the Physical World (Embodied Reasoning QA Evaluation Dataset) [Code]
17. [arxiv 2503] V1-33K: Toward Multimodal Reasoning by Designing Auxiliary Tasks [Dataset] [Code]
18. [arxiv 2502] Spatial457: A Diagnostic Benchmark for 6D Spatial Reasoning of Large Multimodal Models [Dataset] [Code]
19. [arxiv 2502] MM-IQ: Benchmarking Human-Like Abstraction and Reasoning in Multimodal Models [Dataset] [Code]
20. [arxiv 2502] MME-CoT: Benchmarking Chain-of-Thought in Large Multimodal Models for Reasoning Quality, Robustness, and Efficiency [Dataset] [Code]
21. [arxiv 2502] ZeroBench: An Impossible* Visual Benchmark for Contemporary Large Multimodal Models [Dataset] [Code]
22. [arxiv 2502] HumanEval-V: Benchmarking High-Level Visual Reasoning with Complex Diagrams in Coding Tasks [Dataset] [Code]
23. [arxiv 2501] EmbodiedEval: Evaluate Multimodal LLMs as Embodied Agents [Dataset] [Code]
24. [ICLR 2024 & ECCV 2024] The All-Seeing Projects: Towards Panoptic Visual Recognition&Understanding and General Relation Comprehension of the Open World [Code]
25. [arxiv 2411, EMNLP'24 main] Spatial-MM: An Empirical Analysis on Spatial Reasoning Capabilities of Large Multimodal Models [Code]
26. [arxiv 2411] GEOBench-VLM: Benchmarking Vision-Language Models for Geospatial Tasks [Code]
27. [arxiv 2412] Thinking in Space: How Multimodal Large Language Models See, Remember, and Recall Spaces [Dataset]
28. [arxiv 2409] FineCops-Ref: A new Dataset and Task for Fine-Grained Compositional Referring Expression Comprehension [Code]
29. [arxiv 2306] LIBERO:Benchmarking Knowledge Transfer for Lifelong Robot Learning [Code]
30. [RAL'22 ] CALVIN: A benchmark for Language-Conditioned Policy Learning for Long-Horizon Robot Manipulation Tasks [Code]
31. [arxiv 2210, ICML'23] VIMA: General Robot Manipulation with Multimodal Prompts [Dataset] [Code]

Embodied Environment

1. Maniskill [Code]
2. Issac Lab/Gym/Sim [Code]
3. SimplerEnv [Code]
4. Roboverse [code]
5. Bullet Physics SDK [code]
6. Genesis [Code]
7. RoboTwin [Code]
8. RLBench [Code]
9. OmniGibson [Document]
10. AI2-THOR [Link]
11. Legent [code]
12. Gazebo [Link]
13. CoppeliaSim (formerly known as V-REP) [Link]

Foundation model for reasoning

1. Qwen2.5-VL [Code]
2. Bagel [Code]
3. BLIP3-o [Code]
4. Janus-Series: Unified Multimodal Understanding and Generation Models [Code]
5. Kimi-VL [Code]
6. InternVL3 [code]
7. Bytedance ReFT
8. Kimi-K1.5
9. DeepSeek-R1

Code will be released soon

1. [arxiv 2505, CVPR'25 highlight] SpatialLLM: A Compound 3D-Informed Design towards Spatially-Intelligent Large Multimodal Models
2. [ICLR'25] MLLM as Retriever: Interactively Learning Multimodal Retrieval for Embodied Agents [Code will be released soon]
3. [ICLR'25] Chain-of-region: Visual Language Models Need Details for Diagram Analysis
4. [arxiv 2505, ICLR'25] Chain-of-Focus Prompting: Leveraging Sequential Visual Cues to Prompt Large Autoregressive Vision Models [Code will be released soon]
5. [arxiv 2505] 3DLLM-Mem: Long-Term Spatial-Temporal Memory for Embodied 3D Large Language Model [Code will be released soon]
6. [arxiv 2505] PlaceIt3D: Language-Guided Object Placement in Real 3D Scenes [Code will be released soon]
7. [arxiv 2505] WebAgent-R1: Training Web Agents via End-to-End Multi-Turn Reinforcement Learning [Code will be released soon]
8. [arxiv 2505] VisualQuality-R1: Reasoning-Induced Image Quality Assessment via Reinforcement Learning to Rank [Code will be released soon]
9. [arxiv 2505] Patho-R1: A Multimodal Reinforcement Learning-Based Pathology Expert Reasoner [Code will be released soon]
10. [arxiv 2505] STAR-R1: Spacial TrAnsformation Reasoning by Reinforcing Multimodal LLMs [Code will be released soon]
11. [arxiv 2505] Visionary-R1: Mitigating Shortcuts in Visual Reasoning with Reinforcement Learning [Code will be released soon]
12. [arxiv 2505] R1-ShareVL: Incentivizing Reasoning Capability of Multimodal Large Language Models via Share-GRPO [Code will be released soon]
13. [arxiv 2505] Chain-of-Focus: Adaptive Visual Search and Zooming for Multimodal Reasoning via RL [Code will be released soon]
14. [arxiv 2505] Visual Planning: Let’s Think Only with Images [Code will be released soon]
15. [arxiv 2505] X-Reasoner: Towards Generalizable Reasoning Across Modalities and Domains [Code will be released soon]
16. [arxiv 2505] ManipBench: Benchmarking Vision-Language Models for Low-Level Robot Manipulation [Code will be released soon]
17. [arxiv 2504, ICRA'25] Chain-of-Modality: Learning Manipulation Programs from Multimodal Human Videos with Vision-Language-Models [Code will be released soon]
18. [arxiv 2504] InteractVLM: 3D Interaction Reasoning from 2D Foundational Models [Code will be released soon]
19. [arxiv 2504] Fast-Slow Thinking for Large Vision-Language Model Reasoning [Code will be released soon]
20. [arxiv 2504] Relation-R1: Cognitive Chain-of-Thought Guided Reinforcement Learning for Unified Relational Comprehension [Code will be released soon]
21. [arxiv 2504] Perspective-Aware Reasoning in Vision-Language Models via Mental Imagery Simulation [Code will be released soon]
22. [arxiv 2504] Mixed Signals: Decoding VLMs' Reasoning and Underlying Bias in Vision-Language Conflict [Code will be released soon]
23. [arxiv 2504] KeyMPs: One-Shot Vision-Language Guided Motion Generation by Sequencing DMPs for Occlusion-Rich Tasks
24. [arxiv 2503] Can Large Vision Language Models Read Maps Like a Human? [Code will be released soon]
25. [arxiv 2503] ST-VLM: Kinematic Instruction Tuning for Spatio-Temporal Reasoning in Vision-Language Models [Code will be released soon]
26. [arxiv 2503] Q-Insight: Understanding Image Quality via Visual Reinforcement Learning [Code will be released soon]
27. [arxiv 2503] VisualThinker-R1-Zero: R1-Zero's "Aha Moment" in Visual Reasoning on a 2B Non-SFT Model [Code will be released soon]
28. [arxiv 2503] Free-form language-based robotic reasoning and grasping [Code will be released soon]
29. [arxiv 2502] NAVER: A Neuro-Symbolic Compositional Automaton for Visual Grounding with Explicit Logic Reasoning [Code will be released soon]
30. [arxiv 2412] SAT: Dynamic Spatial Aptitude Training for Multimodal Language Models [Code will be released soon]
31. [arxiv 2410] Sparkle: Mastering Basic Spatial Capabilities in Vision Language Models Elicits Generalization to Spatial Reasoning
32. [arxiv 2406] RoboGolf: Mastering Real-World Minigolf with a Reflective Multi-Modality Vision-Language Model [Code will be released soon]
33. [arxiv 2402] PIVOT: Iterative Visual Prompting Elicits Actionable Knowledge for VLMs [Code will be released soon]
34. [arxiv 2412] Video LLMs for Temporal Reasoning in Long Videos
35. [arxiv 2412] Improving Vision-Language-Action Models via Chain-of-Affordance [Code will be released soon]
36. [arxiv 2411] Learning from Feedback: Semantic Enhancement for Object SLAM Using Foundation Models [Code will be released soon]
37. [arxiv 2312] Visual Program Distillation: Distilling Tools and Programmatic Reasoning into Vision-Language Models [Code will be released soon]
38. [ACM MM‘24] RAG-Guided Large Language Models for Visual Spatial Description with Adaptive Hallucination Corrector
39. [ICLR'25 workshop] Evaluating Multi-modal Language Models Through Concept Hacking

Open-Source Projects (Repos without Paper)

Training Framework

• EasyR1 💻

Embodied

• OpenVLA 💻
• PI Series 💻
• MiniVLA 💻

Image

• R1-V 💻
• Multimodal Open R1 💻
• MMR1 💻
• R1-Multimodal-Journey 💻
• R1-Vision 💻
• Ocean-R1 💻
• R1V-Free 💻
• SeekWorld 💻
• R1-Track 💻

Video

• Open R1 Video 💻
• Temporal-R1 💻
• Open-LLaVA-Video-R1 💻

Agent

• VAGEN 💻

Acknowledgement

1. https://github.com/Sun-Haoyuan23/Awesome-RL-based-Reasoning-MLLMs
2. https://github.com/yaotingwangofficial/Awesome-MCoT
3. https://github.com/jonyzhang2023/awesome-embodied-vla-va-vln
4. https://modelscope.cn/studios/AI-ModelScope/awesome-reasoning
5. https://github.com/Songwxuan/Embodied-AI-Paper-TopConf
6. https://github.com/HCPLab-SYSU/Embodied_AI_Paper_List
7. https://github.com/zubair-irshad/Awesome-Robotics-3D
8. https://github.com/LightChen233/Awesome-Long-Chain-of-Thought-Reasoning
9. https://github.com/NishilBalar/Awesome-LVLM-Hallucination
10. https://github.com/zhengxuJosh/Awesome-RAG-Vision
11. https://github.com/BioRAILab/Awesome-Neuroscience-Agent-Reasoning
12. https://github.com/BioRAILab/Neural-Brain-for-Embodied-Agents

Citation

If you find this repository useful for your research and applications, please star us and consider citing:

@misc{Tang2025Awesome-spatial-visual-reasoning-MLLMs,
  title={Awesome-spatial-visual-reasoning-MLLMs},
  author={Jing Tang},
  year={2025},
  howpublished={\url{https://github.com/vaew/Awesome-spatial-visual-reasoning-MLLMs}},
  note={Github Repository},
}