A Multi-Agent Framework For Teleoperation and Autonomous Needle Manipulation on the dVRK

Luigi Muratore^1,2, Federica Barontini², Francesco Marzola², Matteo Pescio^2,3, Alberto Arezzo², Giulio Dagnino^2,5, Giuseppe Averta⁴

¹DET, Politecnico di Torino, Corso Duca degli Abruzzi 24, Turin, 10129, TO, Italy.
²Department of Surgical Sciences, Università degli Studi di Torino, Corso Dogliotti 14, Turin, 10126, TO, Italy.
³DIMEAS, Politecnico di Torino, Corso Duca degli Abruzzi 24, Turin, 10129, TO, Italy.
⁴DAUIN, Politecnico di Torino, Corso Duca degli Abruzzi 24, Turin, 10129, TO, Italy.
⁵Robotics and Mechatronics, University of Twente, Drienerlolaan 5, Enschede, 7522, NB, Netherlands.

---

📋 Abstract

Purpose: Reliable automation of surgical suturing requires accurate and flexible simulation tools that bridge learning and clinical practice. This work presents a unified Unity–ROS simulation framework for autonomous surgical robotics with the da Vinci Research Kit (dVRK). The framework supports reinforcement-learning-based training of surgical tasks such as needle grasping and placement, while also enabling interactive teleoperation and surgical training. Real-time communication with the dVRK master console allows surgeon control of simulated patient-side manipulators (PSMs), and an integrated graphical user interface (GUI) supports environment tuning for reward design, algorithm development, and sim-to-real validation.

Methods: A physics-based surgical environment was developed in Unity, incorporating a full kinematic reconstruction of the dVRK to ensure accurate joint-level and end-effector behavior. Bidirectional ROS communication enables real-time control, state streaming, and visualization between the simulation and the physical master console. Proximal Policy Optimization (PPO) was used for policy training, combined with curriculum learning to progressively increase task difficulty. Domain randomization was applied to improve robustness and generalization. In addition, a multi-agent reinforcement learning strategy is under investigation to enhance cooperative performance in complex tasks such as suturing.

Results: Preliminary simulations demonstrated stable policy convergence and precise execution of autonomous needle manipulation tasks. Policies trained with domain randomization showed increased robustness to visual and positional variations, achieving consistent performance across randomized conditions and indicating improved generalization.

Conclusions: The proposed Unity–ROS–dVRK framework provides a modular, high-fidelity platform for developing and benchmarking autonomous surgical skills. By integrating accurate kinematics, reinforcement learning, and teleoperation, it supports both autonomous training and human-in-the-loop control in realistic surgical scenarios. Direct ROS connectivity and an integrated GUI further facilitate rapid development and sim-to-real experimentation.

Keywords: da Vinci Research Kit (dVRK); surgical robotics; reinforcement learning; Unity; ROS; multi-agent systems; autonomous surgery; simulation.

---

Key Features

• 🤖 Multi-agent coordination and collaboration
• 🎯 Unified reinforcement learning framework
• 🖥️ Graphics User Interface (GUI)
• 🔗 ROS integration
• 🏥 Integration with da Vinci Research Kit

---

🎬 Demo Videos

For interactive demos, visit our project website.

Single Agent Tasks

Reaching Task	Placement Task	Complete Task
Seed 42, 11, 2026	Seed 42, 11, 2026	Seed 42, 11, 2026

Multi-Agent Coordination

Collaborative multi-agent demonstrations showing coordinated surgical task execution.

---

🏗️ System Architecture

System Architecture Overview

Multi-Agent Coordination Framework

---

🤖 Kinematic Model of PSM

Development progression of the Patient Side Manipulator (PSM) kinematic model with constraint implementation.

Kinematic model of the PSM

---

📊 Training Results

Training convergence for different tasks

---

🚀 Getting Started

Prerequisites

• Unity 2021.3+
• ROS Noetic
• Python 3.8+
• dVRK hardware (optional for sim-to-real)

Installation

# Clone the repository
git clone https://github.com/luigimuratore/MAURF-dVRK.git
cd MAURF-dVRK

# Install Python dependencies
pip install -r requirements.txt

# Setup ROS workspace
# Follow instructions in SETUP.md

Running the Simulation

# Launch the Unity simulation
# (Instructions in documentation)

# Start ROS nodes
...

# Train a policy
mlagent ...

For detailed setup instructions, see SETUP.md.

---

---

📚 Citation

If you use this work in your research, please cite:

@article{muratore2025maurf,
    title={MAURF-dVRK: Multi-Agent Unified Reinforcement Framework for da Vinci Research Kit},
    author={Luigi Muratore and Federica Barontini and Giuseppe Averta},
    journal={Conference/Journal Name},
    year={2026}
}

---

📧 Contact

For questions or collaboration opportunities, please reach out:

• Email: gigiomuratore@gmail.com
• GitHub: @luigimuratore

---

© 2025 MAURF-dVRK Project