Add support of Franka Panda robot and Zed camera (not fully tested)

.gitignore

@@ -93,3 +93,4 @@ references/
 **/pretrained_weights
 **/outputs
 reference_repos/
+tests/

README.md

@@ -5,7 +5,7 @@ ManiUniCon is a comprehensive, multi-process robotics control framework designed
 ## 🚀 Features
 
 - **Modular Design**: Plug-and-play architecture for seamless integration of new robot arms and algorithms
-- **Multi-Robot Support**: Compatible with UR5, XArm6, and easily extensible to other robotic arms
+- **Multi-Robot Support**: Compatible with UR5, XArm6, Franka FR3/Panda, and easily extensible to other robotic arms
 - **One-Click Switching**: Effortlessly transition between data collection and policy deployment modes
 - **Real-time Control**: High-frequency control loops with shared memory architecture
 - **Algorithm Agnostic**: Easy integration of new learning algorithms through standardized interfaces
@@ -14,12 +14,6 @@ ManiUniCon is a comprehensive, multi-process robotics control framework designed
 - **Flexible Configuration**: Hydra-based configuration management for quick experiment setup
 - **Safety Features**: Emergency stop, error handling, and reset functionality
 
-## 🏗️ Architecture
-
-ManiUniCon uses a multi-process architecture with shared memory for efficient real-time communication:
-
-![Framework Architecture](medias/framework.png)
-
 ## 🔧 Supported Hardware
 
 ### Robot Arms
@@ -39,14 +33,24 @@ ManiUniCon uses a multi-process architecture with shared memory for efficient re
     <img src="medias/xarm_keyboard_clip.gif" width="20%"/>
   </p>
 
+- **FR3**: FRANKA RESEARCH 3 robot with [franky-control](https://github.com/TimSchneider42/franky)
+- **FRPanda**: Franka Emika Panda robot with [deoxys_control](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys/franka-interface) (Note: Panda is no longer supported and does not work with latest libfranka)
+
 ### Sensors
 - **RealSense Cameras**: Intel RealSense depth cameras
+- **Zed Cameras** [WIP, not tested]: Zed series depth cameras
 
 ### Teleoperation Devices
 - **Meta Quest Controllers**: Meta Quest VR controllers for intuitive 6-DOF manipulation
 - **SpaceMouse**: 3Dconnexion SpaceMouse for precise position and orientation control
 - **Keyboard**: Basic keyboard control for simple teleoperation tasks
 
+## 🏗️ Architecture
+
+ManiUniCon uses a multi-process architecture with shared memory for efficient real-time communication:
+
+![Framework Architecture](medias/framework.png)
+
 ## 📦 Installation
 
 ### Prerequisites
@@ -87,11 +91,14 @@ For **XArm6** robot:
 pip install -e '.[xarm]'
 ```
 
-For **FRANKA** robot with [franky-control](https://github.com/TimSchneider42/franky) library:
+For **FR3** robot with [franky-control](https://github.com/TimSchneider42/franky):
 ```bash
 pip install -e '.[franky_fr3_franky]'
 ```
 
+For **FRPanda** robot with [deoxys_control](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys/franka-interface):
+- Follow the installation guidance of [deoxys_control](https://github.com/UT-Austin-RPL/deoxys_control/tree/main/deoxys/franka-interface) on the server and the client side.
+
 For **RealSense** cameras:
 ```bash
 pip install -e '.[realsense]'
@@ -332,7 +339,7 @@ If you use ManiUniCon in your research, please cite:
 ```bibtex
 @software{maniunicon2025,
   title={ManiUniCon: A Unified Control Interface for Robotic Manipulation},
-  author={Zhengbang Zhu and Minghuan Liu},
+  author={Zhu, Zhengbang and Liu, Minghuan and Han, Xiaoshen},
   year={2025},
   url={https://github.com/Universal-Control/ManiUniCon}
 }

assets/franka_panda_urdf/meshes/collision/finger.obj

@@ -0,0 +1,239 @@
+# File produced by Open Asset Import Library (http://www.assimp.sf.net)
+# (assimp v3.0.1262)
+
+mtllib finger.stl.obj.mtl
+
+# 96 vertex positions
+v  0.01036 0.0264034 0.000154629
+v  0.0104486 0.0025833 0.000146801
+v  -0.0103872 0.00253418 0.000131696
+v  -0.0103872 0.00253418 0.000131696
+v  -0.0104795 0.0161016 0.0189882
+v  -0.0104013 0.0263094 0.00016651
+v  -0.0104013 0.0263094 0.00016651
+v  -0.0104795 0.0161016 0.0189882
+v  -0.0103889 0.0252203 0.0191876
+v  0.0104486 0.0025833 0.000146801
+v  -0.0087304 -2.35252e-05 0.0361648
+v  -0.0103872 0.00253418 0.000131696
+v  0.0104005 0.0252534 0.0190366
+v  -0.0103889 0.0252203 0.0191876
+v  0.00583983 0.0142743 0.0538035
+v  -0.0103872 0.00253418 0.000131696
+v  -0.0104013 0.0263094 0.00016651
+v  0.01036 0.0264034 0.000154629
+v  0.00861608 0.0139887 0.0513279
+v  0.0104948 0.0151026 0.0184356
+v  0.0104005 0.0252534 0.0190366
+v  0.0104948 0.0151026 0.0184356
+v  0.00869277 -0.000132643 0.0501662
+v  0.0104486 0.0025833 0.000146801
+v  0.00861608 0.0139887 0.0513279
+v  0.00869277 -0.000132643 0.0501662
+v  0.0104948 0.0151026 0.0184356
+v  -0.00862294 -5.68019e-05 0.0509528
+v  -0.0103872 0.00253418 0.000131696
+v  -0.0087304 -2.35252e-05 0.0361648
+v  0.0104486 0.0025833 0.000146801
+v  0.00869277 -0.000132643 0.0501662
+v  -0.0087304 -2.35252e-05 0.0361648
+v  0.00869277 -0.000132643 0.0501662
+v  -0.00548142 -9.11208e-05 0.0537247
+v  -0.0087304 -2.35252e-05 0.0361648
+v  0.0104005 0.0252534 0.0190366
+v  0.0104948 0.0151026 0.0184356
+v  0.0104486 0.0025833 0.000146801
+v  0.0104005 0.0252534 0.0190366
+v  0.0104486 0.0025833 0.000146801
+v  0.01036 0.0264034 0.000154629
+v  0.0104005 0.0252534 0.0190366
+v  0.01036 0.0264034 0.000154629
+v  -0.0103889 0.0252203 0.0191876
+v  -0.0103889 0.0252203 0.0191876
+v  -0.00527792 0.0142931 0.053849
+v  0.00583983 0.0142743 0.0538035
+v  -0.00777838 0.0142182 0.0523659
+v  -0.00527792 0.0142931 0.053849
+v  -0.0103889 0.0252203 0.0191876
+v  -0.00862294 -5.68019e-05 0.0509528
+v  -0.00548142 -9.11208e-05 0.0537247
+v  -0.00777838 0.0142182 0.0523659
+v  -0.0103872 0.00253418 0.000131696
+v  -0.00862294 -5.68019e-05 0.0509528
+v  -0.0104795 0.0161016 0.0189882
+v  0.0104005 0.0252534 0.0190366
+v  0.00583983 0.0142743 0.0538035
+v  0.00861608 0.0139887 0.0513279
+v  0.01036 0.0264034 0.000154629
+v  -0.0104013 0.0263094 0.00016651
+v  -0.0103889 0.0252203 0.0191876
+v  -0.0104795 0.0161016 0.0189882
+v  -0.00884117 0.0139176 0.0505894
+v  -0.0103889 0.0252203 0.0191876
+v  -0.00884117 0.0139176 0.0505894
+v  -0.00777838 0.0142182 0.0523659
+v  -0.0103889 0.0252203 0.0191876
+v  0.00583983 0.0142743 0.0538035
+v  -0.00548142 -9.11208e-05 0.0537247
+v  0.00613802 -2.06026e-05 0.0535776
+v  -0.00527792 0.0142931 0.053849
+v  -0.00548142 -9.11208e-05 0.0537247
+v  0.00583983 0.0142743 0.0538035
+v  0.00869277 -0.000132643 0.0501662
+v  0.00613802 -2.06026e-05 0.0535776
+v  -0.00548142 -9.11208e-05 0.0537247
+v  0.00613802 -2.06026e-05 0.0535776
+v  0.00869277 -0.000132643 0.0501662
+v  0.00861608 0.0139887 0.0513279
+v  -0.00548142 -9.11208e-05 0.0537247
+v  -0.00862294 -5.68019e-05 0.0509528
+v  -0.0087304 -2.35252e-05 0.0361648
+v  -0.00777838 0.0142182 0.0523659
+v  -0.00548142 -9.11208e-05 0.0537247
+v  -0.00527792 0.0142931 0.053849
+v  -0.00862294 -5.68019e-05 0.0509528
+v  -0.00884117 0.0139176 0.0505894
+v  -0.0104795 0.0161016 0.0189882
+v  0.00613802 -2.06026e-05 0.0535776
+v  0.00861608 0.0139887 0.0513279
+v  0.00583983 0.0142743 0.0538035
+v  -0.00777838 0.0142182 0.0523659
+v  -0.00884117 0.0139176 0.0505894
+v  -0.00862294 -5.68019e-05 0.0509528
+
+# 0 UV coordinates
+
+# 96 vertex normals
+vn 0.000724164 0.000331308 -1
+vn 0.000724164 0.000331308 -1
+vn 0.000724164 0.000331308 -1
+vn -0.99999 -0.000585782 -0.00447174
+vn -0.99999 -0.000585782 -0.00447174
+vn -0.99999 -0.000585782 -0.00447174
+vn -0.99995 0.00990875 0.00122006
+vn -0.99995 0.00990875 0.00122006
+vn -0.99995 0.00990875 0.00122006
+vn 0.00240304 -0.997479 -0.0709137
+vn 0.00240304 -0.997479 -0.0709137
+vn 0.00240304 -0.997479 -0.0709137
+vn 0.000668274 0.953556 0.301214
+vn 0.000668274 0.953556 0.301214
+vn 0.000668274 0.953556 0.301214
+vn -0.0005789 0.00146393 -0.999999
+vn -0.0005789 0.00146393 -0.999999
+vn -0.0005789 0.00146393 -0.999999
+vn 0.998344 0.00589157 0.0572229
+vn 0.998344 0.00589157 0.0572229
+vn 0.998344 0.00589157 0.0572229
+vn 0.998185 -0.050838 0.0322792
+vn 0.998185 -0.050838 0.0322792
+vn 0.998185 -0.050838 0.0322792
+vn 0.998371 0.000729252 0.0570496
+vn 0.998371 0.000729252 0.0570496
+vn 0.998371 0.000729252 0.0570496
+vn -0.871287 -0.490747 0.00522707
+vn -0.871287 -0.490747 0.00522707
+vn -0.871287 -0.490747 0.00522707
+vn 0.0362712 -0.99794 -0.0529128
+vn 0.0362712 -0.99794 -0.0529128
+vn 0.0362712 -0.99794 -0.0529128
+vn -0.00372285 -0.999988 -0.00316058
+vn -0.00372285 -0.999988 -0.00316058
+vn -0.00372285 -0.999988 -0.00316058
+vn 0.999909 0.00984192 -0.00926313
+vn 0.999909 0.00984192 -0.00926313
+vn 0.999909 0.00984192 -0.00926313
+vn 0.999991 0.00372285 -0.00191858
+vn 0.999991 0.00372285 -0.00191858
+vn 0.999991 0.00372285 -0.00191858
+vn -0.0011495 0.99815 0.0607926
+vn -0.0011495 0.99815 0.0607926
+vn -0.0011495 0.99815 0.0607926
+vn 0.00284562 0.953846 0.300284
+vn 0.00284562 0.953846 0.300284
+vn 0.00284562 0.953846 0.300284
+vn -0.218924 0.920873 0.32259
+vn -0.218924 0.920873 0.32259
+vn -0.218924 0.920873 0.32259
+vn -0.661425 -0.0350314 0.749193
+vn -0.661425 -0.0350314 0.749193
+vn -0.661425 -0.0350314 0.749193
+vn -0.998169 -0.0513123 0.0320353
+vn -0.998169 -0.0513123 0.0320353
+vn -0.998169 -0.0513123 0.0320353
+vn 0.377717 0.867563 0.323518
+vn 0.377717 0.867563 0.323518
+vn 0.377717 0.867563 0.323518
+vn -0.00448618 0.998355 0.0571685
+vn -0.00448618 0.998355 0.0571685
+vn -0.00448618 0.998355 0.0571685
+vn -0.998589 0.0087761 0.0523757
+vn -0.998589 0.0087761 0.0523757
+vn -0.998589 0.0087761 0.0523757
+vn -0.665951 0.690851 0.281485
+vn -0.665951 0.690851 0.281485
+vn -0.665951 0.690851 0.281485
+vn 0.0127505 -0.0155288 0.999798
+vn 0.0127505 -0.0155288 0.999798
+vn 0.0127505 -0.0155288 0.999798
+vn 0.00408502 -0.00870048 0.999954
+vn 0.00408502 -0.00870048 0.999954
+vn 0.00408502 -0.00870048 0.999954
+vn 0.006542 -0.999267 0.0377181
+vn 0.006542 -0.999267 0.0377181
+vn 0.006542 -0.999267 0.0377181
+vn 0.798906 -0.0450007 0.59977
+vn 0.798906 -0.0450007 0.59977
+vn 0.798906 -0.0450007 0.59977
+vn -0.00899609 -0.999957 -0.00218479
+vn -0.00899609 -0.999957 -0.00218479
+vn -0.00899609 -0.999957 -0.00218479
+vn -0.510144 -0.000216662 0.860089
+vn -0.510144 -0.000216662 0.860089
+vn -0.510144 -0.000216662 0.860089
+vn -0.998608 -0.0142745 0.0507836
+vn -0.998608 -0.0142745 0.0507836
+vn -0.998608 -0.0142745 0.0507836
+vn 0.665648 0.00209596 0.746263
+vn 0.665648 0.00209596 0.746263
+vn 0.665648 0.00209596 0.746263
+vn -0.858159 -4.90786e-05 0.513384
+vn -0.858159 -4.90786e-05 0.513384
+vn -0.858159 -4.90786e-05 0.513384
+
+# Mesh '<STL_BINARY>' with 32 faces
+g <STL_BINARY>
+usemtl DefaultMaterial
+f  1//1 2//2 3//3
+f  4//4 5//5 6//6
+f  7//7 8//8 9//9
+f  10//10 11//11 12//12
+f  13//13 14//14 15//15
+f  16//16 17//17 18//18
+f  19//19 20//20 21//21
+f  22//22 23//23 24//24
+f  25//25 26//26 27//27
+f  28//28 29//29 30//30
+f  31//31 32//32 33//33
+f  34//34 35//35 36//36
+f  37//37 38//38 39//39
+f  40//40 41//41 42//42
+f  43//43 44//44 45//45
+f  46//46 47//47 48//48
+f  49//49 50//50 51//51
+f  52//52 53//53 54//54
+f  55//55 56//56 57//57
+f  58//58 59//59 60//60
+f  61//61 62//62 63//63
+f  64//64 65//65 66//66
+f  67//67 68//68 69//69
+f  70//70 71//71 72//72
+f  73//73 74//74 75//75
+f  76//76 77//77 78//78
+f  79//79 80//80 81//81
+f  82//82 83//83 84//84
+f  85//85 86//86 87//87
+f  88//88 89//89 90//90
+f  91//91 92//92 93//93
+f  94//94 95//95 96//96
+