Chair Navigation with SPOT Robot

An AI-powered robotics project that enables the Boston Dynamics SPOT robot to detect chairs, segment them using SAM2, classify chair orientation using deep learning, and autonomously navigate to a desired side of the chair.

---

🚀 Features

• 🪑 Chair detection using YOLOv11
• ✂️ Chair segmentation using SAM2
• 🧭 Chair orientation classification with TensorFlow
• 🤖 Autonomous navigation for SPOT robot
• 📷 Real-time image capture from SPOT hand camera
• 🎯 User-selected target chair interaction
• ↔️ Side-based movement logic (Front / Left / Rear / Right)

---

🏗️ Project Architecture

SPOT Camera Feed
        │
        ▼
 YOLO Chair Detection
        │
        ▼
 SAM2 Chair Segmentation
        │
        ▼
 Orientation Classification
        │
        ▼
 Navigation Decision Logic
        │
        ▼
 SPOT Movement Commands

---

📂 Project Structure

.
├── main.py
├── chairs.py
├── apply_yolo_get_chairs.py
├── apply_sam_segment_chair.py
├── classify_orientation.py
├── spot_robot.py
├── utils.py
├── sam2_hiera_l.yaml
├── model.h5
├── yolo11n.pt
└── README.md

---

⚙️ Technologies Used

• Python
• TensorFlow
• OpenCV
• YOLOv11 (Ultralytics)
• SAM2 (Segment Anything Model 2)
• Boston Dynamics SDK
• NumPy

---

🧠 How It Works

1. Chair Detection

YOLO detects chairs and benches in the scene with confidence filtering.

2. Chair Segmentation

SAM2 segments the selected chair using bounding boxes and masks.

3. Orientation Classification

A TensorFlow model predicts the chair direction:

• Front
• Left
• Rear
• Right

4. Robot Navigation

SPOT moves relative to the detected chair using movement primitives:

• Move Left
• Move Right
• Rotate Left
• Rotate Right

---

📦 Installation

Clone Repository

git clone https://github.com/yourusername/spot-chair-navigation.git
cd spot-chair-navigation

Install Dependencies

pip install -r requirements.txt

---

📋 Requirements

Example dependencies:

tensorflow
opencv-python
numpy
ultralytics
torch
bosdyn-client
bosdyn-api
bosdyn-mission

---

▶️ Usage

Run the main script:

python main.py

Workflow

1. SPOT powers on and stands
2. Camera image is captured
3. User clicks on a chair
4. Chair is segmented and classified
5. User selects destination side
6. SPOT navigates accordingly

---

🧪 Example Navigation Logic

if left_direction > right_direction:
    spot_go_right()
else:
    spot_go_left()

---

📸 Core Modules

apply_yolo_get_chairs.py

Detects chairs using YOLO.

apply_sam_segment_chair.py

Segments chairs using SAM2.

chairs.py

Chair object representation and validation.

classify_orientation.py

Predicts chair orientation.

spot_robot.py

Contains all SPOT robot control utilities.

---

⚠️ Important Notes

• Requires access to a physical SPOT robot
• Ensure robot authentication credentials are configured
• CUDA-enabled GPU recommended for SAM2 inference
• TensorFlow model (model.h5) must be trained beforehand

---

🔐 Security Warning

Avoid hardcoding robot credentials directly in source code. Use environment variables or a secure configuration system instead.

---

🛠️ Future Improvements

• Multi-chair tracking
• Dynamic obstacle avoidance
• Voice command integration
• ROS2 integration
• Real-time SLAM support
• Improved orientation classifier

---

👨‍💻 Author

Developed for robotics navigation and chair interaction research using SPOT + computer vision pipelines.

---

📄 License

This project is licensed under the MIT License.

---

⭐ Acknowledgements

• Ultralytics YOLO
• Meta SAM2
• Boston Dynamics SDK