Skip to content

Black-Bee-Drones/nectar-sdk

BranchesTags

Repository files navigation

Nectar SDK

Bee

ROS 2 software development kit for autonomous aerial systems. Provides unified interfaces for flight control, computer vision, and object detection through a modular, extensible architecture.

Apache License Latest release Commits since Stars

ROS 2 Distro Build & Test Docker
Humble Build Humble Docker
Jazzy Build Jazzy Docker
Kilted Build Kilted Docker

Table of Contents

Features

Drone Control

Protocol-based drone interface with factory instantiation. DroneFactory creates drone implementations by key — currently MAVROS (ArduPilot/PX4) and Parrot Bebop 2, extensible to any platform. All drones implement the same Drone protocol: takeoff, land, move_to, move_to_gps, move_velocity, rtl, obstacle management.

  • Position navigation via PID control or FCU setpoints, in body, world, or takeoff reference frames
  • GPS waypoint missions with EGM96 geoid correction for AMSL altitude
  • Event-based obstacle detection with strategy-based avoidance (pause, axis disable, custom sequences)
  • Return-to-launch with PID or ArduPilot-native RTL modes
  • Per-axis PID tuning via YAML with runtime updates

Control overview · MAVROS details · Obstacles · PID · Bebop

Computer Vision

Camera abstraction and image processing. CameraFactory auto-detects the source type from a string identifier and returns the matching driver. ImageHandler wraps any camera in a ROS 2 timer loop with frame callbacks and optional OpenCV display.

  • Camera drivers: USB (OpenCV), Intel RealSense D4xx, Luxonis OAK-D, ROS 2 topics, Raspberry Pi Camera v2
  • ArUco marker detection with 6-DOF pose estimation
  • Color detection with HSV/LAB calibration and interactive trackbars
  • Line detection with five estimation methods (Hough, RANSAC, rotated rect, fit ellipse, adaptive Hough)
  • Distance estimation via six regression models (linear, polynomial, exponential, logarithmic, inverse power, robust)
  • Hand and face tracking via MediaPipe

Vision overview — camera classes, algorithm API, node parameters, calibration, module structure

AI / Detection

Object detection across three frameworks through Detector, a single factory-based entry point. Auto-detects the framework from the model path or accepts explicit selection. Models load from local files, Ultralytics Hub, or HuggingFace Hub.

AI overview · Detection details — class diagram, core types, framework configs, slicing, CLI, extension guide

Interface

Qt6 / PySide6 desktop application for testing and operating the SDK without writing code. Three tabs: Control (drone connection, arm/takeoff/land, keyboard velocity control, position navigation), Vision (camera streaming with 20+ real-time filters including ArUco detection and MediaPipe tracking), and ROS (topic browser/subscriber/publisher, service caller, parameter viewer, image subscriber).

Interface overview — architecture, tabs, widgets, threading model, camera integration, theming

Nectar Interfaces

Custom ROS 2 messages connecting vision output to control decisions:

Message Fields Published By
ArucoTransforms marker ID, translation vector, yaw ArucoNode
LineInfo center XY, angle, width, height LineDetectionNode
PhotoInfo coordinate array, photo identifier Vision nodes

Interfaces overview — message definitions, usage examples (Python/C++), building, verification

Installation

From Scratch

A standalone bootstrap script handles everything: ROS 2, system packages, MAVROS, GeographicLib, git/SSH, cloning, Python dependencies, workspace build, and verification.

bash <(curl -fsSL https://raw.githubusercontent.com/Black-Bee-Drones/nectar-sdk/main/scripts/bootstrap.sh)

Existing ROS 2 Workspace

cd ~/ros2_ws/src
git clone git@github.com:Black-Bee-Drones/nectar-sdk.git
cd nectar-sdk
make setup

Install by Module

make python-control    # GPS, PID, MAVROS navigation
make python-vision     # Camera drivers, ArUco, color, line detection
make python-ai         # YOLO, DETR, RF-DETR (requires PyTorch)
make python-interface  # Qt6 / PySide6 GUI
make pytorch           # PyTorch (auto-detects CUDA)

Interactive Menu

Run the setup script with no arguments for a guided menu where you can pick individual steps:

./scripts/setup.sh

Docker

Linux:

make docker-build       # SDK image (no AI)
make docker-build-full  # Full image (+ PyTorch + AI)
make docker-run         # Run with X11 + cameras + USB

Windows:

.\docker\run_docker_win.ps1 build humble              # Build SDK image
.\docker\run_docker_win.ps1 build jazzy full-cpu      # Build full image
.\docker\run_docker_win.ps1 run humble                # Run container
Tag Contents PyTorch
:humble control + vision + interface + realsense + oakd None
:humble-full-cpu All above + AI CPU
:humble-full-cu124 All above + AI CUDA 12.4

See docker/README.md for GPU, RealSense, and advanced options.

All versions and package lists live in scripts/lib/config.sh. See docs/INSTALL.md for the full guide.

Quick Start

Drone Control

import rclpy
from rclpy.node import Node
from nectar.control import DroneFactory, MavrosConfig, PoseSource

rclpy.init()
node = Node("flight_node")

config = MavrosConfig(pose_source=PoseSource.GPS)
drone = DroneFactory.create("mavros", config, node)

drone.takeoff(altitude=2.0)
drone.move_to(x=5.0, y=0.0, z=0.0, precision=0.3)
drone.land()

drone.cleanup()
rclpy.shutdown()

Camera Capture

import rclpy
from rclpy.node import Node
from nectar.vision import ImageHandler, OpenCVConfig

class CameraNode(Node):
    def __init__(self):
        super().__init__("camera_node")

        config = OpenCVConfig(width=1280, height=720, fps=30)
        self.handler = ImageHandler(
            node=self,
            image_source="webcam",
            config=config,
            image_processing_callback=self.process,
            show_result="Camera"
        )
        self.handler.run()

    def process(self, frame):
        pass

rclpy.init()
rclpy.spin(CameraNode())

Object Detection

from nectar.ai.detection import Detector

detector = Detector("yolov8n.pt")  # auto-detects framework
detector.load()

result = detector.detect(image, conf=0.5)
for det in result:
    print(f"{det.class_name}: {det.confidence:.2f} at {det.bbox}")

annotated = detector.draw_detections(image, result)

Architecture

High-level architecture diagram showing main components and relationships. For detailed class diagrams, see module-specific READMEs: Control, Vision, AI, Interface.

classDiagram
    namespace Control {
        class DroneFactory {
            <<singleton>>
            +create(type, config, node)$ BaseDrone
            +register(type, factory_func)$
            +available_types()$ List~str~
        }
        class Drone {
            <<protocol>>
            +connect() bool
            +arm() bool
            +takeoff(altitude) bool
            +land() bool
            +move_to(x, y, z, reference, strategy) bool
            +move_to_gps(lat, lon, alt) bool
            +move_velocity(vx, vy, vz, reference)
            +rtl(strategy) bool
            +emergency_stop()
        }
        class BaseDrone {
            <<abstract>>
            #_obstacle_manager ObstacleManager
            +add_obstacle_detector(name, detector, strategy)
            +enable_obstacle_detector(name)
            +cleanup()
        }
        class MavrosDrone {
            -_navigator MavrosNavigator
            -_pose_source PoseSource
            +set_mode(mode) bool
            +set_pid_config(config)
            +publish_setpoint(target)
            +get_altitude(source) Optional~float~
        }
        class BebopDrone {
            +flip(direction)
            +camera_control(tilt, pan)
        }
        class MavrosNavigator {
            +navigate_pid(target, precision) bool
            +navigate_setpoint(target, precision) bool
        }
        class PIDController {
            +update(current_value) float
            +reset()
            +tune(kp, ki, kd)
        }
        class ObstacleManager {
            +should_continue_navigation(drone) bool
            +get_axis_control() tuple
            +add(name, handler)
            +enable(name)
        }
        class AvoidanceStrategy {
            <<abstract>>
            +execute(drone, info) bool
        }
        class GPSUtils {
            <<static>>
            +geoid_height(lat, lon)$ float
            +create_gps_setpoint(lat, lon, alt, heading)$ GeoPoseStamped
            +check_reached(cur, tgt, precision)$ tuple
        }
    }

    namespace Vision {
        class CameraFactory {
            <<singleton>>
            +from_source(source, config, node)$ AbstractCam
            +register(key, builder)$
        }
        class AbstractCam {
            <<abstract>>
            +name str
            +is_running bool
            +start()
            +get_frame() Optional~ndarray~
            +close()
        }
        class DepthCam {
            <<abstract>>
            +get_depth_frame() Optional~ndarray~
            +get_distance(u, v) Optional~float~
        }
        class OpenCVCam {
            +start()
            +get_frame() Optional~ndarray~
        }
        class RealsenseCam {
            +start()
            +get_frame() Optional~ndarray~
            +get_depth_frame() Optional~ndarray~
        }
        class OakdCam {
            +start()
            +get_frame() Optional~ndarray~
            +get_depth_frame() Optional~ndarray~
        }
        class ROSCam {
            +start()
            +get_frame() Optional~ndarray~
        }
        class ROSDepthCam {
            -_color_cam ROSCam
            +get_frame() Optional~ndarray~
            +get_depth_frame() Optional~ndarray~
        }
        class ImageHandler {
            +node Node
            +camera AbstractCam
            +run()
            +take_photo() ndarray
            +cleanup()
        }
        class Aruco {
            +marker_dict int
            +tag_size float
            +detect(img, draw) tuple
            +pose_estimate(img, draw) tuple
        }
        class ColorDetector {
            +filterColor(img)
            +saveColorValues()
            +get_color_values(name) list
        }
        class LineDetector {
            -color_detector ColorDetector
            -estimation_method ILineEstimationMethod
            +detect_line(img, region) tuple
        }
        class ILineEstimationMethod {
            <<abstract>>
            +estimate(img) tuple
        }
        class DistanceEstimator {
            -_model EstimationModel
            +estimate(value) float
            +compare_methods(value) Dict
        }
        class ModelCalibrator {
            +fit_all() Dict
            +best_model(criterion) CalibrationResult
            +save_params(path)
        }
        class EstimationModel {
            <<abstract>>
            +estimate(value) float
            +fit(x, y) Dict
        }
        class HandTracker {
            +detect(frame, draw) ndarray
            +get_hands() List~HandResult~
            +raised_fingers(hand_idx) List~int~
        }
        class FaceMeshTracker {
            +detect(frame, draw) ndarray
            +get_faces() List~FaceResult~
            +get_eye_aspect_ratio(eye) float
        }
    }

    namespace AI {
        class Detector {
            +model_source str
            +framework Framework
            +load() bool
            +detect(image, conf) DetectionResult
            +train(config) TrainingResult
            +evaluate(config) EvaluationMetrics
            +draw_detections(image, result) ndarray
            +register(framework, builder)$
            +enable_slicing(config)
        }
        class BaseDetectionModel {
            <<abstract>>
            +model_name str
            +framework str
            +is_loaded bool
            +load_model(path)
            +detect(image, conf, iou) DetectionResult
            +train(config) TrainingResult
            +save(path) str
        }
        class UltralyticsModel {
            +load_model(path)
            +train(config) TrainingResult
        }
        class TransformersModel {
            +load_model(path)
            +train(config) TrainingResult
        }
        class RFDETRModel {
            +load_model(path)
            +train(config) TrainingResult
        }
        class ModelRegistry {
            <<singleton>>
            +register(name, model_class)$
            +create(name, model_name)$ BaseDetectionModel
        }
        class Detection {
            <<dataclass>>
            +xyxy ndarray
            +confidence float
            +class_id int
            +class_name str
        }
        class DetectionResult {
            <<dataclass>>
            +detections List~Detection~
            +filter_by_confidence(threshold) DetectionResult
            +filter_by_class_id(class_ids) DetectionResult
            +to_supervision() sv.Detections
        }
        class SlicingInference {
            -config SlicingConfig
            +run_sliced_inference(image, callback) sv.Detections
        }
        class BaseMergingStrategy {
            <<abstract>>
            +merge_boxes(detections) Tuple
        }
        class NMSStrategy {
            +merge_boxes(detections) Tuple
        }
        class WBFStrategy {
            +merge_boxes(detections) Tuple
        }
        class ObjectDetectionEvaluator {
            +evaluate() EvaluationMetrics
        }
    }

    namespace Interface {
        class NectarApp {
            -_ros_executor ROSExecutor
            -_control_tab ControlTab
            -_vision_tab VisionTab
            -_ros_tab ROSTab
            +main()$
        }
        class ROSExecutor {
            +node Node
            +start(node_name) bool
            +shutdown()
        }
        class ControlTab {
            +set_node(node)
            +cleanup()
        }
        class VisionTab {
            +set_node(node)
            +cleanup()
        }
        class ROSTab {
            +set_node(node)
            +cleanup()
        }
    }

    namespace Utils {
        class GPSCalculate {
            <<static>>
            +haversine(lat1, lon1, lat2, lon2)$ float
            +bearing(lat1, lon1, lat2, lon2)$ float
            +calculate_gps_offset(x, y, z, lat, lon, alt, heading)$ tuple
            +interp_geo(start, end, frac)$ tuple
        }
        class PositionUtils {
            <<static>>
            +get_body_distance(target, current, heading)$ tuple
            +get_yaw_from_pose(pose)$ float
            +convert_position_to_target(pose, heading, lidar)$ Union
            +transform_takeoff_to_body_velocities(vx, vy, vz, current_yaw, takeoff_yaw)$ tuple
        }
        class ProcessUtils {
            <<static>>
            +start_process(command, name, gui) bool
            +kill_process(name) bool
            +is_node_running(node_pattern) bool
            +wait_for_node(node_pattern, timeout) bool
        }
    }

    Drone <|.. BaseDrone : implements
    BaseDrone <|-- MavrosDrone
    BaseDrone <|-- BebopDrone
    DroneFactory ..> BaseDrone : creates
    MavrosDrone *-- MavrosNavigator
    MavrosNavigator ..> PIDController : creates
    MavrosNavigator ..> GPSUtils : uses
    MavrosNavigator ..> PositionUtils : uses
    BaseDrone o-- ObstacleManager
    ObstacleManager ..> AvoidanceStrategy

    AbstractCam <|-- DepthCam
    AbstractCam <|-- OpenCVCam
    AbstractCam <|-- ROSCam
    DepthCam <|-- RealsenseCam
    DepthCam <|-- OakdCam
    DepthCam <|-- ROSDepthCam
    ROSDepthCam *-- ROSCam
    CameraFactory ..> AbstractCam : creates
    ImageHandler o-- AbstractCam
    ImageHandler ..> CameraFactory
    LineDetector o-- ILineEstimationMethod
    LineDetector o-- ColorDetector
    DistanceEstimator o-- EstimationModel
    DistanceEstimator ..> ModelCalibrator : uses

    BaseDetectionModel <|-- UltralyticsModel
    BaseDetectionModel <|-- TransformersModel
    BaseDetectionModel <|-- RFDETRModel
    Detector ..> BaseDetectionModel : creates
    Detector ..> ModelRegistry : uses
    ModelRegistry ..> BaseDetectionModel : creates
    BaseDetectionModel ..> DetectionResult : returns
    DetectionResult o-- Detection
    BaseDetectionModel ..> SlicingInference : uses
    SlicingInference ..> BaseMergingStrategy : uses
    BaseMergingStrategy <|-- NMSStrategy
    BaseMergingStrategy <|-- WBFStrategy
    BaseDetectionModel ..> ObjectDetectionEvaluator : uses

    NectarApp *-- ROSExecutor
    NectarApp *-- ControlTab
    NectarApp *-- VisionTab
    NectarApp *-- ROSTab
    ControlTab ..> ROSExecutor : uses
    VisionTab ..> ROSExecutor : uses
    ROSTab ..> ROSExecutor : uses
Loading

Design Patterns

The codebase uses the same patterns across all modules, making it predictable to navigate and extend:

Pattern Where What It Does
Factory + Registry DroneFactory, CameraFactory, Detector Decouples creation from usage. New types are registered at runtime and instantiated by key.
Protocol Drone, ObstacleDetector Defines interfaces via structural typing (duck typing). Any class matching the signature is accepted.
Strategy AvoidanceStrategy, ILineEstimationMethod, EstimationModel, BaseMergingStrategy Encapsulates interchangeable algorithms behind a common interface.
Abstract Base Class BaseDrone, AbstractCam, DepthCam, BaseDetectionModel Shares common logic and enforces method contracts for concrete implementations.
Dataclass Config MavrosConfig, OpenCVConfig, TrainingConfig, EvaluationConfig Type-safe configuration with defaults, validation, and YAML serialization.

Every factory supports runtime registration, so adding a new drone type, camera driver, or detection framework follows the same pattern:

# New drone
DroneFactory.register("custom", lambda cfg, node: MyDrone(cfg, node))
drone = DroneFactory.create("custom", config, node)

# New camera
CameraFactory.register("thermal", ThermalCamera)
camera = CameraFactory.from_source("thermal")

# New detection framework
Detector.register("custom", lambda name, **kw: CustomModel(name, **kw))
detector = Detector("model.bin", framework="custom")

Documentation

Document Contents
Installation Guide Bootstrap, workspace setup, module install, PyTorch, RealSense, Docker, troubleshooting
Control Module Drone protocol, factory, configuration, movement API, obstacle system
MAVROS Implementation MAVLink, flight modes, coordinate frames, altitude types, PID vs setpoint, GPS utilities, ROS 2 topics/services
Bebop Implementation Bebop 2 control, velocity, acrobatics
Obstacle Detection Detector protocol, avoidance strategies, handler configuration
PID Controller Tuning guide, YAML config, runtime updates, default indoor/outdoor configs
Vision Module Camera drivers, ArUco, color, line, distance, MediaPipe, nodes, calibration
AI Module Detector API, training, evaluation, device management
Detection Module Class diagram, core types, framework configs, slicing, CLI, extension guide
Interface Module GUI tabs, widgets, threading model, camera integration, theming
Nectar Interfaces ROS 2 message definitions, Python/C++ usage
Docker Guide Build variants, GPU, RealSense, dependency strategy
Contributing Development setup, code style, PR process
Releasing Version bump, CI workflows, Docker Hub push

ROS 2 Nodes

Pre-built nodes for common tasks:

# GUI
ros2 run nectar app.py

# ArUco detection
ros2 run nectar aruco_node.py --ros-args \
    -p image_source:=webcam -p marker_dict:=5 -p tag_size:=0.05

# Line detection
ros2 run nectar line_detection_node.py --ros-args \
    -p line_colors:="blue,red" -p method:=HoughLinesP

# Color calibration
ros2 run nectar color_calibration_node.py --ros-args -p image_source:=webcam

# Camera calibration
ros2 run nectar calibration.py --ros-args -p chessboard_size:="9,7"

# Webcam publisher
ros2 run nectar webcam_publisher_node.py --ros-args -p width:=1280 -p height:=720

# Object detection
ros2 run nectar detector_example.py --ros-args -p model_source:=yolov8n.pt

Examples

Working examples in nectar/nectar/examples/:

Example Description
basic.py Takeoff, velocity, land
sensors.py Monitor GPS/vision data
pid_simulation.py PID controller simulation
mavros_navigation.py Navigation test (BODY, TAKEOFF, GPS references)
mavros_obstacles.py Obstacle avoidance
camera_example.py Camera capture
depth_example.py Depth visualization
detector_example.py Object detection
batch_detector.py Batch image/video processing

See: control examples · vision examples · AI examples

Directory Structure

nectar-sdk/
├── scripts/                    # Setup and installation
│   ├── bootstrap.sh            # Standalone curl installer
│   ├── setup.sh                # CLI + interactive menu
│   └── lib/                    # Modular shell functions
│       ├── config.sh           # Versions, packages (single source of truth)
│       ├── common.sh           # Logging
│       ├── system.sh           # apt packages
│       ├── ros2.sh             # ROS 2 install + env
│       ├── python.sh           # pip from pyproject.toml
│       ├── realsense.sh        # Intel RealSense from source
│       ├── workspace.sh        # Build, clean, verify
│       └── git.sh              # Git/SSH setup
├── docker/
│   ├── Dockerfile              # x86_64: sdk + sdk-full stages
│   └── Dockerfile.jetson       # ARM64: Jetson Orin Nano
├── docs/
│   ├── INSTALL.md              # Full installation guide
│   ├── CONTRIBUTING.md         # Development setup, code style, PR process
│   ├── RELEASING.md            # Version bump, CI, Docker Hub push
│   └── SECURITY.md
├── nectar_interfaces/          # ROS 2 custom messages
│   ├── CMakeLists.txt
│   ├── package.xml
│   └── msg/
├── nectar/                     # Main ROS 2 package
│   ├── CMakeLists.txt
│   ├── package.xml
│   ├── pyproject.toml
│   └── nectar/                 # Python package
│       ├── control/            # Drone control
│       ├── vision/             # Computer vision
│       ├── ai/                 # AI / detection
│       ├── interface/          # Qt6 GUI
│       ├── examples/
│       └── utils/
├── Makefile
└── README.md

Contributing

We welcome contributions. Please see docs/CONTRIBUTING.md for development setup, code style (PEP 8, NumPy docstrings, type hints), and the PR process.

  1. Check GitHub Issues for existing discussions
  2. Follow Conventional Commits for commit messages
  3. Read our Code of Conduct
Contributors

Black Bee Drones

Black Bee Drones is Latin America's first academic autonomous drone team. Founded as a research project at the Federal University of Itajuba (UNIFEI) in 2014, the team develops unmanned aircraft for complex missions requiring computer vision and artificial intelligence. We compete nationally and internationally, including IMAV (3rd place indoor 2023 and 2025, 3rd place outdoor 2015, best autonomous indoor flight worldwide), and participate in events like DroneShow LA and Campus Party.

Nectar SDK started in 2023 as a way to stop rewriting the same camera, PID, and detection code for each competition mission. It evolved into a ROS 2 package with consistent interfaces across drone control, computer vision, and AI detection — the shared foundation our team uses for every project. It has been turned into an open-source project to enable continuous development and to help other teams and researchers build autonomous systems more rapidly.

Acknowledgments

Nectar SDK exists because of the open source projects it builds on. We are grateful to the ROS 2 community and Open Robotics for the middleware that connects everything. MAVROS and the MAVLink protocol for bridging ROS 2 with ArduPilot and PX4 flight controllers. OpenCV for the computer vision foundation. Ultralytics, HuggingFace, and Roboflow for making object detection accessible through YOLO, Transformers, RF-DETR, and supervision. PyTorch for the deep learning backend. Intel RealSense and Luxonis for depth camera SDKs. Google MediaPipe for hand and face tracking. Qt for Python for the GUI framework.

License

This project is licensed under the Apache-2.0 License — see the LICENSE file for details.

About

ROS2 SDK designed to simplify drone control and computer vision tasks

Topics

python opencv computer-vision drone ardupilot mavros ros2

Resources

Readme

License

Apache-2.0 license

Code of conduct

Code of conduct

Contributing

Contributing

Security policy

Security policy
Activity
Custom properties

Stars

7 stars

Watchers

2 watching

Forks

0 forks
Report repository

Releases 2

Nectar SDK (v1.0.0) - A New Hope Latest
Feb 18, 2026
+ 1 release

Contributors

Languages