Official project repository for:
MAPPO-dMPC: Efficient Differentiable Model Predictive Control-Based Multi-Agent Reinforcement Learning for Drone Swarms
This project provides the core implementation of a bi-level MARL + differentiable MPC framework for agile drone swarm formation and dynamic gate traversal.
MAPPO-dMPC combines:
- Outer loop (MARL / MAPPO): learns high-level intent by outputting optimization parameters.
- Inner loop (Differentiable MPC): solves a physics-informed flatness-based LQR problem to generate smooth, dynamically feasible control actions.
- End-to-end training: gradients are propagated through the differentiable control layer.
Comparison of training efficiency across prediction horizons (
| Method | H=2 Time | H=2 Speed (FPS) | H=2 Fwd (s) | H=5 Time | H=5 Speed (FPS) | H=5 Fwd (s) | H=10 Time | H=10 Speed (FPS) | H=10 Fwd (s) |
|---|---|---|---|---|---|---|---|---|---|
| MAPPO-MLP | -- | -- | -- | -- | -- | -- | 17.8h | 9567 | 0.0013 |
| MAPPO-dMPC | 19.7h | 8370 | 0.0043 | 20.4h | 8221 | 0.0049 | 21.3h | 7910 | 0.0061 |
| MAPPo-Acados | 5d 8.2h | 1379 | 0.0028 | 15d 5.5h | 1242 | 0.0039 | 23d 0h | 876 | 0.0078 |
| MAPPO-Theseus | 89d 18.7h | 271 | 0.1167 | 108d 16.1h | 202 | 0.1296 | 130d 11.7h | 168 | 0.1447 |
| MAPPO-MPC.pytorch | 34d 4.5h | 501 | 0.0160 | 164d 8.4h | 104 | 0.0380 | 423d 21.7h | 45 | 0.0780 |
Note: “Time” denotes projected total training time for
$2 \times 10^8$ steps. “Speed” is FPS during training. “Fwd” is forward pass time.
Compared with the differentiable MPC + RL pipeline based on MPC.pytorch reported in:
Romero A, Aljalbout E, Song Y, et al. Actor–Critic Model Predictive Control: Differentiable Optimization Meets Reinforcement Learning for Agile Flight. IEEE Transactions on Robotics, 2025, 42: 673–692.
our MAPPO-dMPC implementation is well beyond 35× faster in training wall-clock efficiency under the same
- at
$H=2$ : about 41.6× faster (34d 4.5h vs 19.7h), - at
$H=5$ : about 193× faster (164d 8.4h vs 20.4h), - at
$H=10$ : about 478× faster (423d 21.7h vs 21.3h).
This repository has been slimmed to keep the main training path for:
python3 train.py
- The project is intended to be run on Linux with NVIDIA Isaac Sim compatible setup.
- Training logs are redirected to
scripts/logs/. - This is a focused research codebase for the MAPPO-dMPC pipeline
Coming soon.
This project is distributed under the license specified in LICENSE.