Simulation of multiple vehicle to test swarming

[NachoMurillo]

Hello,

First of all, I would like to congratulate you for this fantastic repo. I have a small doubt about running the simulation with several drones at the same time, in order to test some swarming algorithms and train RL models for this purpose.

Have you ever considered this option? It would be quick to adapt the code to make it possible?

Thank you in advance for your response.

[spencerfolk]

Hey thanks for this suggestion. I've been thinking about multi agent sim lately as a few others have inquired about it. At face value, I think this would be very easy to implement, but the challenge will be making the code run fast enough especially for RL.

I think Python's built in multiprocessing module is a step in the right direction as each agent could run on its own process, but the issue will be how and how often agents communicate information between each other. I think the most recent version of Python got rid of the GIL though!

Questions for you @NachoMurillo :

1. How many agents are you interested in simulating at once? 5? 10? 100?
2. What is the nature of the agent-agent communication? Are you looking for centralized/decentralized communication and planning capabilities?
3. Is aerodynamics important for your simulations? Without aero the simulations are much faster and will scale better to swarms.
4. Similar to question 3, do you care about rotor wash and other agent-agent effects?

[NachoMurillo]

Thanks for your quick response:

1. For the use case I had thought it would be enough to start with 5-10 agents, but it would be very interesting to simulate as many as possible, although as you say it will be limited by the computation.
2. I suppose the quickest way is to assume that it is a centralised scheme and that all agents have information about each other, although it would be very productive to be able to simulate a decentralised communication in which agents can only communicate at certain distances or have a certain probability of communication failures. What are you think about this point?
3. I think the aerodynamics can be left out for this part in order to speed up the simulations. I also think it could be complicated to simulate the turbulence generated by the agents between them. However, for practical purposes it could be very useful to take it into account in the future.
4. Replied in 3.

[spencerfolk]

Thanks for your input on requirements. I think 5-10 agents is pretty manageable, but I'll have to confirm the memory requirements of a single agent to understand at what point the simulator would start eating up too much memory for the multi agent case.

I think a first pass at this would implement some sort of centralized simulation and communication scheme that then the end user can decide ultimately what information each agent uses. Essentially there's a "ground truth communication channel" and things like communication delay, noise, max communication range, etc., can be modeled by the end user. Does that seem ok?

As for aero, we can leave this as a bool to turn on/off but no guarantees on performance when aero is on.

@NachoMurillo I'm happy to start development on this but no timeline can be guaranteed. Happy to help if you'd like to start a new branch on the repo. I think a good base to build off of is the quadrotor gym environment from the most recent update.

[NachoMurillo]

Do you mean to adapt quadrotor gym environment in order to manage, for example, several Multirotor objects and step all of them in different threads?

[spencerfolk]

Yep. Right now I'm imagining a base class that instantiates N instances of the quadrotor gym environment on individual processes (using multiprocessing?). Then agent communication can be handled by the base class.

Major bottleneck will probably be data transfer between processes. Although with Python 3.13 the GIL is removed, so I wonder if that would help.

[spencerfolk]

@NachoMurillo not sure if you've made any progress on this but I wanted to point you to a new example script I just added in 12c5ff4.

It demonstrates formation flight of 10 MAVs. With Python's multiprocessing I'm getting around 1x real time factor on my laptop with 10 agents. Keep in mind in that script the mavs can't communicate with each other or with a global planner. Despite this, I think it might be a good start to demonstrate how to run many instances of quadrotors in an environment.

[NachoMurillo]

Hello, thank you for the script, it is a great start. I have not been able to make much progress on the subject, but I would like to integrate this multiprocessing into the reinforcement learning environment when I have more time. Thanks again for your interest in this enhancement.