Hello Aymen and others,
After running into performance problems with DragonHPC on a different project, I created a small benchmark that tests task throughput.
See the code here: https://github.com/radical-cybertools/digital.twins/blob/devel/perf/af.py
I'm testing the overall task throughput (func calls per second) that can run. Right now, switching to rhapsody's Dragon backend results in a throughput cut of about 22x.
On Perlmutter, 1 compute node:
ConcurrentExecutionBackend: runs 2200 tasks per second
DragonExecutionBackend: 100 tasks per second
Note, though I use AsyncFlow, I'm focusing on backend throughput.
The importance is that in a digital twin context, we have many inference tasks that are spawned continuously. Then, combined with simulation tasks, surrogate training tasks, and other tasks, dragon execution on task throughput quickly becomes a bottleneck. (In the digital twin application, I've been measuring about 1 second latency for inference task submission)
Hello Aymen and others,
After running into performance problems with DragonHPC on a different project, I created a small benchmark that tests task throughput.
See the code here: https://github.com/radical-cybertools/digital.twins/blob/devel/perf/af.py
I'm testing the overall task throughput (func calls per second) that can run. Right now, switching to rhapsody's Dragon backend results in a throughput cut of about 22x.
On Perlmutter, 1 compute node:
ConcurrentExecutionBackend: runs 2200 tasks per second
DragonExecutionBackend: 100 tasks per second
The importance is that in a digital twin context, we have many inference tasks that are spawned continuously. Then, combined with simulation tasks, surrogate training tasks, and other tasks, dragon execution on task throughput quickly becomes a bottleneck. (In the digital twin application, I've been measuring about 1 second latency for inference task submission)