Thank you for open-sourcing this project. I have been working with your code and found it to be very well-structured.
Optimization
While using the repository, I noticed an opportunity to further optimize the GPU kernel launch overhead. I have implemented an optimization that decouples the CPU and GPU launch processes using a shared flag variable. It reduces pipeline bubbles caused by launch overhead and improves the overall end-to-end latency.
I have already implemented and tested these changes in my fork. You can review the code here: https://github.com/nagic0/demand-layering/tree/sync_opt
Proposal
If you are interested in this optimization, could you please create a new branch named sync_opt in this repository?
I would like to submit a PR to that specific branch. This would allow other users to access this optimization experimentally without affecting the stability of the master branch immediately.
Credits
This optimization is contributed by the Institute of Parallel and Distributed Systems (IPADS).
Thank you for open-sourcing this project. I have been working with your code and found it to be very well-structured.
Optimization
While using the repository, I noticed an opportunity to further optimize the GPU kernel launch overhead. I have implemented an optimization that decouples the CPU and GPU launch processes using a shared flag variable. It reduces pipeline bubbles caused by launch overhead and improves the overall end-to-end latency.
I have already implemented and tested these changes in my fork. You can review the code here: https://github.com/nagic0/demand-layering/tree/sync_opt
Proposal
If you are interested in this optimization, could you please create a new branch named sync_opt in this repository?
I would like to submit a PR to that specific branch. This would allow other users to access this optimization experimentally without affecting the stability of the master branch immediately.
Credits
This optimization is contributed by the Institute of Parallel and Distributed Systems (IPADS).