Project Focus: Improve the throughput of tree-based speculative decoding by training custom overhead policies that guide tree generation.
Team Members: Nishchay Jasuja, Alistair Cheong, Neil Purohit
jasujanish/speculative_decoding_speedup
Folders and files
| Name | Name | Last commit date | ||
|---|---|---|---|---|