Skip to content

feat: run:ai gpu_direct model loading with quantization-aware device maps - #635

Open
khaiwang wants to merge 5 commits into
devfrom
zikai/runai-gpu-direct
Open

feat: run:ai gpu_direct model loading with quantization-aware device maps#635
khaiwang wants to merge 5 commits into
devfrom
zikai/runai-gpu-direct

fix: use from_pretrained for TP instead of run:ai CPU-clone

cbd93ba
Select commit
Loading
Failed to load commit list.

Select a check to view from the sidebar