Thank you for releasing the paper and code. I have three questions about the HiSVD + GPTQ-4bit result in Table 5:
-
What HiSVD compression ratio or rank budget was used before quantization to achieve 2.8 GB memory?
-
In compress.py, svd_u and svd_v are multiplied and written back as a dense weight. This does not reduce storage or inference memory. Is this operation only used for evaluation, while actual deployment stores the two low-rank factors separately?
-
Did GPTQ quantize svd_u and svd_v separately, or quantize the reconstructed dense matrix svd_u @ svd_v? Is there a significant accuracy difference between these two approaches?
Could you provide the quantization code or configuration used for Table 5?
Thank you for releasing the paper and code. I have three questions about the HiSVD + GPTQ-4bit result in Table 5:
What HiSVD compression ratio or rank budget was used before quantization to achieve 2.8 GB memory?
In compress.py, svd_u and svd_v are multiplied and written back as a dense weight. This does not reduce storage or inference memory. Is this operation only used for evaluation, while actual deployment stores the two low-rank factors separately?
Did GPTQ quantize svd_u and svd_v separately, or quantize the reconstructed dense matrix svd_u @ svd_v? Is there a significant accuracy difference between these two approaches?
Could you provide the quantization code or configuration used for Table 5?