How to eval based quant model? #8
Description
Hi，you have provided GPTQ int4 quant model in https://huggingface.co/LLMQ/LLaMA-3-8B-GPTQ-4bit-b128/tree/main, How to evaluate metric in your code?
I have try run "sh scripts/eval_gptq_commonsenseqa.sh", but it only evaluate FP32 model metric，when remove '--quant_method gptq' in script, it meet error like this.
Can you provide solution for this question? Thanks.