hf/bench.py: need to specify bfloat16 otherwise it consumes twice as much memory in A10. - #9

hf/bench.py: need to specify bfloat16 otherwise it consumes twice as much memory in A10.#9

ivanbaldo wants to merge 1 commit into

hamelsmu:masterhamelsmu/llama-inference:masterfrom

ivanbaldo:hf-bench-bfloat16ivanbaldo/llama-inference:hf-bench-bfloat16Copy head branch name to clipboard