feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype - #103
Open
lucaspirola wants to merge 2 commits into
Open
feat(kvcache): 8-bit KV cache (q8_0 / fp8_e4m3) behind --kv-cache-dtype#103lucaspirola wants to merge 2 commits into
lucaspirola wants to merge 2 commits into
