Add fp8 KV-cache quantization (--kv-dtype) - #126

Enhance your code review process with GitHub Actions

GitHub Actions make it easy to automate all your software workflows, now with world-class CI/CD.
Build, test, and deploy your code right from GitHub. Learn more about GitHub Actions.

Linux, macOS, Windows, and containers

Matrix builds

Any language

Live logs

Built-in secret store

Multi-container testing