AIPython★ 1,370
0xSero/turboquant
KV-Cache Quantization for LLM
KV-cache quantization using Triton kernels and vLLM integration: 3-bit keys and 2-bit values for efficient large language model operation.
// KEY FEATURES
- 3-bit keys and 2-bit values
- Optimization via Triton kernels
- vLLM integration
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram