R_REDDYX.XYZ
AIPython★ 1,370

0xSero/turboquant

KV-Cache Quantization for LLM

KV-cache quantization using Triton kernels and vLLM integration: 3-bit keys and 2-bit values for efficient large language model operation.

// KEY FEATURES

  • 3-bit keys and 2-bit values
  • Optimization via Triton kernels
  • vLLM integration
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

awesome-deepseek-harness - SPEEDING WORK WITH DEEPSEEK★ 1370REVERSE ENGINEERING LAB★ 1371meta-ads-mcp - META ADS FOR AI★ 1369VISION AND LANGUAGE: TOP MODELS★ 1368AI Model Price Finder★ 1372AI CODE SAFETY★ 1373

← FULL CATALOG

0xSero/turboquant — KV-Cache Quantization for LLM