R_REDDYX.XYZ
AIPython★ 159

huawei-csl/KVarN

KVarN: KV-Cache Quantization

KVarN is a native KV-cache quantization backend for vLLM, increasing context by 3-5x, raising throughput above FP16 and maintaining FP16 accuracy without calibration, activated by one flag.

// KEY FEATURES

  • Native KV-cache quantization for vLLM without external dependencies
  • 3-5x increase in available context compared to baseline
  • Throughput above FP16 while preserving FP16 accuracy
  • Enabled with one flag; no calibration required
#agentic-ai#kv-cache#llm#llm-inference#long-context#quantization#vllm
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

Weight-Direction One-Step Image Synthesis★ 158Open Super Agent★ 158Shizuku Apps Catalog★ 160Tokyo 3D Browser Model★ 158GTA IV Native on Apple Silicon★ 160CJK-First Agent Memory Module★ 160

← FULL CATALOG

huawei-csl/KVarN — KVarN: KV-Cache Quantization | REDDYX AI