R_REDDYX.XYZ
AIPython★ 1,254⑂ 200 forksApache-2.0

NVIDIA/kvpress

KVPRESS: LLM CACHE COMPRESSION

Library for easy compression of LLM KV‑cache, accelerating generation and reducing GPU memory consumption.

// KEY FEATURES

  • Automatic KV‑cache compression without loss of accuracy
  • Compatible with HuggingFace, vLLM and other engines
  • Minimalist API: a single function compress_kvcache
#inference#kv-cache#kv-cache-compression#large-language-models#llm#long-context#python#pytorch#transformers
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

MAGIC: AI ASSISTANT FOR DEVELOPMENT★ 1255MULTIMODAL AI FOR CONTENT★ 1253LLMGATEWAY: UNIFIED API FOR ALL LLM★ 1252ANGELSLIM: COMPRESSING MODELS WITHOUT LOSS★ 1257FAST ALTERNATIVE FIRECRAWL IN RUST★ 1257BRIAN2 SPIKING NETWORK SIMULATOR★ 1258

← FULL CATALOG

NVIDIA/kvpress — Kvpress: LLM Cache Compression | REDDYX AI