AIC++★ 1,329⑂ 126 forks
kvmem/kvmem
KV Memory Virtualizer
KVMem virtualizes KV memory across backends, enabling ultra‑long‑context LLMs on low‑RAM hardware by transparently paging the cache.
// KEY FEATURES
- Virtualizes key-value memory across multiple inference backends
- Supports ultra-long-context LLMs on devices with limited RAM
- Reduces memory footprint by transparently paging KV cache
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram