R_REDDYX.XYZ
AIPython★ 2,041⑂ 240 forksApache-2.0

ovg-project/kvcached

KVCACHED: ACCELERATE GPU CACHE

Virtualized elastic KV‑cache for dynamic GPU sharing, accelerating LLM inference and reducing latency.

// KEY FEATURES

  • Dynamic GPU memory allocation among tasks
  • Compatible with popular frameworks PyTorch and TensorFlow
  • Minimal latency for cache read/write
#elastic-kvcache#gpu-mutiplexing#gpu-sharing#inference-engine#kvcache#kvcache-optimization#kvcached#llm#llm-framework#llm-inference#llm-serving#ollama
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

LLAMA.VIM: LLM RIGHT IN VIM★ 2042AI-Powered Code Protection★ 2042AWESOME LANGGRAPH: EVERYTHING UNDER MEASURE★ 2043INDEX‑TRANSLATE MULTILINGUAL AI★ 2044AI Agents for Teams★ 2045RUN NEURAL NETWORKS LOCALLY★ 2045

← FULL CATALOG

ovg-project/kvcached — Kvcached: Accelerate GPU Cache