AIPython★ 8,292
LMCache/LMCache
LMCache: LLM Acceleration
The fastest KV cache layer for large language models, boosting performance and reducing latency.
// KEY FEATURES
- KV cache optimization
- Reduced inference latency
- Support for scalable LLMs
#amd#cuda#fast#inference#kv-cache#llm#pytorch#rocm#speed#vllm
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram