AIC++★ 283⑂ 69 forksApache-2.0
alibaba/tair-kvcache
TAIR-KVCACHE: ACCELERATING LLM INFERENCE
A high-performance KV caching system from Alibaba Cloud that accelerates LLM inference.
// KEY FEATURES
- Global cache management to reduce latency
- HiSim inference simulation for testing
- Written in C++ for maximum performance
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram