AIPython★ 356⑂ 132 forksMIT
ModelEngine-Group/unified-cache-management
ACCELERATE LLM WITH CACHE
The unified-cache-management project enables storing and reusing KV Cache to accelerate large language models.
// KEY FEATURES
- Persistent storage of KV Cache
- Reuse of cache
- Written in Python
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram