AIPython★ 778⑂ 99 forksApache-2.0
thu-nics/C2C
CACHE-TO-CACHE: DIRECT LLM EXCHANGE
Implementation of the Cache-to-Cache algorithm for direct semantic exchange between large language models. Enables transferring KV‑cache without deserialization speeding up multi‑agent interaction. Supports integration with any LLM in Python.
// KEY FEATURES
- Transfers KV‑cache directly between LLMs without intermediate text.
- Reduces latency of multi‑agent dialogue to milliseconds.
- Compatible with any HuggingFace Transformers models via a simple API.
- Allows adjusting the compression degree of semantic cache for accuracy‑speed balance.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram