AIPython★ 627
raymin0223/mixture_of_recursions
MIXING RECURSIONS FOR LLM
The Mixture-of-Recursions project offers dynamic recursion depth selection per token in a transformer, enabling adaptive compute allocation. This reduces KV‑cache costs and speeds up inference without losing quality.
// KEY FEATURES
- Dynamically selects recursion depth for each token during inference.
- Reduces computation by skipping unnecessary recursive layers when the model is confident.
- Preserves quality by triggering early exit after a confidence threshold.
- Integrates with KV‑cache and works with HuggingFace Transformers via a standard API.
#adaptive-computation#early-exiting#kv-cache#llm#recursive-transformers#router
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram