AIPython★ 1,287⑂ 43 forks
eagle-hq/EAGLE-4
EAGLE-4 Speculative Decoding
Official inference code for EAGLE-4, a speculative decoding method that accelerates LLM generation by drafting tokens via feature-level extrapolation.
// KEY FEATURES
- Drafts tokens using feature extrapolation from previous hidden states
- Reduces LLM inference latency with minimal quality degradation
- Supports Llama, Vicuna, and other transformer-based models
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram