AIC++★ 144⑂ 9 forksMIT
Betelgeuse-AI/Betelgeuse
MoE Inference on Consumer GPUs
Runs frontier-scale MoE models on RTX 3060 (12GB VRAM) by keeping dense layers on GPU and swapping experts from NVMe into a bounded VRAM pool.
// KEY FEATURES
- Offloads routed experts to NVMe, loads only active experts into 12GB VRAM pool
- Keeps dense computation resident on GPU for low-latency token generation
- Supports frontier-scale MoE models on 16GB RAM + consumer GPU hardware
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram