AIC++★ 1,108⑂ 84 forksApache-2.0
wici-ai/inferssd
RUN MODELS LARGER THAN VRAM
inferssd runs MoE models exceeding RAM and VRAM by streaming experts from SSD. Built on llama.cpp with GGUF, CUDA and Apple Silicon support. 288 stars on GitHub.
// KEY FEATURES
- Streams MoE experts from SSD, bypassing VRAM and RAM limits
- Runs on llama.cpp with GGUF format
- Supports CUDA and Apple Silicon
- Enables running models larger than available memory
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram