R_REDDYX.XYZ
AIC++★ 1,108⑂ 84 forksApache-2.0

wici-ai/inferssd

RUN MODELS LARGER THAN VRAM

inferssd runs MoE models exceeding RAM and VRAM by streaming experts from SSD. Built on llama.cpp with GGUF, CUDA and Apple Silicon support. 288 stars on GitHub.

// KEY FEATURES

  • Streams MoE experts from SSD, bypassing VRAM and RAM limits
  • Runs on llama.cpp with GGUF format
  • Supports CUDA and Apple Silicon
  • Enables running models larger than available memory
#apple-silicon#cuda#gguf#inference#llama-cpp#llm#metal#mixture-of-experts#ssd
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

OPENJUDGE - NEW AI EVALUATOR★ 1105MOMO CODE: AI AGENT FOR CODE★ 1112MOTTY — SSH AND AI IN ONE WINDOW★ 1104JAMAIBASE AI TABLE FOR TEAM★ 1103Burp Suite AI Agent★ 1102VM0 Virtual AI Assistant★ 1114

← FULL CATALOG

wici-ai/inferssd — Run Models Larger Than VRAM | REDDYX AI