AIC++★ 1,176⑂ 206 forksMIT
gufo-org/gufo
GUFO: INFERENCE ENGINE FOR STRIX HALO
gufo — inference engine for AMD Strix Halo in C++. Qwen Flash Next Q4_K_XL delivers 59.41 tok/s (1 user) and 157.22 tok/s (8 users). Qwen27B Q4_K_XL with DFlash2 gives 70.56 tok/s. 210 stars on GitHub.
// KEY FEATURES
- Runs LLM on AMD Strix Halo via DFlash2
- Qwen Flash Next Q4_K_XL: 59.41 tok/s (1 user)
- Scales to 157.22 tok/s for 8 users
- Qwen27B Q4_K_XL delivers 70.56 tok/s
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram