AIRust★ 231⑂ 29 forks
RustingStudio/RustingStrata
PURE-RUST ENGINE FOR QWEN GGUF
RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.7x llama.cpp on RTX 3060 12GB.
// KEY FEATURES
- Runs Qwen3–Qwen3.8 dense and MoE up to 770B+ in GGUF
- CUDA/Vulkan on NVIDIA/AMD, hybrid CPU+GPU experts
- MTP speculative decoding accelerates generation
- OpenAI/Anthropic API server, up to 1.7x llama.cpp on 3060
#anthropic-api#cuda#gguf#inference#llama-cpp#llm#local-llm#mixture-of-experts#moe#mtp#openai-api#qwen
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram