R_REDDYX.XYZ
AIRust★ 231⑂ 29 forks

RustingStudio/RustingStrata

PURE-RUST ENGINE FOR QWEN GGUF

RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.7x llama.cpp on RTX 3060 12GB.

// KEY FEATURES

  • Runs Qwen3–Qwen3.8 dense and MoE up to 770B+ in GGUF
  • CUDA/Vulkan on NVIDIA/AMD, hybrid CPU+GPU experts
  • MTP speculative decoding accelerates generation
  • OpenAI/Anthropic API server, up to 1.7x llama.cpp on 3060
#anthropic-api#cuda#gguf#inference#llama-cpp#llm#local-llm#mixture-of-experts#moe#mtp#openai-api#qwen
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

Claude for German law★ 231HEALTHCHAIN: AI FOR MEDICAL DATA★ 231GOGRAPH CODE ANALYSIS FOR AI★ 231SIMPLERLLM: Easy LLM Use★ 231KUBETERM: KUBERNETES AT HAND★ 231GO-WIND CMS: FULLY GO STACK★ 231

← FULL CATALOG

RustingStudio/RustingStrata — Pure-Rust Engine For Qwen GGUF