Mixture Of Experts: open-source projects on GitHub
20 repositories on Mixture Of Experts, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
DeepSpeed: optimization library for training
DeepSpeed is an optimization deep learning library ensuring efficient distributed training and inference. Supports ZeRO, MoE and parallelism for training models with hund
★ 43,332 · PythonKIMI K3: 2.78 Trillion Parameters
Running the Kimi K3 model with 2.78 trillion parameters on a single CPU, using only 8.24 GB of RAM without BLAS, frameworks, or GPU.
★ 12,109 · CAI Infrastructure Book
Open‑source Chinese book on AI infrastructure, covering LLM inference and training system design with quantitative analysis, plus PDF, code tools, and experiments.
★ 7,748 · PythonOPTILLM — SPEEDING LLM INFERENCE
optillm — a proxy accelerator for LLMs that reduces latency and saves resources during inference.
★ 4,343 · PythonPROJECT MAYA: LOCAL LLM ENGINE
Project Maya lets you run GLM-5.3-Flash (321B MoE) locally on a single GPU or up to 16 GPUs, supporting NVIDIA and AMD on Linux and Windows. The package includes engine,
★ 2,898 · C++FAST LAUNCH QWEN3.8 ON AMD
The project provides scripts for instant deployment of Qwen3.8-Flash-Next on AMD Strix Halo (gfx1151) GPU via HIP. All dependencies are built automatically, and inference
★ 1,349 · ShellRUN 284B LLM ON CPU WITHOUT GPU
llama-modchip runs DeepSeek-V4-Flash 284B on pure 64GB CPU RAM: 44.4 tok/s prefill, 9-14 tok/s generation. 30B MoE models up to 111 tok/s. 256K context, Claude Code resto
★ 1,234 · PythonRUN MODELS LARGER THAN VRAM
inferssd runs MoE models exceeding RAM and VRAM by streaming experts from SSD. Built on llama.cpp with GGUF, CUDA and Apple Silicon support. 288 stars on GitHub.
★ 1,108 · C++RUN LARGE LLMS LOCALLY
PolyStrata runs Qwen3.8-Flash, GLM 5.3 Flash, DeepSeek V4 and more on your PC. One-click install for Windows and Linux with OpenAI/Anthropic API on localhost.
★ 1,037 · C++TUTEL ACCELERATES MOE IN C
Optimized Mixture-of-Experts library for accelerated LLM inference with FP8, NVFP4, and MXFP4 support.
★ 1,028 · CBIGMOEONEDGE - SUPER MOE ON PHONE
The project enables running 284 B Mixture‑of‑Experts models on a device with 12 GB of RAM, using only CPU and llama.cpp without loss of accuracy.
★ 855 · C++JANAS: FAST LLM INFERENCE!!
janas runs huge LLM (up to 48 GB) on a 32 GB laptop, outpacing llama.cpp 8‑10×. On Qwen3-Next-80B it gives 37 tok/s w/o draft and 62 tok/s reading GGUF. Supports CPU+GPU
★ 712 · CSLOTSTREAM LAUNCHES QWEN3.8 ON MAC
Run Qwen3.8‑Flash‑Next (125B MoE, 104 GB at 4‑bit) on Mac by streaming experts from SSD on demand. Implemented with MLX + Swift, it offers an Ollama‑compatible API.
★ 673 · SwiftRUNNING DEEPSEEK-V4.1 552B ON A SINGLE PC
Xolotl is an engine for running large MoE models (DeepSeek-V4.1-Flash 552B, GLM-5.3-Flash, Qwen3.8, LFM3-Omni) on a single PC with RTX 4090. It delivers 33 tok/s on agent
★ 534 · C++FonaTech Chronos
The Chronos project achieves zero‑latency Mixture‑of‑Experts inference via lookahead prediction and asynchronous DMA prefetch from SSD, plus GPU‑optimized kernels. Hybrid
★ 299 · PythonUNIMMAD ANOMALY DETECTION IN MULTIMEDIA MODEL
UniMMAD offers a unified approach to anomaly detection in multimedia data by combining images, audio, and text. Using mixture‑of‑experts, it performs feature decompressio
★ 248 · PythonPULSAR — ULTRAFAST MOE INFERENCE
Pulsar — an SSD-streaming engine for giant MoE models on Rust+CUDA.
★ 238 · RustPURE-RUST ENGINE FOR QWEN GGUF
RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.
★ 231 · RustLLM Alignment Tuning
Automatic calibration of the language model's censorship layer via direct weight modification and the use of LoRA and MoE experts.
★ 221 · Python125B MoE Inference on RTX
Run Qwen3.8-Flash-Next (125B MoE) with 1M context on one or two RTX 3080 10GB cards. Tuned, benchmarked fork of Strata.
★ 71 · C++New projects on this topic in Telegram
Telegram →