R_REDDYX.XYZ

Mixture Of Experts: open-source projects on GitHub

20 repositories on Mixture Of Experts, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Moe 23Local LLM 107Gguf 47Cuda 48Qwen 106LLM Inference 76Quantization 34Inference 95Inference Engine 18Openai API 86Speculative Decoding 23Llama Cpp 56Deepseek 229
deepspeedai/DeepSpeed

DeepSpeed: optimization library for training

DeepSpeed is an optimization deep learning library ensuring efficient distributed training and inference. Supports ZeRO, MoE and parallelism for training models with hund

★ 43,332 · Python
FareedKhan-dev/kimi-k3-in-c

KIMI K3: 2.78 Trillion Parameters

Running the Kimi K3 model with 2.78 trillion parameters on a single CPU, using only 8.24 GB of RAM without BLAS, frameworks, or GPU.

★ 12,109 · C
bojieli/ai-infra-book

AI Infrastructure Book

Open‑source Chinese book on AI infrastructure, covering LLM inference and training system design with quantitative analysis, plus PDF, code tools, and experiments.

★ 7,748 · Python
algorithmicsuperintelligence/optillm

OPTILLM — SPEEDING LLM INFERENCE

optillm — a proxy accelerator for LLMs that reduces latency and saves resources during inference.

★ 4,343 · Python
mw00/project-maya

PROJECT MAYA: LOCAL LLM ENGINE

Project Maya lets you run GLM-5.3-Flash (321B MoE) locally on a single GPU or up to 16 GPUs, supporting NVIDIA and AMD on Linux and Windows. The package includes engine,

★ 2,898 · C++
peonist-ai/halogen-flash-server

FAST LAUNCH QWEN3.8 ON AMD

The project provides scripts for instant deployment of Qwen3.8-Flash-Next on AMD Strix Halo (gfx1151) GPU via HIP. All dependencies are built automatically, and inference

★ 1,349 · Shell
synchopate/llama-modchip

RUN 284B LLM ON CPU WITHOUT GPU

llama-modchip runs DeepSeek-V4-Flash 284B on pure 64GB CPU RAM: 44.4 tok/s prefill, 9-14 tok/s generation. 30B MoE models up to 111 tok/s. 256K context, Claude Code resto

★ 1,234 · Python
wici-ai/inferssd

RUN MODELS LARGER THAN VRAM

inferssd runs MoE models exceeding RAM and VRAM by streaming experts from SSD. Built on llama.cpp with GGUF, CUDA and Apple Silicon support. 288 stars on GitHub.

★ 1,108 · C++
simplexai-labs/PolyStrata

RUN LARGE LLMS LOCALLY

PolyStrata runs Qwen3.8-Flash, GLM 5.3 Flash, DeepSeek V4 and more on your PC. One-click install for Windows and Linux with OpenAI/Anthropic API on localhost.

★ 1,037 · C++
microsoft/Tutel

TUTEL ACCELERATES MOE IN C

Optimized Mixture-of-Experts library for accelerated LLM inference with FP8, NVFP4, and MXFP4 support.

★ 1,028 · C
Helldez/BigMoeOnEdge

BIGMOEONEDGE - SUPER MOE ON PHONE

The project enables running 284 B Mixture‑of‑Experts models on a device with 12 GB of RAM, using only CPU and llama.cpp without loss of accuracy.

★ 855 · C++
prabanta-dev/janas

JANAS: FAST LLM INFERENCE!!

janas runs huge LLM (up to 48 GB) on a 32 GB laptop, outpacing llama.cpp 8‑10×. On Qwen3-Next-80B it gives 37 tok/s w/o draft and 62 tok/s reading GGUF. Supports CPU+GPU

★ 712 · C
carloslfu/slotstream

SLOTSTREAM LAUNCHES QWEN3.8 ON MAC

Run Qwen3.8‑Flash‑Next (125B MoE, 104 GB at 4‑bit) on Mac by streaming experts from SSD on demand. Implemented with MLX + Swift, it offers an Ollama‑compatible API.

★ 673 · Swift
Indras-Mirror/Xolotl

RUNNING DEEPSEEK-V4.1 552B ON A SINGLE PC

Xolotl is an engine for running large MoE models (DeepSeek-V4.1-Flash 552B, GLM-5.3-Flash, Qwen3.8, LFM3-Omni) on a single PC with RTX 4090. It delivers 33 tok/s on agent

★ 534 · C++
FonaTech/Project_Chronos

FonaTech Chronos

The Chronos project achieves zero‑latency Mixture‑of‑Experts inference via lookahead prediction and asynchronous DMA prefetch from SSD, plus GPU‑optimized kernels. Hybrid

★ 299 · Python
yuanzhao-CVLAB/UniMMAD

UNIMMAD ANOMALY DETECTION IN MULTIMEDIA MODEL

UniMMAD offers a unified approach to anomaly detection in multimedia data by combining images, audio, and text. Using mixture‑of‑experts, it performs feature decompressio

★ 248 · Python
giannisanni/pulsar

PULSAR — ULTRAFAST MOE INFERENCE

Pulsar — an SSD-streaming engine for giant MoE models on Rust+CUDA.

★ 238 · Rust
RustingStudio/RustingStrata

PURE-RUST ENGINE FOR QWEN GGUF

RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.

★ 231 · Rust
wuwangzhang1216/abliterix

LLM Alignment Tuning

Automatic calibration of the language model's censorship layer via direct weight modification and the use of LoRA and MoE experts.

★ 221 · Python
DevDock-AI/Speedy

125B MoE Inference on RTX

Run Qwen3.8-Flash-Next (125B MoE) with 1M context on one or two RTX 3080 10GB cards. Tuned, benchmarked fork of Strata.

★ 71 · C++

New projects on this topic in Telegram

Telegram →
Mixture Of Experts — 20 open-source GitHub projects | REDDYX AI