Moe: open-source projects on GitHub
23 repositories on Moe, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
High-performance LLM engine
A high-performance and memory-efficient engine for inference and serving large language models.
★ 79,783 · PythonLlamaFactory: Unified LLM Tuning
LlamaFactory offers a unified library for efficient fine‑tuning of 100+ LLMs and VLMs, with support for LoRA, QLoRA and distributed multi‑GPU training.
★ 71,186 · PythonSGLang: LLM Framework
SGLang is a high-performance framework for serving large language and multimodal models.
★ 27,854 · PythonFreeToken: Local Model Run
FreeToken lets you run large AI models on your PC, delivering data‑center‑class speed and efficiency without needing cloud resources.
★ 17,197 · Pythonms-swift: LLM training
A framework for fine-tuning 600+ language and 300+ multimodal models. Supports PEFT, full fine-tuning, CPT/SFT/DPO/GRPO.
★ 14,178 · PythonKIMI K3: 2.78 Trillion Parameters
Running the Kimi K3 model with 2.78 trillion parameters on a single CPU, using only 8.24 GB of RAM without BLAS, frameworks, or GPU.
★ 12,109 · CPROJECT MAYA: LOCAL LLM ENGINE
Project Maya lets you run GLM-5.3-Flash (321B MoE) locally on a single GPU or up to 16 GPUs, supporting NVIDIA and AMD on Linux and Windows. The package includes engine,
★ 2,898 · C++UCCL: GPU Communication at Max
A fast library for efficient data exchange between GPUs: collective operations, P2P, and inference acceleration.
★ 1,586 · C++TUTEL ACCELERATES MOE IN C
Optimized Mixture-of-Experts library for accelerated LLM inference with FP8, NVFP4, and MXFP4 support.
★ 1,028 · CHALO: FRAMEWORK FOR LLM TRAINING
Halo is an open-source framework from White Circle for training large language and multimodal models. It supports agent-based training, async-RL, distributed training, an
★ 862 · PythonIMPLEMENTING LLM ARCHITECTURES FROM SCRATCH
The project contains clean implementations of modern LLM architectures in PyTorch. Includes DeepSeek-R1, DeepSeek-V3, Gemma 3, Kimi K2, and Llama 4. Perfect for studying
★ 856 · PythonBIGMOEONEDGE - SUPER MOE ON PHONE
The project enables running 284 B Mixture‑of‑Experts models on a device with 12 GB of RAM, using only CPU and llama.cpp without loss of accuracy.
★ 855 · C++SWIFTLM: SWIFT LLM ON APPLE
Native LLM inference server in Swift with MLX, OpenAI‑compatible API, SSD streaming for MoE >100B and TurboQuant KV‑cache compression.
★ 799 · SwiftPEGAINFER: Rust + CUDA Infer
Pure Rust LLM engine with CUDA acceleration, OpenAI API compatible, runs Qwen3 and Kimi‑K2 without PyTorch.
★ 742 · RustJANAS: FAST LLM INFERENCE!!
janas runs huge LLM (up to 48 GB) on a 32 GB laptop, outpacing llama.cpp 8‑10×. On Qwen3-Next-80B it gives 37 tok/s w/o draft and 62 tok/s reading GGUF. Supports CPU+GPU
★ 712 · CRUNNING DEEPSEEK-V4.1 552B ON A SINGLE PC
Xolotl is an engine for running large MoE models (DeepSeek-V4.1-Flash 552B, GLM-5.3-Flash, Qwen3.8, LFM3-Omni) on a single PC with RTX 4090. It delivers 33 tok/s on agent
★ 534 · C++DS4 on Spark: 3X Speedup
One‑command installation of the ds4 fork on NVIDIA DGX Spark with ~3× prefill speedup and ~1.5× decode speedup.
★ 415 · ShellZERAIX: LOCAL AI DESKTOP
Local open-source environment for running AI models on device. Speeds up inference without cloud.
★ 405 · TypeScriptNot a Llama, no stance
Auto-tuned launcher for GGUF models on llama.cpp — server with multi‑GPU support, hardware‑specific tuning, and crash recovery. Stars: 226.
★ 284 · GoPULSAR — ULTRAFAST MOE INFERENCE
Pulsar — an SSD-streaming engine for giant MoE models on Rust+CUDA.
★ 238 · RustPURE-RUST ENGINE FOR QWEN GGUF
RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.
★ 231 · RustLLM Alignment Tuning
Automatic calibration of the language model's censorship layer via direct weight modification and the use of LoRA and MoE experts.
★ 221 · PythonMEMRA - AI Accelerator for RTX
Memra is a from‑scratch LLM inference implementation optimized for RTX 5090 (sm_120a) and H100 (sm_90a) architectures in Rust.
★ 6 · RustNew projects on this topic in Telegram
Telegram →