R_REDDYX.XYZ

Moe: open-source projects on GitHub

23 repositories on Moe, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Cuda 48Inference 95Mixture Of Experts 20Qwen 106Gguf 47Quantization 34Deepseek 229LLM Inference 76Local LLM 107Qwen3 30Glm 24Openai API 86Llama 106
vllm-project/vllm

High-performance LLM engine

A high-performance and memory-efficient engine for inference and serving large language models.

★ 79,783 · Python
hiyouga/LlamaFactory

LlamaFactory: Unified LLM Tuning

LlamaFactory offers a unified library for efficient fine‑tuning of 100+ LLMs and VLMs, with support for LoRA, QLoRA and distributed multi‑GPU training.

★ 71,186 · Python
sgl-project/sglang

SGLang: LLM Framework

SGLang is a high-performance framework for serving large language and multimodal models.

★ 27,854 · Python
FlashML-org/FreeToken

FreeToken: Local Model Run

FreeToken lets you run large AI models on your PC, delivering data‑center‑class speed and efficiency without needing cloud resources.

★ 17,197 · Python
modelscope/ms-swift

ms-swift: LLM training

A framework for fine-tuning 600+ language and 300+ multimodal models. Supports PEFT, full fine-tuning, CPT/SFT/DPO/GRPO.

★ 14,178 · Python
FareedKhan-dev/kimi-k3-in-c

KIMI K3: 2.78 Trillion Parameters

Running the Kimi K3 model with 2.78 trillion parameters on a single CPU, using only 8.24 GB of RAM without BLAS, frameworks, or GPU.

★ 12,109 · C
mw00/project-maya

PROJECT MAYA: LOCAL LLM ENGINE

Project Maya lets you run GLM-5.3-Flash (321B MoE) locally on a single GPU or up to 16 GPUs, supporting NVIDIA and AMD on Linux and Windows. The package includes engine,

★ 2,898 · C++
uccl-project/uccl

UCCL: GPU Communication at Max

A fast library for efficient data exchange between GPUs: collective operations, P2P, and inference acceleration.

★ 1,586 · C++
microsoft/Tutel

TUTEL ACCELERATES MOE IN C

Optimized Mixture-of-Experts library for accelerated LLM inference with FP8, NVFP4, and MXFP4 support.

★ 1,028 · C
whitecircle/halo

HALO: FRAMEWORK FOR LLM TRAINING

Halo is an open-source framework from White Circle for training large language and multimodal models. It supports agent-based training, async-RL, distributed training, an

★ 862 · Python
YeonwooSung/llm-architecture-from-scratch

IMPLEMENTING LLM ARCHITECTURES FROM SCRATCH

The project contains clean implementations of modern LLM architectures in PyTorch. Includes DeepSeek-R1, DeepSeek-V3, Gemma 3, Kimi K2, and Llama 4. Perfect for studying

★ 856 · Python
Helldez/BigMoeOnEdge

BIGMOEONEDGE - SUPER MOE ON PHONE

The project enables running 284 B Mixture‑of‑Experts models on a device with 12 GB of RAM, using only CPU and llama.cpp without loss of accuracy.

★ 855 · C++
SharpAI/SwiftLM

SWIFTLM: SWIFT LLM ON APPLE

Native LLM inference server in Swift with MLX, OpenAI‑compatible API, SSD streaming for MoE >100B and TurboQuant KV‑cache compression.

★ 799 · Swift
pegainfer-project/pegainfer

PEGAINFER: Rust + CUDA Infer

Pure Rust LLM engine with CUDA acceleration, OpenAI API compatible, runs Qwen3 and Kimi‑K2 without PyTorch.

★ 742 · Rust
prabanta-dev/janas

JANAS: FAST LLM INFERENCE!!

janas runs huge LLM (up to 48 GB) on a 32 GB laptop, outpacing llama.cpp 8‑10×. On Qwen3-Next-80B it gives 37 tok/s w/o draft and 62 tok/s reading GGUF. Supports CPU+GPU

★ 712 · C
Indras-Mirror/Xolotl

RUNNING DEEPSEEK-V4.1 552B ON A SINGLE PC

Xolotl is an engine for running large MoE models (DeepSeek-V4.1-Flash 552B, GLM-5.3-Flash, Qwen3.8, LFM3-Omni) on a single PC with RTX 4090. It delivers 33 tok/s on agent

★ 534 · C++
Entrpi/ds4-on-spark

DS4 on Spark: 3X Speedup

One‑command installation of the ds4 fork on NVIDIA DGX Spark with ~3× prefill speedup and ~1.5× decode speedup.

★ 415 · Shell
zeraix/zeraix

ZERAIX: LOCAL AI DESKTOP

Local open-source environment for running AI models on device. Speeds up inference without cloud.

★ 405 · TypeScript
raketenkater/ggrun

Not a Llama, no stance

Auto-tuned launcher for GGUF models on llama.cpp — server with multi‑GPU support, hardware‑specific tuning, and crash recovery. Stars: 226.

★ 284 · Go
giannisanni/pulsar

PULSAR — ULTRAFAST MOE INFERENCE

Pulsar — an SSD-streaming engine for giant MoE models on Rust+CUDA.

★ 238 · Rust
RustingStudio/RustingStrata

PURE-RUST ENGINE FOR QWEN GGUF

RustingStrata is a pure-Rust inference engine for Qwen GGUF (Qwen3–3.8, MoE up to 770B+). Supports CUDA/Vulkan, hybrid CPU+GPU experts, MTP speculative decoding. Up to 1.

★ 231 · Rust
wuwangzhang1216/abliterix

LLM Alignment Tuning

Automatic calibration of the language model's censorship layer via direct weight modification and the use of LoRA and MoE experts.

★ 221 · Python
avifenesh/memra

MEMRA - AI Accelerator for RTX

Memra is a from‑scratch LLM inference implementation optimized for RTX 5090 (sm_120a) and H100 (sm_90a) architectures in Rust.

★ 6 · Rust

New projects on this topic in Telegram

Telegram →
Moe — 23 open-source GitHub projects | REDDYX AI