Sglang: open-source projects on GitHub
24 repositories on Sglang, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
MOONCAKE - KIMI LLM PLATFORM, 5400+ STARS
Mooncake is the backend platform for Kimi from Moonshot AI. High-performance open-source LLM service in C++.
★ 5,429 · C++GPU cluster manager
Management of GPU clusters with automatic configuration and orchestration of inference engines (vLLM, SGLang) for high-performance deployment of AI models.
★ 5,041 · PythonZERO-TO-SGLANG: LLM INFERENCE TRAINING
The Datawhale and SGLang course teaches LLM inference from scratch: build a mini‑SGLang, study the source code, and land your first PR. Available in English and Chinese.
★ 2,136 · PythonKVCACHED: ACCELERATE GPU CACHE
Virtualized elastic KV‑cache for dynamic GPU sharing, accelerating LLM inference and reducing latency.
★ 2,041 · PythonSGLANG-OMNI: OMNI-SERVICE FOR AI
SGLang-Omni provides high-performance execution of TTS, ASR, and speech omni-models on Python.
★ 1,791 · PythonEAGLE-4 Speculative Decoding
Official inference code for EAGLE-4, a speculative decoding method that accelerates LLM generation by drafting tokens via feature-level extrapolation.
★ 1,287 · PythonQUICKLY TRAIN SPECULATIVE DECODER
SpecForge simplifies training speculative decoding models and exporting them to SGLang for high-performance serving.
★ 1,285 · PythonLLM Inference Optimization Roadmap
A 10‑week plan to study the service and accelerate large language models, covering vLLM, SGLang, quantization, and speculative decoding practices.
★ 1,179 · HTMLMOVA Scalable Video-Audio
The MOVA project generates synchronized video and audio, scaling to large datasets in Python.
★ 1,144 · PythonInferenceX – Inference Benchmark
Open-source continuous inference benchmark for Qwen3.5, DeepSeek, GPTOSS and new accelerators.
★ 1,016 · PythonSPARK-X2.6: AGENTS ON DEVICE
Open Spark-x2.5 model series focused on agentic capabilities. Runs locally on device without cloud. 814 stars on GitHub.
★ 911SMG - Universal LLM Gateway
Engine-agnostic LLM gateway in Rust with full OpenAI and Anthropic API compatibility, gRPC pipeline, and KV-cache caching.
★ 723 · RustMEMT-1.0: LONG-TERM MEMORY FOR LLM
Official MemT-1.0 repository. Implements long-term memory for LLM agents. Supports long context and DeepSeek integration.
★ 537 · PythonLLM Serving in Kubernetes
OME is an operator for running LLMs in Kubernetes with smart GPU scheduling and support for SGLang, vLLM, TensorRT-LLM, and Triton.
★ 527 · GoMATRIXHUB: AI Hub – Open Source
Self-hosted platform for AI models with Hugging Face compatibility and vLLM/SGLang acceleration. Stars: 246 ⭐
★ 367 · GoSMG - Universal LLM Gateway
SMG — engine‑agnostic LLM gateway on Rust with full compatibility with OpenAI and Anthropic API.
★ 322 · RustRGB - Scaling LLM Services
A Go package enabling fast deployment of LLM services on Kubernetes with minimal setup.
★ 315 · GoHYPERLOOM AUTO-OPTIMIZATION LLM ON AMD
Hyperloom — agentic system that automatically optimizes LLM workloads on AMD GPUs by selecting kernels and tuning parameters for peak performance without manual setup.
★ 271 · PythonTORCHSPEC SPECULATIVE DECODING
PyTorch library for training speculative decoding models, accelerating text generation.
★ 268 · PythonCOMPACT TTS WITH VOICE CLONING
Multilingual text-to-speech model with zero-shot voice cloning. Runs locally on GPU, CPU, Apple Silicon, and in browser via WebGPU.
★ 219 · PythonDeepSeek Local Setup Guide
A Russian‑language guide that walks you through installing and running DeepSeek models (V4, R1) locally on PC, Mac, or server using tools like Ollama, llama.cpp, LM Studi
★ 192 · ShellOn-Demand Model Alignment
Aligns LLMs instantly per request for verified users or always-on mode. Integrates with Hermes, OpenClaw, and custom harnesses.
★ 126 · PythonDGX Spark Monitoring Dashboard
Provides a read-only web dashboard and rack panel to monitor DGX Spark, ASUS GX10, and MSI EdgeXpert nodes running vLLM or SGLang.
★ 70 · JavaScriptSpeculative Decoding Training Framework
ASpec accelerates LLM inference by training efficient draft models for speculative decoding with high-throughput distributed training.
★ 53 · PythonNew projects on this topic in Telegram
Telegram →