R_REDDYX.XYZ

Sglang: open-source projects on GitHub

24 repositories on Sglang, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Vllm 69LLM Inference 76Pytorch 98Inference 95Cuda 48Deepseek 229Qwen 106Openai 690Rocm 12Ollama 249AI Agents 1319Llama 106LLM Serving 24
kvcache-ai/Mooncake

MOONCAKE - KIMI LLM PLATFORM, 5400+ STARS

Mooncake is the backend platform for Kimi from Moonshot AI. High-performance open-source LLM service in C++.

★ 5,429 · C++
gpustack/gpustack

GPU cluster manager

Management of GPU clusters with automatic configuration and orchestration of inference engines (vLLM, SGLang) for high-performance deployment of AI models.

★ 5,041 · Python
datawhalechina/zero-to-sglang

ZERO-TO-SGLANG: LLM INFERENCE TRAINING

The Datawhale and SGLang course teaches LLM inference from scratch: build a mini‑SGLang, study the source code, and land your first PR. Available in English and Chinese.

★ 2,136 · Python
ovg-project/kvcached

KVCACHED: ACCELERATE GPU CACHE

Virtualized elastic KV‑cache for dynamic GPU sharing, accelerating LLM inference and reducing latency.

★ 2,041 · Python
sgl-project/sglang-omni

SGLANG-OMNI: OMNI-SERVICE FOR AI

SGLang-Omni provides high-performance execution of TTS, ASR, and speech omni-models on Python.

★ 1,791 · Python
eagle-hq/EAGLE-4

EAGLE-4 Speculative Decoding

Official inference code for EAGLE-4, a speculative decoding method that accelerates LLM generation by drafting tokens via feature-level extrapolation.

★ 1,287 · Python
sgl-project/SpecForge

QUICKLY TRAIN SPECULATIVE DECODER

SpecForge simplifies training speculative decoding models and exporting them to SGLang for high-performance serving.

★ 1,285 · Python
patchy631/time-to-first-token

LLM Inference Optimization Roadmap

A 10‑week plan to study the service and accelerate large language models, covering vLLM, SGLang, quantization, and speculative decoding practices.

★ 1,179 · HTML
OpenMOSS/MOVA

MOVA Scalable Video-Audio

The MOVA project generates synchronized video and audio, scaling to large datasets in Python.

★ 1,144 · Python
SemiAnalysisAI/InferenceX

InferenceX – Inference Benchmark

Open-source continuous inference benchmark for Qwen3.5, DeepSeek, GPTOSS and new accelerators.

★ 1,016 · Python
XHToken/Spark-X2.6

SPARK-X2.6: AGENTS ON DEVICE

Open Spark-x2.5 model series focused on agentic capabilities. Runs locally on device without cloud. 814 stars on GitHub.

★ 911
smg-project/smg

SMG - Universal LLM Gateway

Engine-agnostic LLM gateway in Rust with full OpenAI and Anthropic API compatibility, gRPC pipeline, and KV-cache caching.

★ 723 · Rust
MemTensor/MemT-1.0

MEMT-1.0: LONG-TERM MEMORY FOR LLM

Official MemT-1.0 repository. Implements long-term memory for LLM agents. Supports long context and DeepSeek integration.

★ 537 · Python
ome-projects/ome

LLM Serving in Kubernetes

OME is an operator for running LLMs in Kubernetes with smart GPU scheduling and support for SGLang, vLLM, TensorRT-LLM, and Triton.

★ 527 · Go
matrixhub-ai/matrixhub

MATRIXHUB: AI Hub – Open Source

Self-hosted platform for AI models with Hugging Face compatibility and vLLM/SGLang acceleration. Stars: 246 ⭐

★ 367 · Go
lightseekorg/smg

SMG - Universal LLM Gateway

SMG — engine‑agnostic LLM gateway on Rust with full compatibility with OpenAI and Anthropic API.

★ 322 · Rust
sgl-project/rbg

RGB - Scaling LLM Services

A Go package enabling fast deployment of LLM services on Kubernetes with minimal setup.

★ 315 · Go
AMD-AGI/Hyperloom

HYPERLOOM AUTO-OPTIMIZATION LLM ON AMD

Hyperloom — agentic system that automatically optimizes LLM workloads on AMD GPUs by selecting kernels and tuning parameters for peak performance without manual setup.

★ 271 · Python
lightseekorg/TorchSpec

TORCHSPEC SPECULATIVE DECODING

PyTorch library for training speculative decoding models, accelerating text generation.

★ 268 · Python
Zyphra/ZONOS3-Mini

COMPACT TTS WITH VOICE CLONING

Multilingual text-to-speech model with zero-shot voice cloning. Runs locally on GPU, CPU, Apple Silicon, and in browser via WebGPU.

★ 219 · Python
justxor/deepseek-local-guide-ru

DeepSeek Local Setup Guide

A Russian‑language guide that walks you through installing and running DeepSeek models (V4, R1) locally on PC, Mac, or server using tools like Ollama, llama.cpp, LM Studi

★ 192 · Shell
Aevonix/FlashAlign

On-Demand Model Alignment

Aligns LLMs instantly per request for verified users or always-on mode. Integrates with Hermes, OpenClaw, and custom harnesses.

★ 126 · Python
juliankang4/spark-scope

DGX Spark Monitoring Dashboard

Provides a read-only web dashboard and rack panel to monitor DGX Spark, ASUS GX10, and MSI EdgeXpert nodes running vLLM or SGLang.

★ 70 · JavaScript
inclusionAI/ASpec

Speculative Decoding Training Framework

ASpec accelerates LLM inference by training efficient draft models for speculative decoding with high-throughput distributed training.

★ 53 · Python

New projects on this topic in Telegram

Telegram →
Sglang — 24 open-source GitHub projects | REDDYX AI