AIGo★ 527⑂ 107 forksApache-2.0
ome-projects/ome
LLM Serving in Kubernetes
OME is an operator for running LLMs in Kubernetes with smart GPU scheduling and support for SGLang, vLLM, TensorRT-LLM, and Triton.
// KEY FEATURES
- Automatic GPU scheduling
- Support for SGLang, vLLM, TensorRT-LLM
- Model lifecycle management
#deepseek#k8s#kimi-k2#llama#llm#llm-inference#model-as-a-service#model-serving#multi-node-kubernetes#oracle-cloud#pd-disaggregation#qwen
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram