R_REDDYX.XYZ
AIGo★ 527⑂ 107 forksApache-2.0

ome-projects/ome

LLM Serving in Kubernetes

OME is an operator for running LLMs in Kubernetes with smart GPU scheduling and support for SGLang, vLLM, TensorRT-LLM, and Triton.

// KEY FEATURES

  • Automatic GPU scheduling
  • Support for SGLang, vLLM, TensorRT-LLM
  • Model lifecycle management
#deepseek#k8s#kimi-k2#llama#llm#llm-inference#model-as-a-service#model-serving#multi-node-kubernetes#oracle-cloud#pd-disaggregation#qwen
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

VLA MODELS FOR CARS★ 527MCP FOR DETECTION ENGINEERING★ 527FIRE RED ASR2S SOTA RECOGNITION★ 528ModelScope Developer Handbook★ 526Hypothesis Discovery Workbench★ 528CATALOG OF AI AGENTS AND TOOLS★ 528

← FULL CATALOG

ome-projects/ome — LLM Serving in Kubernetes | REDDYX AI