AIShell★ 3,247
llm-d/llm-d
LLM inference acceleration on Kubernetes
Achieve state of the art inference performance with modern accelerators on Kubernetes
// KEY FEATURES
- Optimized for GPU and TPU in K8s
- Automatic scaling of inference workloads
- Minimalist shell interface without dependencies
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram