AIPython★ 26,027⑂ 3,195 forksApache-2.0
deepseek-ai/DeepInfer
High-Performance DeepSeek Inference
DeepInfer accelerates DeepSeek models with optimized kernels, distributed parallelism, and low-latency serving for large-scale deployment.
// KEY FEATURES
- Optimizes DeepSeek models with custom CUDA kernels for faster inference
- Enables tensor and pipeline parallelism across multiple GPUs and nodes
- Provides dynamic batching and KV caching for low-latency serving
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram