AIPython★ 7,873
InternLM/lmdeploy
LMDeploy: Easy LLM Deployment
A toolkit for compressing, deploying, and serving LLMs. ⭐ 7873 | Python
// KEY FEATURES
- Model optimization for inference
- ~~(TurboMind, PyTorch, Socket modes)
- Simple deployment of LLM servers
- Support for multiple models
#codellama#cuda-kernels#deepspeed#fastertransformer#internlm#llama#llama2#llama3#llm#llm-inference#turbomind
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram