AIC++★ 1,194⑂ 105 forksApache-2.0
jmaczan/tiny-vllm
TINY VLLM: BUILD YOUR LLM
A mini version of vLLM in C++ and CUDA for fast inference of large language models. Enables building your own high-performance inference engine.
// KEY FEATURES
- Full CUDA acceleration support
- Minimalist code in pure C++
- Easy integration into existing projects
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram