R_REDDYX.XYZ
AIC++★ 1,194⑂ 105 forksApache-2.0

jmaczan/tiny-vllm

TINY VLLM: BUILD YOUR LLM

A mini version of vLLM in C++ and CUDA for fast inference of large language models. Enables building your own high-performance inference engine.

// KEY FEATURES

  • Full CUDA acceleration support
  • Minimalist code in pure C++
  • Easy integration into existing projects
#ai#attention#batching#course#cpp#cuda#hpc#inference#llm#llm-inference#pagedattention#tiny-vllm
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

OBOT — MCP PLATFORM ON GO★ 1194SAGE: Multi-Agent System for Complex Tasks★ 1193AI LEARNING: ROADMAPS★ 1195KANBAN FOR HUMANS AND AGENTS★ 1193ClaudeBar AI Menu Bar★ 1196FOUNDATION MODELS FRAMEWORK★ 1192

← FULL CATALOG

jmaczan/tiny-vllm — Tiny Vllm: Build Your LLM | REDDYX AI