AIPython★ 23,423⑂ 2,213 forksApache-2.0
mlc-ai/mlc-llm
MLC‑LLM UNIVERSAL ENGINE LLM
MLC‑LLM is an engine for universal LLM deployment using ML compilation with TVM. It supports many model formats and optimizes them for any device via a single API.
// KEY FEATURES
- Compiles LLM into efficient TVM libraries for CPU, GPU and ARM in minutes.
- Unified Python API and CLI for loading and running models.
- Supports int8/float16 quantization and dynamic batching for low latency.
- Exports optimized models to .so, .wasm and .exe formats for edge devices.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram