R_REDDYX.XYZ
AIPython★ 23,423⑂ 2,213 forksApache-2.0

mlc-ai/mlc-llm

MLC‑LLM UNIVERSAL ENGINE LLM

MLC‑LLM is an engine for universal LLM deployment using ML compilation with TVM. It supports many model formats and optimizes them for any device via a single API.

// KEY FEATURES

  • Compiles LLM into efficient TVM libraries for CPU, GPU and ARM in minutes.
  • Unified Python API and CLI for loading and running models.
  • Supports int8/float16 quantization and dynamic batching for low latency.
  • Exports optimized models to .so, .wasm and .exe formats for edge devices.
#language-model#llm#machine-learning-compilation#tvm
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

AKSHARE — FINANCIAL DATA ON PYTHON★ 23341Agent Memory★ 23568Resources for AI Research★ 23220Autonomous Agent Prototype★ 23138AI CONTROLS DESKTOP RELIABLY★ 23711Headless Browser for AI Agents★ 23747

← FULL CATALOG

mlc-ai/mlc-llm — Mlc‑llm Universal Engine LLM | REDDYX AI