R_REDDYX.XYZ
AIPython★ 1,997⑂ 357 forksApache-2.0

vllm-project/vllm-metal

VLLM-METAL: ACCELERATING LLM ON APPLE SILICON

vllm-metal is a community‑maintained plugin that brings Metal‑based acceleration to vLLM on Apple Silicon. It uses MLX to run LLMs on macOS with low latency and high throughput, no CUDA required.

// KEY FEATURES

  • Supports hardware acceleration via Metal on Apple Silicon (M1/M2/M3).
  • Integrates with vLLM as a plugin, replacing the CUDA backend with MLX.
  • Provides low‑latency LLM inference on macOS without external drivers.
  • Supports quantization and dynamic batching to increase throughput.
#apple-silicon#llm#macos#metal#mlx#vllm
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

SEO & GEO AI Skills★ 1996OBSIDIAN AI TEXT GENERATOR★ 1996PLUGINS FOR CLAUDE CODE★ 1994EdgeQuake: GraphRAG in Rust★ 1992MCP - Agent with MS Docs★ 1992LAWS OF ALL COUNTRIES IN GIT★ 2002

← FULL CATALOG

vllm-project/vllm-metal — Vllm-Metal: Accelerating LLM ON…