R_REDDYX.XYZ
AIPython★ 2,172⑂ 499 forksApache-2.0

ashhart/TensorFold

TENSORFOLD: FAST LLM DECODING ON MAC.

TensorFold provides exact and fast LLM decoding on Apple Silicon via MLX, offering an OpenAI-compatible API. The Python project has already earned 526 stars and supports the model inference without the approximations.

// KEY FEATURES

  • Exact LLM decoding on Apple Silicon via MLX, without loss of the accuracy.
  • OpenAI-compatible HTTP endpoint for invoking the models via the standard API.
  • Works on Python with MLX low memory consumption and high speed on M1/M2 cores.
  • Supporting loading models from HuggingFace and fast inference without conversion.
#ai#ai-tools#llm#llm-inference#llm-tools
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

Agent-Native RE Audit Workbench★ 2172WAKU-AGENT — PERSONAL AI AGENT★ 2173DAILY RADAR AI-TERMS★ 2173MiroFish: Offline Multi-Agent Engine★ 2170Secure Key Vault★ 2170ZEROSHOT - AUTONOMOUS DEVELOPER TEAM★ 2174

← FULL CATALOG

ashhart/TensorFold — Tensorfold: Fast LLM Decoding ON Mac.