AIPython★ 2,172⑂ 499 forksApache-2.0
ashhart/TensorFold
TENSORFOLD: FAST LLM DECODING ON MAC.
TensorFold provides exact and fast LLM decoding on Apple Silicon via MLX, offering an OpenAI-compatible API. The Python project has already earned 526 stars and supports the model inference without the approximations.
// KEY FEATURES
- Exact LLM decoding on Apple Silicon via MLX, without loss of the accuracy.
- OpenAI-compatible HTTP endpoint for invoking the models via the standard API.
- Works on Python with MLX low memory consumption and high speed on M1/M2 cores.
- Supporting loading models from HuggingFace and fast inference without conversion.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram