AIPython★ 14,445
jundot/omlx
LLM Server for Apple Silicon
A powerful inference server with support for continuous batching and SSD caching, running on Apple Silicon and controlled via the macOS menu bar.
// KEY FEATURES
- Continuous batching
- SSD caching for speed
- Integration with macOS menu
#apple-silicon#inference-server#llm#macos#mlx#openai-api
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram