AIRust★ 354⑂ 53 forksMIT
guoqingbao/xinfer
XINFER - FAST LLM INFERENCE
Xinfer — ultrafast LLM inference in pure Rust, without PyTorch and Python. Enables running models with minimal latency.
// KEY FEATURES
- Written in pure Rust
- No dependencies on PyTorch/Python
- Minimal latency and high throughput
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram