R_REDDYX.XYZ
AIRust★ 354⑂ 53 forksMIT

guoqingbao/xinfer

XINFER - FAST LLM INFERENCE

Xinfer — ultrafast LLM inference in pure Rust, without PyTorch and Python. Enables running models with minimal latency.

// KEY FEATURES

  • Written in pure Rust
  • No dependencies on PyTorch/Python
  • Minimal latency and high throughput
#agent#llm#qwen#rust#vllm
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

LANGCHAIN + AWS = AI AGENTS★ 354VIRTUAL DESIGN KIT FOR AI★ 354Lottie Animation Generator★ 353KLAATCODE - AI AGENT FOR CODE★ 353DANGER ON SITE YOLO HDBSCAN★ 353MINDEPOCKET: AI BOOKMARKS★ 353

← FULL CATALOG

guoqingbao/xinfer — Xinfer - Fast LLM Inference | REDDYX AI