AIRust★ 240⑂ 40 forksApache-2.0
pie-project/pie
PIE — MODEL SERVING ON RUST
Programmable engine for high-performance LLM and diffusion model serving. Written in Rust, supports batching, KV caching, and dynamic adapter loading.
// KEY FEATURES
- Serves LLM and diffusion models via unified API
- Request batching and KV-cache for faster inference
- Hot-loading LoRA adapters without restart
- Written in Rust, delivers low latency and high throughput
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram