AIRust★ 238⑂ 29 forks
giannisanni/pulsar
PULSAR — ULTRAFAST MOE INFERENCE
Pulsar — an SSD-streaming engine for giant MoE models on Rust+CUDA.
// KEY FEATURES
- Measures PCIe bandwidth and places attention and experts.
- Automatic multi-GPU without settings.
- Runs on Rust + CUDA, delivering 2-7 tok/s on a 16 GB GPU.
#cuda#gguf#glm#inference-engine#llm#local-llm#mixture-of-experts#moe#multi-gpu#quantization#rust#ssd-streaming
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram