AIC++★ 3,339⑂ 710 forksApache-2.0
Neroued/ninfer
Single-GPU Inference Engine
Optimized C++ inference engine for running selected model checkpoints on single GPUs with high throughput.
// KEY FEATURES
- Runs optimized inference on single GPUs for specific model architectures
- Supports quantized model checkpoints with custom kernels
- Achieves high throughput via GPU-specific tensor operations
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram