AIPython★ 953⑂ 27 forksApache-2.0
mode-io/vllm-jev
vLLM Serving for Jev Decision Models
Provides native vLLM serving support for Jev decision models, enabling high-throughput inference with vLLM's optimized engine.
// KEY FEATURES
- Integrates Jev decision models with vLLM's PagedAttention for efficient serving
- Supports batched inference and continuous batching for high throughput
- Exposes OpenAI-compatible API endpoints for Jev models
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram