AIPython★ 108⑂ 17 forksApache-2.0
openJiuwen-ai/agent-infer
Agentic Inference Framework
openJiuwen-ai/agent-infer is a vLLM-based framework for efficient execution of agentic workloads. It simplifies deployment and scaling of LLM inference in multi-agent scenarios.
// KEY FEATURES
- Fast LLM inference through an optimized vLLM backend with batch processing support.
- Support for dynamic agent orchestration with shared KV cache usage.
- Streaming token generation with low latency and flexible parameter configuration.
- Easy integration with popular agent frameworks (LangChain, AutoGPT, BabyAGI).
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram