AIPython★ 154⑂ 12 forks
mit-han-lab/EvolvingInfra
Adaptive Agent Inference Framework
EvolvingInfra dynamically optimizes inference for diverse agent workloads, adapting scheduling and resource allocation in real-time.
// KEY FEATURES
- Auto-tunes batching and scheduling per agent task profile
- Supports heterogeneous models and dynamic workload shifts
- Integrates with vLLM and SGLang for low-latency serving
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram