R_REDDYX.XYZ
AIPython★ 4,343⑂ 401 forksApache-2.0

algorithmicsuperintelligence/optillm

OPTILLM — SPEEDING LLM INFERENCE

optillm — a proxy accelerator for LLMs that reduces latency and saves resources during inference.

// KEY FEATURES

  • Support for HuggingFace, vLLM, TensorRT‑LLM
  • Dynamic batching and KV caching
  • HTTP API compatible with OpenAI client
#agent#agentic-ai#agentic-framework#agentic-workflow#agents#api-gateway#chain-of-thought#genai#large-language-models#llm#llm-inference#llmapi
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

MemoryLanceDB Pro★ 4347Mini SWE‑agent★ 4350Screenshots for AI Agents★ 4335Claude Code: All Levels★ 4357LLAMA-SWAP - RELIABLE MODEL REPLACEMENT★ 4328DEEPFLOW: eBPF TRACING AND PROFILING★ 4325

← FULL CATALOG

algorithmicsuperintelligence/optillm — Optillm — Speeding…