AIPython★ 249⑂ 25 forksMIT
sybil-solutions/inference-optimisation-skill
Auto-Tune LLM Serving Speed
Automatically optimizes local LLM serving speed via hill-climbing across decode, speculative decoding, prefill, KV cache, and startup metrics on any hardware.
// KEY FEATURES
- Hill-climbs decode, speculative decoding, prefill, KV cache, and TTFT parameters
- Works across any hardware and quantization configuration automatically
- Agent skill integrates with optimization loops for continuous speed gains
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram