R_REDDYX.XYZ
AIPython★ 249⑂ 25 forksMIT

sybil-solutions/inference-optimisation-skill

Auto-Tune LLM Serving Speed

Automatically optimizes local LLM serving speed via hill-climbing across decode, speculative decoding, prefill, KV cache, and startup metrics on any hardware.

// KEY FEATURES

  • Hill-climbs decode, speculative decoding, prefill, KV cache, and TTFT parameters
  • Works across any hardware and quantization configuration automatically
  • Agent skill integrates with optimization loops for continuous speed gains
#agent-skill#benchmarking#claude-code#inference#llm#speculative-decoding
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

SKILLS FOR CLOUDFLARE AND AI★ 249LARAVEL AI TRANSLATOR: INSTANT TRANSLATION★ 249SCREENMIND - YOUR SCREEN MEMORY★ 249DS AUTO-REVIEW:AI SYSTEM★ 249SECRET AI FOR YOUR DESK★ 249CARBON TRACK CLAUDE CODE!★ 249

← FULL CATALOG

sybil-solutions/inference-optimisation-skill — Auto-Tune LLM…