R_REDDYX.XYZ
AIPython★ 2,713⑂ 340 forksApache-2.0

syv-ai/HyperQwen

HyperQwen: Qwen on Consumer GPUs

The project optimizes running large Qwen models on available GPUs using vLLM patches and requantization. It achieves 127 tokens/s on 24 GB.

// KEY FEATURES

  • Qwen3.8-27B on a single 24 GB GPU
  • 127 tokens/s (single user), 1035 tokens/s with 64 parallel
  • Context support 150–262k tokens
  • vLLM patches and requantization pipeline
#consumer-gpu#cuda#inference-optimization#kv-cache#llm-inference#local-llm#quantization#qwen#qwen3#rtx-3090#speculative-decoding#vllm
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

Unified rule manager★ 2710PAPERSGPT — AI for Zotero★ 2709Universal Content Translator★ 2718SOLANA MCP VYBE: OPEN REGISTRY★ 2720MTPLX SUPERFAST LAUNCH★ 2706AGENTS - EXPER SKILLS★ 2705

← FULL CATALOG

syv-ai/HyperQwen — HyperQwen: Qwen on Consumer GPUs