AIC++★ 380⑂ 44 forksMIT
JakeATX/llamAmpere
LLAMAMPERE: LLM ACCELERATION ON RTX 3090
llama.cpp fork optimized for Ampere GPUs. Delivers 110+ tok/s on Qwen 3.8 at 100K context. Runs on 12 GB VRAM with 200K+ context.
// KEY FEATURES
- 110+ tok/s on Qwen 3.8 at 100K context
- Full 262K context support on RTX 3090/3090 Ti
- KV-cache quantization for 12 GB VRAM operation
- llama.cpp fork with CUDA optimizations for Ampere
#ampere#consumer-gpu#cuda#ggml#gguf#kv-cache-quantization#llama-cpp#llm#llm-inference#local-llm#long-context#mtp
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram