R_REDDYX.XYZ
AIC★ 712⑂ 72 forksGPL-3.0

prabanta-dev/janas

JANAS: FAST LLM INFERENCE!!

janas runs huge LLM (up to 48 GB) on a 32 GB laptop, outpacing llama.cpp 8‑10×. On Qwen3-Next-80B it gives 37 tok/s w/o draft and 62 tok/s reading GGUF. Supports CPU+GPU inference in pure C with AVX2.!!!!

// KEY FEATURES

  • Runs Qwen3-Next-80B (48 GB) on a 32 GB laptop, givingnow 37 tok/s w/out draft.
  • Beats llama.cpp by8‑10× in token generation speed across all 13 Q4_K_M benchmark models
  • Reads GGUF models at 62 tok/s, nearly 4× faster than llama.cpp in CPU inference.
  • Works on CPU and GPU inference in pure C with AVX2 acceleration under Linux.
#avx2#c#cpu-inference#gguf#inference-engine#linux#llm#local-llm#mixture-of-experts#moe#nvme#quantization
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

SOULFORGE — AI CODE AGENT★ 712HTML Video Layer for Agents★ 712🤖 SPEC_DRIVEN_DEVELOP - FROM CHAOS TO ORDER★ 713BRAIN.MD – AI MEMORY FOR CODE★ 711SEARCHGPT: AI SEARCH WITH SOURCES★ 711FREE EMBROIDERY DIGITIZER★ 711

← FULL CATALOG

prabanta-dev/janas — Janas: Fast LLM inference!! | REDDYX AI