R_REDDYX.XYZ
LLMPython★ 814⑂ 110 forks

divagr18/qwenputer

RISC-V CPU COMPILED INTO QWEN3.5 WEIGHTS

Qwen3.5-4B with a 32-bit RISC-V CPU compiled directly into model weights. CPU uses Gated DeltaNet state as RAM and executes code inside model forward pass.

// KEY FEATURES

  • RV32IM CPU with bignum coprocessor in 11 layers before Qwen
  • Gated DeltaNet works as addressable memory (RAM)
  • Runs C and MicroPython inside forward pass without tools
  • Passes official riscv-tests, 3200 trace tokens/sec on L4

// DEEP DIVE: HOW IT WORKS

Qwenputer embeds a 32-bit RISC-V CPU (RV32IM with Zba and bignum coprocessor) directly into Qwen3.5-4B weights. The CPU occupies 11 transformer layers before Qwen's own; all weights are computed by a compiler, not learned. Memory is the Gated DeltaNet state: in DeltaNet heads gate is fixed at α=1, write strength β=1, keys are one-hot vectors, so each update overwrites exactly one row — a RAM cell.

Unlike tool-use or sandbox approaches, execution happens inside the model forward pass. The RvChat driver loads firmware into DeltaNet state after the prompt, and the Triton executor runs the CPU at ~3200 trace tokens/sec on L4. The model emits `calc` tool-call tokens in Qwen3.5 format, but the call is handled by computer layers, then the model emits the result digit by digit. On arithmetic benchmarks (DeepMind mul_big, div_big, etc.) Qwenputer reaches 97.5–100% vs 55–90% for base; replay of 230k positions showed zero mismatches with reference machine.

Requires GPU with Triton support (tested on L4). Install via uv and huggingface-hub; model loads as standard Qwen3_5ForCausalLM. CPU code and firmware in experimental/riscv/. Text behavior matches base (KL 6e-4) on prompts without tool-call markup; transcripts with completed calls are unsuitable for perplexity scoring.

// USE CASES

Best for: Researchers of interpretability and LLM architectures with built-in code execution

  • Study CPU compilation into transformer weights without training
  • Verify LLM arithmetic accuracy via embedded RISC-V
  • Experiment with DeltaNet as addressable memory

// QUICK START

git clone https://github.com/divagr18/qwenputer && cd qwenputer/experimental/riscv && uv sync --extra hf
hf download divagr1925/Qwen3.5-4B-Qwenputer --local-dir qwenputer-rv

Commands are taken from the repository README — check the current version on GitHub before running.

#compiled-weights#gated-deltanet#interpretability#llm#qwen#transformers
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

FULL LOGS OF LLM EXPERIMENTS UNDERSTANDING NEURAL NETWORK WEIGHTS★ 1243STATIC CHAT FOR DEEPSEEK API★ 1701OPENLLM: RUN LLMS AS OPENAI API★ 12607OPEN-SOURCE FINANCIAL LLMS FOR DATA ANALYSIS★ 21591REA - REVERSE ENGINEERING WITHOUT BOUNDARIES★ 460891DeepSeek Plugin Framework★ 281341

← FULL CATALOG

divagr18/qwenputer — Risc-V CPU Compiled Into Qwen3.5 Weights