R_REDDYX.XYZ
AIJupyter Notebook★ 2,960⑂ 286 forks

datawhalechina/diy-llm

DIY-LLM: FULL LLM COURSE WITH CODE

The diy-llm project offers six step‑by‑step Jupyter notebooks covering data pre‑training, tokenization, Transformer, MoE, distributed training, scaling laws, inference and LLM alignment. Each assignment progressively increases in difficulty, enabling you to build a full‑stack LLM from scratch.

// KEY FEATURES

  • Creates and trains a custom tokenizer on any text.
  • Implements Transformer blocks with Mixture‑of‑Experts (MoE) layers.
  • Trains models on multiple GPUs using PyTorch Distributed and DeepSpeed.
  • Evaluates scaling laws, performs inference and aligns LLMs via RLHF.
#datawhale#gpu-programming#grpo#llm#nlp#rl#sft#transformer
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

J-SPACE COGNITIVE SUITE★ 2958VORTEX — MOST STRUCTURED DATA FORMAT★ 2966AI Agent Orchestration Hub★ 2953Claude Extension Hub★ 2971AI AGENTS: THE DEFINITIVE GUIDE★ 2949OPEN COMPUTER CONTROL★ 2975

← FULL CATALOG

datawhalechina/diy-llm — Diy-LLM: Full LLM Course With Code