AIJupyter Notebook★ 5,831⑂ 947 forksApache-2.0
rasbt/reasoning-from-scratch
LLM FROM SCRATCH IN PYTORCH
The reasoning-from-scratch project shows how to build a Reasoning LLM from scratch in PyTorch, step by step. Implemented in Jupyter notebooks: architecture, training with CoT and distillation, plus experiments with GRPO.
// KEY FEATURES
- Step-by-step transformer architecture and LLM training from scratch in PyTorch
- Integration of chain-of-thought (CoT) technique to improve model reasoning
- Application of distillation and GRPO methods to improve answer quality with less resources
- Everything done in Jupyter notebooks with visualization of training steps and inference
#ai#artificial-intelligence#chain-of-thought#deep-learning#distillation#grpo#inference-time-scaling#large-language-models#llm#llms#machine-learning#math-reasoning
Open on GitHub →New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram