AIJupyter Notebook★ 53
walkinglabs/modern-llm-notebook
LLM course from scratch
Practical course on 23 Jupyter notebooks where you build tokenizers, attention, MoE, RLHF, inference, evaluation, and distillation of modern LLMs in PyTorch.
// KEY FEATURES
- Implement BPE and WordPiece tokenizers from scratch in PyTorch
- Build multi-head attention and transformer blocks, train them on data
- Construct a mixture-of-experts model and apply RLHF for alignment
- Run inference, evaluate metrics, and perform distillation of large models
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram