Understand and build large language models, intuitively.
From tokens to a trained transformer, then through pre-training, post-training and the research frontier. Every hard idea gets a picture, a worked example with tiny numbers, an interactive you can poke at, and code you can run.
LLM Fundamentals
ArchitecturePre-trainingPost-training
Linear attentionDistillationLoRAMoEOptimizersRL & RLHFInference
35chapters
~28hof guided study
118interactive visualizations
140+original figures
1 yearroadmap to a working LLM
How this course teaches
Most LLM material is either a wall of equations or a wall of hype. This course sits in between: it explains the hard part, then makes it obvious with an example, a picture, and something you can drag.
Problem firstEvery chapter opens with what breaks without the idea. Then the idea feels inevitable.
Tiny numbersEvery equation gets a worked example with 3-dimensional vectors you can check by hand.
InteractivesSliders, steppers and heatmaps for attention, softmax, RoPE, scaling laws, MoE routing, LoRA rank, and more.
Build itRunnable PyTorch in
code/: a BPE tokenizer, attention, a full GPT-2 that loads OpenAI's weights, a training loop.Check yourselfQuizzes with explanations and exercises with solution sketches at the end of every chapter.
Track progressMark chapters complete. Progress lives in your browser. Press ⌘K to search everything.
Your one-year roadmap
You can absolutely do this in a year with 4–6 focused hours per week. The path is designed so you always have something running before moving on.
Month 1Foundations: the math you need, PyTorch, tokenization, embeddings. Ship: your own BPE tokenizer.
Month 2Positional encoding, attention, the transformer block. Ship: a working attention layer with tests.
Month 3Next-token prediction, sampling, GPT-2 from scratch. Ship: GPT-2 that loads OpenAI weights and generates text.
Month 4–5Pre-training: objectives, modern architecture details, scaling laws, data. Ship: a 10M-parameter model trained on a small corpus.
Month 6Infrastructure: mixed precision, parallelism, FlashAttention, evaluation. Ship: a multi-GPU or gradient-accumulated run with eval curves.
Month 7–8Post-training: SFT, DPO, GRPO, tool use, safety. Ship: an instruction-following chat model built on your own pre-trained model.
Month 9–10Advanced: linear attention and SSMs, LoRA, MoE, optimizers, distillation. Ship: a LoRA fine-tune and a tiny MoE.
Month 11RL fundamentals and RLHF deep dive. Ship: a reward model and a PPO/GRPO loop on a toy task.
Month 12Inference, quantization, capstone. Ship: a quantized, KV-cached model served behind an API, with a write-up.
Course map
0 of 35 chapters completed
Who this is for
- Engineers who use LLMs and want to know what is really happening inside, well enough to fine-tune, debug and deploy.
- Students and researchers who need to get current on transformers, scaling laws, RLHF and the post-transformer frontier, fast.
- Curious people with some Python and high-school math. The math chapter fills the rest.
Built as an open study companion. All figures are original; papers are cited inline so you can go to the source.