Language Models and Generative AI
How text becomes numbers, how attention lets a token gather context from the whole sequence, and what pretraining then fine-tuning actually do to the weights.
Advanced440 XP~2 h90% to advance
Sign in to take quizzes, earn XP, and unlock stages as you reach 90% mastery.
Tokens and Embeddings
30 min · 120 XPText to token ids to vectors, and why an embedding layer is a lookup table rather than a matrix multiplication.
Open lesson →Sign in to take the 3-question quiz.
Attention, Step by Step
45 min · 180 XPQueries, keys and values on a worked three-token example: score, scale, normalise, blend - with every number computed by hand.
Open lesson →Sign in to take the 4-question quiz.
Pretraining, Then Fine-Tuning
35 min · 140 XPNext-token prediction on unlabelled text to obtain a foundation model, then a small labelled stage that adapts it - and what cross-entropy is really measuring.
Open lesson →Sign in to take the 3-question quiz.
Complete all modules → earn the Language Models and Generative AI badge