Learning Probabilistic Models
When the data are complete, learning a probability model is counting - the derivative of the log likelihood does the rest. When variables are hidden there is nothing to count, and the repair is to guess the counts, refit, and repeat until the likelihood stops rising.
Sign in to take quizzes, earn XP, and unlock stages as you reach 90% mastery.
Maximum Likelihood with Complete Data
25 min · 100 XPThe three-step recipe - write the likelihood, differentiate the log, set it to zero - applied to a discrete parameter, to a network of them, and to a Gaussian, together with the small-sample failure it walks straight into.
Open lesson →Sign in to take the 3-question quiz.
Priors, Smoothing, and Naive Bayes
25 min · 100 XPWhat maximum likelihood does to an event it has never seen, why a Beta prior is the natural repair, and the classifier that takes the whole apparatus, assumes away every dependence, and works anyway.
Open lesson →Sign in to take the 3-question quiz.
The EM Algorithm and Hidden Variables
30 min · 120 XPMixtures of Gaussians as the standard case of learning without labels, expected counts as the substitute for counts, the guarantee that the likelihood never falls, and a measured case where soft assignment recovers a structure hard assignment cannot.
Open lesson →Sign in to take the 3-question quiz.