Skip to content
Kudos AI

Tagged “overfitting”

3 articles.

4 min readStatistical Learning Theory

One Parameter, Infinite Capacity

A classifier with exactly one real parameter fits all 1,048,576 labellings of twenty points, every time, and predicts a twenty-first at 0.5038 accuracy over twenty thousand trials. Counting parameters measures neither an upper nor a lower bound on what a model class can fit, which is why capacity has to be measured some other way.

Machine LearningMathematics
7 min readStatistical Learning Theory

Why Learning From Data Works At All

The gap between the error you measure and the error you will suffer, why picking the best of a thousand identical hypotheses makes it look 0.1149 better than chance, how capacity is counted for infinite model classes, and the theorem that equalises every learner - with the assumption that makes it true.

Machine LearningMathematics
7 min readStatistical Learning Foundations

The Bias-Variance Tradeoff

The exact decomposition of expected test error into squared bias, variance, and irreducible noise, demonstrated numerically with a 2,000-run simulation where all three terms are measured separately and shown to add up.

StatisticsMachine LearningMathematics