Tagged “learning-theory”

2 articles.

14 September 20264 min readStatistical Learning Theory

The Theorem That Says Nothing About Your Problem

Averaged over all 256 functions from three bits to one, a nearest-neighbour learner and a learner built to be wrong on purpose both score exactly 0.500000 off the training set. That is the no free lunch theorem, it is exactly true, and the moment the average is restricted to the six functions that depend on a single bit the two separate to 0.333333 and 0.666667.

Machine LearningMathematics

11 September 20267 min readStatistical Learning Theory

Why Learning From Data Works At All

The gap between the error you measure and the error you will suffer, why picking the best of a thousand identical hypotheses makes it look 0.1149 better than chance, how capacity is counted for infinite model classes, and the theorem that equalises every learner - with the assumption that makes it true.

Machine LearningMathematics