Skip to content
Kudos AI

Tagged “foundations”

7 articles.

7 min readSearch and Games

Constraint Satisfaction and Propagation

What changes when you describe a problem as variables, domains and constraints instead of as a black box: commutativity that shrinks the tree for free, propagation that proves branches hopeless before searching them, and a measurement showing the most famous ordering heuristic does nothing on its own.

Artificial IntelligenceSearch & Planning
8 min readSearch and Games

Classical Search: From Breadth-First to A*

Turning a problem into a state space and letting an algorithm walk it: what completeness and optimality actually cost, why memory rather than time defeats breadth-first search, and the two conditions on a heuristic that make A* provably optimal.

Search & PlanningArtificial Intelligence
8 min readNeural Networks

What Is a Neural Network?

Layers as parameterised transformations, the forward pass, and why depth and non-linearity are not optional: a proof that no single linear layer can compute XOR, and a two-layer network that does, worked entirely by hand.

Deep LearningMathematics
8 min readBuilding a Language Model

Tokenization and Embeddings

How text becomes numbers a model can train on: building a vocabulary, why byte pair encoding never needs an unknown token, the embedding layer as a lookup that is provably one-hot times a matrix, and why position has to be added back in by hand.

Generative AINatural Language ProcessingDeep Learning
7 min readProbability Foundations

Probability from Zero: The Language of Uncertainty

Build probability from the ground up: possible worlds, the sample space, the two basic axioms, and the addition and multiplication rules, each derived rather than asserted, with worked numeric examples.

ProbabilityMathematicsArtificial Intelligence
7 min readStatistical Learning Foundations

What Is Statistical Learning?

The setup behind every predictive model: estimating an unknown function f from data, the split between reducible and irreducible error, and why prediction and inference pull in different directions.

StatisticsMachine LearningMathematics
7 min readSupervised Learning

Linear Regression from First Principles

Derive the least-squares coefficients by differentiating the residual sum of squares, then work a complete five-observation fit by hand: coefficients, fitted values, residuals, RSS, and R-squared, each verified numerically.

StatisticsMachine LearningMathematics