Encyclopedia

A concise, cross-linked reference. Each entry connects to related concepts and the articles that go deeper.

Browse by topic

A

Activation Function

The non-linear function applied to a layer’s output, without which a network of any depth would collapse to a single linear transformation.

Deep Learning

B

Backpropagation

The algorithm that computes the gradient of a neural network’s loss with respect to every weight, by applying the chain rule backwards through the network.

Deep LearningOptimizationMathematics

C

Convolutional Neural Network

A neural network that applies learned filters across an input’s spatial extent, sharing weights so the same pattern is detected wherever it occurs.

Deep LearningComputer Vision

Cross-Entropy

A measure of the difference between two probability distributions, used as the standard loss function for classification.

Information TheoryDeep LearningMachine Learning

N

Neural Network

A model composed of layers of simple units, each computing a weighted sum followed by a non-linear function, fitted by gradient descent using backpropagation.

Deep LearningMachine LearningArtificial Intelligence

P

Pretraining and Fine-Tuning

The two-stage recipe of first training a model on a large generic corpus, then adapting it to a specific task with a much smaller labelled dataset.

Generative AIDeep LearningMachine Learning

S

Self-Attention

A mechanism that lets every position in a sequence attend to every other, computing each output as a weighted sum of values whose weights come from query-key similarity.

Generative AIDeep LearningNatural Language Processing

T

Transformer

A neural architecture built on stacked self-attention and feed-forward layers, which replaced recurrence as the standard for sequence modelling.

Generative AIDeep LearningNatural Language Processing