8 min readBuilding a Language Model
Tokenization and Embeddings
How text becomes numbers a model can train on: building a vocabulary, why byte pair encoding never needs an unknown token, the embedding layer as a lookup that is provably one-hot times a matrix, and why position has to be added back in by hand.
Generative AINatural Language ProcessingDeep Learning