Yoshua Bengio

Advanced neural approaches to language, including distributed word representations.

Biography

A Canadian computer scientist whose work on neural language models introduced learning distributed representations of words jointly with the model that uses them, an idea that became foundational for subsequent natural language processing.

Key contributions

• Neural probabilistic language models learning distributed word representations.
• Work on attention mechanisms for sequence-to-sequence models.
• Sustained theoretical and empirical work on training deep architectures.

Impact

Learned dense word representations replaced hand-built lexical features in natural language processing, and the attention mechanisms developed in this line of work were the direct precursor to the transformer.