A short history of neural networks
Eighty years of ideas, two long winters, and one learning loop that never really changed.
- 1943
The artificial neuron
Warren McCulloch and Walter Pitts describe a simple mathematical model of a neuron that can compute logic.
- 1958
The perceptron
Frank Rosenblatt introduces the perceptron, a neuron that learns its weights from examples.
- 1960
ADALINE
Bernard Widrow and Ted Hoff at Stanford build ADALINE and the least-mean-squares learning rule, still used in signal processing today.
- 1969
Perceptrons, the book
Marvin Minsky and Seymour Papert show the limits of single-layer networks. Funding and interest decline, the first of the so-called AI winters for neural networks.
- 1986
Backpropagation
David Rumelhart, Geoffrey Hinton and Ronald Williams popularize backpropagation for training multi-layer networks.
- 1989
Reading handwriting
Yann LeCun and colleagues train a convolutional network to read handwritten zip codes, an early practical success.
- 1997
LSTM
Sepp Hochreiter and Jürgen Schmidhuber introduce long short-term memory networks, which can learn from long sequences.
- 2012
AlexNet
A deep convolutional network trained on GPUs wins the ImageNet image-recognition challenge by a wide margin, launching the deep learning boom.
- 2014
GANs
Ian Goodfellow and colleagues introduce generative adversarial networks, two networks competing to generate realistic images.
- 2017
The Transformer
Google researchers publish the Transformer architecture, the design behind today's large language models.
- 2019
Turing Award
Geoffrey Hinton, Yann LeCun and Yoshua Bengio receive the 2018 Turing Award for their work on deep learning.
- 2022
ChatGPT
A Transformer-based chatbot reaches a mass audience, bringing neural networks into everyday life.
- 2024
Nobel Prize
John Hopfield and Geoffrey Hinton win the Nobel Prize in Physics for foundational work on neural networks.