1958
The Perceptron
Foundational Papers That Made AI What It Is Today
From early neural networks and optimization methods to transformers, diffusion models, and LLMs.
Neural Networks
Transformers
Reinforcement Learning
Computer Vision
Generative Models
1. Foundations of Machine Learning
Early work on perceptrons, backpropagation, and neural network approximation laid the groundwork for deep learning.
1986
Learning Representations by Back-Propagating Errors
1989
Multilayer Feedforward Networks are Universal Approximators
2. Optimization & Training Deep Networks
Techniques that made training deep neural networks stable and efficient at scale.
1998
Gradient-Based Learning Applied to Document Recognition (LeNet)
2014
Adam: A Method for Stochastic Optimization
2015
Batch Normalization
3. Deep Learning for Computer Vision
Breakthrough convolutional network architectures that unlocked modern computer vision.
1998
LeNet-5
2012
ImageNet Classification with Deep Convolutional Neural Networks (AlexNet)
2014
Very Deep Convolutional Networks for Large-Scale Image Recognition (VGG)
2015
Deep Residual Learning for Image Recognition (ResNet)
2020
An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale (ViT)
4. Representation Learning
Learning compact, useful representations of data—embeddings, language models, and more.
2006
Reducing the Dimensionality of Data with Neural Networks
2013
Distributed Representations of Words and Phrases and their Compositionality (Word2Vec)
2014
GloVe: Global Vectors for Word Representation
5. Transformers & Large Language Models
The architectures and models that defined the transformer era in NLP and beyond.
2017
Attention Is All You Need
2018
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
2018
Improving Language Understanding by Generative Pre-Training (GPT-1)
2019
Language Models are Unsupervised Multitask Learners (GPT-2)
2020
Language Models are Few-Shot Learners (GPT-3)
2021
Learning Transferable Visual Models From Natural Language Supervision (CLIP)
6. Reinforcement Learning Landmarks
From tabular Q-learning to deep RL and game-playing agents like AlphaGo and AlphaZero.
1992
Q-Learning
2013–2015
Playing Atari with Deep Reinforcement Learning (DQN)
2016
Asynchronous Methods for Deep Reinforcement Learning (A3C)
2017
Proximal Policy Optimization Algorithms (PPO)
2016–2017
Mastering the Game of Go / Chess / Shogi with Deep Neural Networks and Search (AlphaGo / AlphaZero)
7. Generative Models
GANs, VAEs, and diffusion models that power modern image and media generation.
2013–2014
Auto-Encoding Variational Bayes (VAE)
2014
Generative Adversarial Networks (GAN)
2020
Denoising Diffusion Probabilistic Models (DDPM)
2022
High-Resolution Image Synthesis with Latent Diffusion Models (Stable Diffusion)
8. Scaling & Foundation Models
Work on scaling laws, open large models, and efficient fine-tuning that drives current foundation model development.
2022
Training Compute-Optimal Large Language Models
2023
LLaMA: Open and Efficient Foundation Language Models
2023
QLoRA: Efficient Finetuning of Quantized LLMs