Learning paths
Deep learning essentials
Read these guides in order to go from a single neuron to the architectures behind modern AI, and the training details that make them work.
Stage 1: How networks learn
- Deep learning
A practical map of deep learning: neural network building blocks, frameworks such as PyTorch and JAX, and architectures from CNNs to transformers and diffusion. - Neural networks
Neural networks learn by adjusting weights in layers of simple units. Learn neurons, layers, the forward pass, training and the MLP, CNN, RNN and GNN. - Backpropagation
Backpropagation computes how each weight in a neural network affects the error, using the chain rule, so gradient descent can improve the network. - Activation functions
Activation functions add the non-linearity that lets neural networks learn complex patterns. Compare sigmoid, tanh, ReLU, GELU, SiLU, softmax and SwiGLU. - Optimizers (SGD, Adam, AdamW)
Optimizers decide how neural network weights move using gradients. Learn SGD, momentum, Adam, AdamW, learning-rate schedules and how to choose. - PyTorch
PyTorch is an open-source deep learning framework with eager execution, autograd and torch.compile. Learn how it works and why research and industry use it.
Stage 2: Architectures
- CNN
CNN (convolutional neural network) explained step by step: filters, pooling, history, a worked example, limits, and how it relates to tabular models. - Recurrent neural networks
Recurrent neural networks read sequences one step at a time and carry a memory forward. Learn training, vanishing gradients, LSTM, GRU and where they fit. - Attention mechanism
Attention lets a neural network weigh which parts of the input matter for each output. Learn queries, keys and values, self-attention and multi-head attention. - Transformers
Transformers explained: how self-attention works, where the architecture came from, and why it powers language, vision and tabular foundation models. - ViT
ViT, the vision transformer, splits an image into patches and processes them with self-attention. Learn how it works and how it compares with CNNs. - Diffusion models
Diffusion models explained: how they learn to reverse a noising process to generate images, audio and tables, with math, code, limits and tabular uses.
Stage 3: Training well
- Normalization layers
Normalization layers rescale activations inside neural networks to stabilise and speed up training. Learn batch norm, layer norm, RMSNorm and their uses. - Weight initialization
Weight initialization sets a network's starting weights so signals neither vanish nor explode. Learn zero, random, Xavier, He, orthogonal and LSUV methods. - Learning rate schedules
A learning rate schedule changes the step size during training for faster, more stable results. Learn step, cosine, warm-up, one-cycle and plateau schedules. - Mixed precision
Mixed precision training uses 16-bit or 8-bit math for speed and keeps sensitive values in 32-bit. Learn FP16, BF16, FP8 and loss scaling in practice.
How to use this path
Read the steps in order; each guide links onward to related topics, so you can go deeper wherever you need to. If you already know a stage, skip to the next one. All learning paths are listed on Learning paths.