# Deep learning essentials

Read these guides in order to go from a single neuron to the architectures behind modern AI, and the training details that make them work.

## Stage 1: How networks learn

1. **[Deep learning](/topics/deep-learning)**  
   A practical map of deep learning: neural network building blocks, frameworks such as PyTorch and JAX, and architectures from CNNs to transformers and diffusion.
2. **[Neural networks](/tools/neural-networks)**  
   Neural networks learn by adjusting weights in layers of simple units. Learn neurons, layers, the forward pass, training and the MLP, CNN, RNN and GNN.
3. **[Backpropagation](/tools/backpropagation)**  
   Backpropagation computes how each weight in a neural network affects the error, using the chain rule, so gradient descent can improve the network.
4. **[Activation functions](/tools/activation-functions)**  
   Activation functions add the non-linearity that lets neural networks learn complex patterns. Compare sigmoid, tanh, ReLU, GELU, SiLU, softmax and SwiGLU.
5. **[Optimizers (SGD, Adam, AdamW)](/tools/optimizers)**  
   Optimizers decide how neural network weights move using gradients. Learn SGD, momentum, Adam, AdamW, learning-rate schedules and how to choose.
6. **[PyTorch](/tools/pytorch)**  
   PyTorch is an open-source deep learning framework with eager execution, autograd and torch.compile. Learn how it works and why research and industry use it.

## Stage 2: Architectures

7. **[CNN](/tools/cnn)**  
   CNN (convolutional neural network) explained step by step: filters, pooling, history, a worked example, limits, and how it relates to tabular models.
8. **[Recurrent neural networks](/tools/recurrent-neural-networks)**  
   Recurrent neural networks read sequences one step at a time and carry a memory forward. Learn training, vanishing gradients, LSTM, GRU and where they fit.
9. **[Attention mechanism](/tools/attention-mechanism)**  
   Attention lets a neural network weigh which parts of the input matter for each output. Learn queries, keys and values, self-attention and multi-head attention.
10. **[Transformers](/tools/transformers)**  
   Transformers explained: how self-attention works, where the architecture came from, and why it powers language, vision and tabular foundation models.
11. **[ViT](/tools/vit)**  
   ViT, the vision transformer, splits an image into patches and processes them with self-attention. Learn how it works and how it compares with CNNs.
12. **[Diffusion models](/tools/diffusion-models)**  
   Diffusion models explained: how they learn to reverse a noising process to generate images, audio and tables, with math, code, limits and tabular uses.

## Stage 3: Training well

13. **[Normalization layers](/tools/normalization-layers)**  
   Normalization layers rescale activations inside neural networks to stabilise and speed up training. Learn batch norm, layer norm, RMSNorm and their uses.
14. **[Weight initialization](/tools/weight-initialization)**  
   Weight initialization sets a network's starting weights so signals neither vanish nor explode. Learn zero, random, Xavier, He, orthogonal and LSUV methods.
15. **[Learning rate schedules](/tools/learning-rate-schedules)**  
   A learning rate schedule changes the step size during training for faster, more stable results. Learn step, cosine, warm-up, one-cycle and plateau schedules.
16. **[Mixed precision](/tools/mixed-precision)**  
   Mixed precision training uses 16-bit or 8-bit math for speed and keeps sensitive values in 32-bit. Learn FP16, BF16, FP8 and loss scaling in practice.

## How to use this path

Read the steps in order; each guide links onward to related topics, so you can go deeper wherever you need to. If you already know a stage, skip to the next one. All learning paths are listed on [Learning paths](/paths).
