Özer Cem KELAHMETKELAHMET
Learning paths

Deep learning essentials

Read these guides in order to go from a single neuron to the architectures behind modern AI, and the training details that make them work.

Stage 1: How networks learn

  1. Deep learning
    A practical map of deep learning: neural network building blocks, frameworks such as PyTorch and JAX, and architectures from CNNs to transformers and diffusion.
  2. Neural networks
    Neural networks learn by adjusting weights in layers of simple units. Learn neurons, layers, the forward pass, training and the MLP, CNN, RNN and GNN.
  3. Backpropagation
    Backpropagation computes how each weight in a neural network affects the error, using the chain rule, so gradient descent can improve the network.
  4. Activation functions
    Activation functions add the non-linearity that lets neural networks learn complex patterns. Compare sigmoid, tanh, ReLU, GELU, SiLU, softmax and SwiGLU.
  5. Optimizers (SGD, Adam, AdamW)
    Optimizers decide how neural network weights move using gradients. Learn SGD, momentum, Adam, AdamW, learning-rate schedules and how to choose.
  6. PyTorch
    PyTorch is an open-source deep learning framework with eager execution, autograd and torch.compile. Learn how it works and why research and industry use it.

Stage 2: Architectures

  1. CNN
    CNN (convolutional neural network) explained step by step: filters, pooling, history, a worked example, limits, and how it relates to tabular models.
  2. Recurrent neural networks
    Recurrent neural networks read sequences one step at a time and carry a memory forward. Learn training, vanishing gradients, LSTM, GRU and where they fit.
  3. Attention mechanism
    Attention lets a neural network weigh which parts of the input matter for each output. Learn queries, keys and values, self-attention and multi-head attention.
  4. Transformers
    Transformers explained: how self-attention works, where the architecture came from, and why it powers language, vision and tabular foundation models.
  5. ViT
    ViT, the vision transformer, splits an image into patches and processes them with self-attention. Learn how it works and how it compares with CNNs.
  6. Diffusion models
    Diffusion models explained: how they learn to reverse a noising process to generate images, audio and tables, with math, code, limits and tabular uses.

Stage 3: Training well

  1. Normalization layers
    Normalization layers rescale activations inside neural networks to stabilise and speed up training. Learn batch norm, layer norm, RMSNorm and their uses.
  2. Weight initialization
    Weight initialization sets a network's starting weights so signals neither vanish nor explode. Learn zero, random, Xavier, He, orthogonal and LSUV methods.
  3. Learning rate schedules
    A learning rate schedule changes the step size during training for faster, more stable results. Learn step, cosine, warm-up, one-cycle and plateau schedules.
  4. Mixed precision
    Mixed precision training uses 16-bit or 8-bit math for speed and keeps sensitive values in 32-bit. Learn FP16, BF16, FP8 and loss scaling in practice.

How to use this path

Read the steps in order; each guide links onward to related topics, so you can go deeper wherever you need to. If you already know a stage, skip to the next one. All learning paths are listed on Learning paths.

Last updated Nov 16, 2026
Type to search.
    ↑↓ navigate↵ open