CNN

Overview

A CNN, or convolutional neural network, is a neural network that slides small learned filters across a grid of values, such as the pixels of an image. Each filter looks at a small neighbourhood, so the network learns local patterns first and combines them into larger ones. Because the same filter is reused at every position, a CNN needs far fewer parameters than a fully connected network and handles shifts in the input gracefully.

CNNs powered the modern wave of computer vision and remain a workhorse for images, audio spectrograms, signals and any data with a grid or sequence layout. They are also useful for understanding tabular learning. Tables do not have the neighbourhood structure that convolutions assume, which explains why models for tables took a different path. This article covers how a CNN works, a worked example, strengths and limits, and its relation to models that learn from raw tables.

What it is

A CNN is a stack of layers that includes at least one convolution layer. A convolution layer holds a set of small weight grids called filters or kernels. Each kernel is placed over a patch of the input, multiplied element by element, and summed to a single number. The kernel then moves one step and repeats. The grid of resulting numbers is called a feature map.

Everyday analogy: imagine a stencil with a little pattern cut into it, say a short diagonal line. You slide the stencil over a drawing and mark each spot where the drawing matches the pattern. The marked sheet is a feature map. A CNN learns hundreds of such stencils, and learns what patterns to cut into them.

The three ideas behind it

• Local connectivity. Each output looks only at a small window of the input. Nearby values matter most in images.
• Weight sharing. The same kernel is used everywhere. A cat's ear is a cat's ear in the top left or the bottom right.
• Hierarchy. Early layers find edges and colours. Middle layers find textures and parts. Late layers find whole objects.

Typical building blocks

Where it came from

The story of CNNs starts with biology and ends in a huge engineering effort.

• Visual cortex studies in the mid 20th century showed that neurons respond to simple patterns in small regions of the visual field. This inspired layered, local feature detectors.
• Neocognitron (1980) was an early layered model with local feature extraction and a form of pooling.
• LeNet (late 1980s to 1998) trained a small convolutional network with backpropagation to read handwritten digits. It is the direct ancestor of modern CNNs.
• A quiet period followed. Support vector machines and hand-made features dominated image tasks, and compute was limited.
• AlexNet (2012) won a large image recognition contest by a wide margin. It used a deep CNN, ReLU units, dropout and GPUs. This result started the deep learning boom in vision.
• VGG (2014) showed that stacks of small 3 by 3 filters work well. Inception modules mixed filter sizes in parallel.
• ResNet (2015) added skip connections so that very deep networks could be trained. See ResNet.
• EfficientNet (2019) scaled depth, width and resolution together. See EfficientNet.
• Vision transformers (2020) challenged CNNs by treating images as sequences of patches. See ViT.
• ConvNeXt (2022) modernised the CNN with design choices borrowed from transformers and closed the gap. See ConvNeXt.

The pattern is clear: ideas travel in both directions. CNNs gave transformers the idea of hierarchy, and transformers gave CNNs better training recipes.

How it works

Let us follow one image through a small CNN.

Step 1: The input as numbers

A colour image is three grids, one each for red, green and blue. A 32 by 32 image has shape 3 by 32 by 32. Values are scaled to a small range, often by subtracting a mean and dividing by a spread.

Step 2: Convolution

A kernel of size 3 by 3 spans all three channels, so it holds 27 weights plus a bias. At each position it computes a weighted sum of the 27 input values under it. Sliding the kernel over all positions gives one feature map. A layer with 16 kernels gives 16 feature maps stacked together.

For a single channel, the operation at one position looks like this:

input patch      kernel         output
1 2 0            1 0 -1
3 1 2     *      1 0 -1    =   (1*1 + 2*0 + 0*-1) +
0 1 4            1 0 -1         (3*1 + 1*0 + 2*-1) +
                                (0*1 + 1*0 + 4*-1) = 1 + 1 - 4 = -2

This particular kernel responds to vertical edges: it gives a large number where the left side is bright and the right side is dark.

Step 3: Nonlinearity

Each value passes through a function such as ReLU, which sets negatives to zero. Without this step, a stack of convolutions would collapse into one big linear operation.

Step 4: Pooling or striding

Max pooling takes the largest value in each small window, for example 2 by 2. It halves the height and width and makes the result less sensitive to tiny shifts. Many modern networks use strided convolutions instead.

Step 5: Repeat to build a hierarchy

After each block the maps become smaller but more numerous. A typical pattern is: 32 by 32 with 16 maps, then 16 by 16 with 32 maps, then 8 by 8 with 64 maps. The network trades spatial detail for richer features.

Step 6: The receptive field grows

The receptive field of a unit is the region of the input that can influence it. One 3 by 3 layer sees 3 by 3 pixels. Two stacked layers see 5 by 5. Three see 7 by 7. Pooling and stride make it grow even faster. This is how deep layers come to see whole objects.

Step 7: Classification head

The final maps are averaged or flattened into a vector. A linear layer produces one score per class, and a softmax turns the scores into probabilities.

Step 8: Learning

The loss, usually cross entropy, compares predictions with labels. Backpropagation computes how each kernel weight should change. An optimiser such as stochastic gradient descent or Adam updates the weights. Over many passes, kernels become useful detectors without being told what to detect.

Output size formula

For input width W, kernel size K, padding P and stride S, the output width is:

out = floor((W - K + 2P) / S) + 1

For W = 32, K = 3, P = 1, S = 1, the output is 32. For W = 32, K = 3, P = 0, S = 2, the output is 15.

Parameter counting

A convolution with C_in input channels, C_out output channels and a K by K kernel has C_in * C_out * K * K + C_out parameters. A 3 by 3 layer from 64 to 128 channels has 73,856 parameters. A fully connected layer joining a 32 by 32 by 3 input to 1,000 units would have more than three million. The saving comes from weight sharing.

Key concepts and terms

A worked example

Numeric example

Take a tiny 5 by 5 single-channel input and a 3 by 3 kernel with stride 1 and no padding.

input                kernel
0 0 1 1 1            1 0 -1
0 0 1 1 1            1 0 -1
0 0 1 1 1            1 0 -1
0 0 1 1 1
0 0 1 1 1

The output is 3 by 3. At the top left position the patch is columns 0 to 2 of rows 0 to 2. The sum is: row by row, 0*1 + 0*0 + 1*-1 = -1, three times, giving -3. At the next position, columns 1 to 3: 0*1 + 1*0 + 1*-1 = -1, three rows, giving -3. At the right position, columns 2 to 4: 1*1 + 1*0 + 1*-1 = 0, giving 0.

So each row of the output is -3 -3 0. The kernel fires strongly where it crosses the boundary from dark to bright. Learning replaces this hand-made kernel with kernels found by gradient descent.

A small PyTorch model

import torch
import torch.nn as nn
import torch.nn.functional as F

class SmallCNN(nn.Module):
    def __init__(self, n_classes=10):
        super().__init__()
        self.conv1 = nn.Conv2d(3, 16, kernel_size=3, padding=1)
        self.bn1 = nn.BatchNorm2d(16)
        self.conv2 = nn.Conv2d(16, 32, kernel_size=3, padding=1)
        self.bn2 = nn.BatchNorm2d(32)
        self.conv3 = nn.Conv2d(32, 64, kernel_size=3, padding=1)
        self.bn3 = nn.BatchNorm2d(64)
        self.head = nn.Linear(64, n_classes)

    def forward(self, x):                       # (B, 3, 32, 32)
        x = F.max_pool2d(F.relu(self.bn1(self.conv1(x))), 2)   # 16x16
        x = F.max_pool2d(F.relu(self.bn2(self.conv2(x))), 2)   # 8x8
        x = F.max_pool2d(F.relu(self.bn3(self.conv3(x))), 2)   # 4x4
        x = x.mean(dim=(2, 3))                  # global average pool
        return self.head(x)

model = SmallCNN()
x = torch.randn(8, 3, 32, 32)
print(model(x).shape)                           # torch.Size([8, 10])
print(sum(p.numel() for p in model.parameters()))

A training loop

opt = torch.optim.AdamW(model.parameters(), lr=1e-3, weight_decay=1e-2)
for epoch in range(10):
    model.train()
    for xb, yb in train_loader:
        logits = model(xb)
        loss = F.cross_entropy(logits, yb)
        opt.zero_grad()
        loss.backward()
        opt.step()
    model.eval()
    correct = total = 0
    with torch.no_grad():
        for xb, yb in val_loader:
            correct += (model(xb).argmax(1) == yb).sum().item()
            total += yb.numel()
    print(epoch, correct / total)

Shapes through the small model

Tracking tensor shapes is the best way to understand a CNN. Here is the flow through the small model above for a batch of 8.

Notice how spatial size falls from 32 to 4 while channels rise from 3 to 64. The network exchanges where for what. Early on it knows exactly where a pixel is. At the end it knows which patterns are present, with only coarse position.

Common layer variants

• Depthwise separable convolution. Splits a convolution into a per-channel spatial filter and a 1x1 mix across channels. It cuts cost sharply and is the base of many mobile models.
• Grouped convolution. Splits channels into groups that are processed separately.
• Dilated convolution. Spreads the kernel taps apart to widen the view without extra weights.
• Transposed convolution. Upsamples a map, used in segmentation and in decoders.
• Residual connection. Adds the input of a block to its output, which keeps gradients flowing. See ResNet.
• Squeeze and excitation. Re-weights channels based on global context.
• Large kernels. Seven by seven depthwise kernels are used in modern designs such as ConvNeXt.

What the layers learn

If you visualise first-layer filters of a trained image network, they look like oriented edges, colour blobs and simple gratings. Second-layer units respond to corners and curves. Deeper units respond to textures, then parts like eyes or wheels, then whole objects. This is not hand-designed. It emerges because each layer builds on the one before it, and because the training signal rewards whatever helps the final task.

A useful everyday picture is reading. First you recognise strokes, then letters, then words, then sentences. Each level uses the one below it. A CNN does the same with visual features.

A 1D example for signals

Convolutions work on sequences too. A one-dimensional CNN slides a kernel along time. It suits sensor streams, audio and logs.

class SignalCNN(nn.Module):
    def __init__(self, in_ch=4, n_classes=3):
        super().__init__()
        self.net = nn.Sequential(
            nn.Conv1d(in_ch, 32, 7, padding=3), nn.ReLU(),
            nn.MaxPool1d(2),
            nn.Conv1d(32, 64, 5, padding=2), nn.ReLU(),
            nn.AdaptiveAvgPool1d(1),
        )
        self.head = nn.Linear(64, n_classes)

    def forward(self, x):                       # (B, channels, time)
        return self.head(self.net(x).squeeze(-1))

A convolution over a column of a table

Even though columns lack neighbours, the values inside one sorted or time-ordered column do. Here is a sketch that embeds a variable-length list of values from one column into a fixed vector, then uses it as a column descriptor.

class ColumnEncoder(nn.Module):
    """Embeds a sample of values from one column into a single vector."""
    def __init__(self, dim=64):
        super().__init__()
        self.conv = nn.Sequential(
            nn.Conv1d(1, 32, 5, padding=2), nn.ReLU(),
            nn.Conv1d(32, dim, 5, padding=2), nn.ReLU(),
        )

    def forward(self, values):                 # (n_values,)
        v = (values - values.mean()) / (values.std() + 1e-6)
        v = v.sort().values                    # order by magnitude
        h = self.conv(v.view(1, 1, -1))        # (1, dim, n)
        return h.mean(dim=-1).squeeze(0)       # (dim,)

enc = ColumnEncoder()
print(enc(torch.randn(500)).shape)             # torch.Size([64])

Sorting the values first turns a set into a smooth curve, which is a shape a convolution can read. Two columns with similar distributions get similar vectors, even if they have different names or none at all. This is one practical way a column can be described by its content.

Strengths and limitations

Strengths

• Efficient use of parameters. Weight sharing keeps models small.
• Good inductive bias for grids. Locality and translation equivariance match images and signals.
• Strong with modest data. CNNs often beat transformers when data is limited.
• Fast on hardware. Convolutions map well to GPUs and accelerators, and many tools optimise them. See TensorRT and ONNX Runtime.
• Mature ecosystem. Pre-trained models, recipes and tooling are plentiful, and transfer learning works well.
• Interpretable features. Early filters and activation maps can be visualised.

Limitations

• Locality can be a handicap. A small kernel cannot link distant parts of an input in one step, so long-range structure needs depth.
• Weak on non-grid data. Tables, graphs and sets have no natural neighbourhood. See GCN for graphs.
• Sensitive to scale and rotation. Equivariance covers shifts, not rotation or zoom. Augmentation is used to cover this.
• Fixed input geometry. Standard networks assume similar image sizes.
• Pooling discards position. Fine location is partly lost.
• Can lean on texture. Networks sometimes rely on background or texture rather than shape.
• Data hungry at scale. Very large CNNs still need big datasets or strong augmentation.

How it compares with alternatives

A key lesson for practice: pick the model whose built-in assumptions match the structure of the data. CNNs assume neighbourhoods. Trees assume axis-aligned splits. Transformers assume little and learn the rest from data.

Role in tabular foundation models

Models that learn from raw tables must deal with a structure unlike an image. Understanding where CNN ideas fit and where they do not is useful.

Why plain CNNs do not fit tables

A pixel's neighbours are meaningful. A cell's neighbours in a table are not, because column order is usually arbitrary. If you swap two columns, the table means the same thing, but a convolution would see different neighbours. Rows are also exchangeable: shuffling them changes nothing. A model for tables should respect these symmetries, and a vanilla CNN does not.

This is why strong tabular models use attention between columns and between rows, rather than sliding windows. See FT-Transformer, SAINT, TabPFN, TabICL and axial attention.

Where CNN ideas still help

• Table images and charts. If the table arrives as a scanned page or a chart, a CNN reads it. Document understanding often starts with a convolutional backbone.
• Within a column. For long sorted columns or time-stamped series, one-dimensional convolutions can pick up local patterns such as trends and seasonality.
• Embedding of cells. Some designs use small convolutions over the digits or characters of a value to embed codes, ids and strings.
• Wide, ordered feature sets. When columns have a real order, such as spectral bands or lag features, a 1D convolution across columns is sensible.
• Weight sharing as a design idea. Reusing the same small network for every column or cell, with a column embedding, plays a similar role to a shared kernel.

Mixed types and missing values

CNNs need dense numeric tensors. Tables bring categories, dates, text and gaps. To use a CNN-style component you must encode categories as embeddings or numbers and decide how to represent missing values, for example with a mask channel. Foundation models for tables handle this at the cell level: each cell becomes a token with a type-aware embedding, and missing cells get a mask token. See masked cell modeling and imputation networks.

No column names

A CNN never needed names, since pixels have positions. A tabular model that ignores names must learn what each column is from its values and co-variation. Statistics of a column, such as its distribution, play the role that local texture plays in an image. A small convolutional or set-based encoder over a column's values can produce a column embedding that stands in for a missing header.

Pre-training, in-context learning and continual learning

Vision CNNs rely on pre-training followed by fine-tuning. Table models extend this with pre-training on large numbers of synthetic or real tables, so that a new task can be solved by reading labelled rows in context, with no weight update. See few-shot learning and transfer learning. When new data keeps arriving, methods from continual learning protect old skills. See experience replay and EWC.

Imputation, anomalies and semantics

• Imputation. Convolutional autoencoders fill gaps in images and signals. For tables, masked cell models do the same job across columns.
• Anomaly detection. A convolutional autoencoder that rebuilds normal signals badly on faulty ones gives a score. Table models use per-cell reconstruction error. See anomaly detection.
• Sector and semantic classification. Image classifiers label what a picture shows. Semantic heads on a table encoder label what a column or a table is about. See semantic sector heads.

In short, CNNs are the right tool when your data is a grid. Tabular foundation models borrow the spirit of CNNs (reuse, hierarchy, pre-training) but replace the sliding window by attention. For how this plays out in a full system, see /model/architecture, /model/research and /model/benchmarks.

Practical guidance and best practices

Start from a pre-trained backbone

For images, begin with a model trained on a large image set, such as ResNet or EfficientNet, and fine-tune. This usually beats training from scratch unless you have very large data.

Use data augmentation

Random crops, flips, colour jitter and small rotations expand the effective dataset. Choose augmentations that keep the label valid. Flipping a digit 6 may turn it into something else.

Normalise inputs and use batch normalisation

Scale inputs to a consistent range. Batch normalisation or a similar layer makes deeper networks easier to train.

Pick sensible kernel sizes

Small 3 by 3 kernels stacked in depth are usually better than one large kernel. Large kernels are making a comeback in modern designs, but they need care.

Watch the learning rate

Use warm-up and a decaying schedule. If loss explodes early, lower the rate. If it stalls, check data and labels before changing the model.

Regularise

Weight decay, dropout in the head, data augmentation and early stopping all help. Use validation data that matches real use.

Use efficient training

Mixed precision speeds up training and reduces memory. Data parallelism spreads batches across devices. PyTorch and TensorFlow both have mature convolution support.

Inspect what the model learns

Look at activation maps and saliency maps. Check that the model attends to the object and not the background.

Deploy with care

Export to a portable format, quantise if needed, and test accuracy after each change. Latency depends on hardware, so measure on the target device.

Regularisation recipe in more detail

Choosing input size

Higher resolution reveals small details but costs more. Compute grows with the square of the side length. A common practice is to train at a modest size, then fine-tune for a short time at a larger size. Match the size used at test time to the size used at the end of training.

Measuring the right thing

Use metrics that fit the task. Accuracy suits balanced classes. For imbalance, report precision, recall and area under the curve. For detection and segmentation, use overlap measures. Always report variance across seeds, because small gains can be noise.

Handling small datasets

• Freeze the early layers of a pre-trained network and train the head only.
• Use strong augmentation.
• Try a smaller model before a larger one.
• Consider a self-supervised pre-training stage on unlabelled data, such as MAE or SimCLR.

Quick checklist before training

1. Confirm labels are correct on a random sample.
2. Check that train, validation and test sets do not share near-duplicates.
3. View a batch of augmented images to confirm they still look valid.
4. Overfit a tiny subset first. If the model cannot reach near-zero loss on 32 samples, there is a bug.
5. Log the learning rate, loss and accuracy per epoch.
6. Save the best checkpoint by validation score, not by the final epoch.

Common mistakes

• Using a CNN on data without neighbourhood structure. Shuffled columns in a table make convolutions meaningless.
• Forgetting to match the preprocessing of a pre-trained model. Mean, spread and image size must match.
• Leaking test data into augmentation or scaling statistics.
• Over-pooling early. Removing too much resolution in the first layers loses small details.
• Ignoring class imbalance. Accuracy can look high while rare classes fail.
• Training and evaluating at different image sizes without checking. Accuracy can shift.
• Skipping a baseline. A simple model can reveal that the CNN is not the issue.
• Trusting accuracy on a single split. Use several seeds or folds.
• Reading feature maps too literally. Visualisations help, but they are not proof of reasoning.
• Neglecting batch size effects on normalisation. Very small batches make batch normalisation noisy.

Frequently asked questions

What does CNN stand for?

CNN stands for convolutional neural network. It is a neural network built around convolution layers, which slide small learned filters across a grid of values. The name is also used for a news broadcaster, but in machine learning it always means this type of network.

Why are CNNs good at images?

Images have local structure: nearby pixels are related, and the same pattern can appear anywhere. A CNN builds those facts into its design through local filters and weight sharing. This makes it efficient and accurate with fewer parameters and less data than a model with no such assumptions.

What is a convolution in simple words?

It is a sliding weighted sum. A small grid of weights is placed over a patch of the input, each pair of overlapping numbers is multiplied, and the products are added to give one output value. The grid is then moved and the process repeats across the whole input.

What is pooling for?

Pooling shrinks a feature map by summarising each small window with its largest or average value. This reduces computation, enlarges the receptive field of later layers, and makes the output less sensitive to small shifts of the input. Some modern networks replace it with strided convolutions.

Can CNNs be used on text or time series?

Yes. A one-dimensional CNN slides filters along a sequence and can detect local patterns such as phrases, spikes or cycles. They are fast and easy to train, though transformers and state space models are often preferred when long-range dependencies matter.

Are CNNs still relevant now that transformers exist?

Yes. Modern CNNs are competitive with vision transformers, train well on modest data, and run efficiently on many devices. Designs such as ConvNeXt show that convolutional models can match transformer accuracy when trained with similar recipes.

Can I use a CNN on a spreadsheet?

You can, but it is rarely the best choice. Columns in a spreadsheet do not have a natural neighbour relationship, so convolutions have no meaningful local pattern to find. Tree models, attention-based table models and tabular foundation models are better matched to this kind of data.

How many layers should a CNN have?

There is no fixed answer. Small tasks work with a handful of layers, while large image tasks use dozens or hundreds, made trainable by skip connections. Start from a known design that fits your compute budget and adjust depth and width based on validation results.

What is the difference between a CNN and a fully connected network?

A fully connected layer links every input to every output with its own weight. A convolution links each output only to a small window and reuses the same weights everywhere. This makes CNNs far smaller for grid-like inputs and gives them built-in tolerance to shifts.

How do CNNs relate to tabular foundation models?

They share the spirit of reuse and pre-training but differ in structure. Tabular foundation models replace sliding windows with attention across cells, rows and columns, because tables lack neighbourhoods. CNN ideas still appear in components that embed values, read table images, or scan ordered series inside a column.

Glossary

• Activation function. A nonlinear function applied after a layer.
• Backpropagation. The method that computes gradients through the network.
• Batch normalisation. A layer that standardises activations over a batch.
• Channel. One slice of depth in a tensor.
• Convolution. A sliding weighted sum with shared weights.
• Dilation. Spacing between kernel elements.
• Equivariance. Output shifts when the input shifts.
• Feature map. The output of one filter.
• Filter or kernel. A small grid of learned weights.
• Global average pooling. Averaging each map to one value.
• Max pooling. Taking the largest value in each window.
• Padding. Extra values added at borders.
• Receptive field. The input region that affects a unit.
• ReLU. A function that keeps positives and zeros out negatives.
• Stride. The step of the sliding window.
• Weight sharing. Reusing the same weights at every position.

Summary

A CNN slides small learned filters over a grid, builds a hierarchy of features from edges to objects, and shares weights so that it stays compact and tolerant of shifts. It grew from early layered models and handwritten digit readers into the base of modern computer vision, and it still performs well on images, audio and signals.

Tables lack the neighbourhood structure that convolutions need, so models that learn from raw tables use attention across cells, rows and columns instead. CNN ideas remain useful as components and as design inspiration: shared weights, hierarchy and pre-training. Pick the architecture whose assumptions match your data, start from strong pre-trained models when you can, and verify with sound validation. Related pages include ResNet, ConvNeXt and transformers.

Related

It belongs to Architectures in my stack, under AI and machine learning. Others in the same group: