Convolution: how a CNN sees edges
Convolutional networks look at images through small filters that slide across them. Choose a filter, edit its nine numbers and watch the feature map appear.
Interactive simulation. It needs JavaScript; the text below explains what it shows.
What you are looking at
The left grid is a handwritten 8; the white square is the 3×3 window the kernel is looking at. The kernel's nine weights multiply the nine pixels under it and the sum, after ReLU, becomes one pixel of the feature map on the right. Move the mouse over either grid to inspect any position.
Things to try
- Vertical edges: the left and right strokes light up, the horizontal ones vanish.
- Blur: the digit softens; this is what pooling and downsampling smooth out.
- Turn off ReLU: negative responses appear in blue.
- Type your own kernel, for example all zeros with a 1 in a corner, to shift the image.
How it works in real systems
A trained CNN learns its kernels instead of using hand-made ones. The first layer of an image model typically ends up with edge and colour detectors very like these; deeper layers combine them into textures, parts and objects.
Code lab
The same idea in real code, step by step. Copy each block into a notebook or a file and run it in order.
Step 1 · Convolution by hand
Slide a 3×3 kernel over the image; each output pixel is the sum of the elementwise product. (Deep-learning libraries compute cross-correlation and call it convolution.)
import numpy as np
def conv2d(img, k, stride=1, pad=0):
img = np.pad(img, pad); kh, kw = k.shape
H = (img.shape[0] - kh) // stride + 1; W = (img.shape[1] - kw) // stride + 1
out = np.zeros((H, W))
for i in range(H):
for j in range(W):
out[i, j] = (img[i*stride:i*stride+kh, j*stride:j*stride+kw] * k).sum()
return out
sobel_x = np.array([[-1, 0, 1], [-2, 0, 2], [-1, 0, 1]])
img = np.zeros((8, 8)); img[:, 4:] = 1 # a vertical edge
print(conv2d(img, sobel_x))
Step 2 · The same in PyTorch, then learn the kernels
import torch, torch.nn as nn
x = torch.tensor(img, dtype=torch.float32)[None, None] # (batch, channels, H, W)
conv = nn.Conv2d(1, 1, 3, bias=False); conv.weight.data[:] = torch.tensor(sobel_x, dtype=torch.float32)
print(conv(x)[0, 0])
cnn = nn.Sequential(nn.Conv2d(1, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),
nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(), nn.AdaptiveAvgPool2d(1), nn.Flatten(), nn.Linear(32, 10))
print(sum(p.numel() for p in cnn.parameters()), "parameters")
Train cnn with the loop from the training-run code lab (reshape the digits to 1×8×8) and the first-layer kernels become edge and stroke detectors.