Content
Summary
A convolutional neural network turns an image into a stack of feature maps by sliding small learned filters across it, applying a nonlinearity, and pooling to shrink the result. Repeating this builds from edges to textures to object parts, and the final features feed a classifier.
Intuition
Imagine scanning a medical image, such as a chest X-ray, with a tiny magnifying window. At each position the window asks one question, for example 'is there an edge here running left to right?' The answers across all positions form a feature map. Many filters ask many questions, giving many maps. A ReLU keeps only positive evidence, and pooling summarizes neighbouring answers so small shifts do not matter. Deeper layers look at maps instead of pixels, so they combine simple patterns into complex ones. Weights are shared across positions, so the network needs far fewer parameters than a fully connected one.
Key equations
Convolution (strictly cross-correlation): filter k slides over all input channels c to produce one output map.
ReLU nonlinearity applied element by element.
Output size from input size N, filter size F, padding P and stride S.
Max pooling keeps the strongest response in each window.
Derivation
- Start with input X of size N by N by C, for example a grayscale image with C=1.
- Choose K filters, each of size F by F by C. Each filter has F*F*C weights plus one bias.
- Place filter k at position (i,j), multiply elementwise with the patch under it, sum, and add the bias to get one number.
- Slide by stride S across the image; this fills a map of size O by O, with O given by the size formula.
- Stack the K maps to get an O by O by K volume.
- Apply ReLU to each value so negative responses become zero.
- Apply pooling with window 2 and stride 2, which halves height and width but keeps K channels.
- Repeat conv, ReLU, pool; then flatten or globally average and pass to a dense layer with softmax.
Worked example
A 32x32x3 image passes through a conv layer with 8 filters of 5x5, stride 1, padding 0, then ReLU, then 2x2 max pooling with stride 2. Give the output shape and the number of conv parameters.
- Conv output size: (32-5+0)/1+1=28, so the volume is 28x28x8.
- ReLU does not change shape.
- Pooling: 28/2=14, so the volume is 14x14x8.
- Parameters per filter: 5*5*3=75 weights plus 1 bias = 76.
- Total for 8 filters: 8*76=608.
Answer Output shape 14x14x8; the conv layer has 608 parameters.
Common mistakes
- Forgetting that each filter spans all input channels, so its depth equals the input depth.
- Thinking pooling has learnable parameters; it has none.
- Miscounting output size by forgetting the +1 or the padding term.
- Believing the number of output maps equals the number of input channels; it equals the number of filters.
- Skipping the nonlinearity, which would make stacked layers collapse into a single linear operation.