C23 · Learned image methods
A network, step by step
From a familiar filter to channels, nonlinearity and one actual learning step.
Background if needed
Theory
An image becomes feature maps
Apply a familiar filter to each color channel using its own weights, then add the responses. The new map contains filter responses, not red, green or blue brightness. This example has one output channel; eight maps would need eight independent weight sets.
Next comes ReLU: max(0, z). Why? A chain of linear filters can collapse into another linear transform. Zeroing the negative part adds nonlinearity. Here you can see exactly which values change; pooling then shrinks the map. Tensor means a multidimensional array, such as this 16×16×3 input.
One filter sees three channels
First, the filter detects edges: three color channels produce one response map. Select ReLU in the diagram: that map becomes the input, and negative values turn into zeros on the right. Pooling then combines four neighboring cells into one. Select a map cell to see its calculation.
- RGB input16×16×3 · three channels
Exact values are in the point inspector
RGB; display range 0…1
Exact values are in the point inspector
Blue − · white 0 · red +; scale -0.75…0.75
Inspect the selected point
What the plot shows
The filter responds to local changes; ReLU keeps one response sign, and pooling keeps the strongest or mean response in each block. These are three different operations. Diagram shapes are width × height × channels.
- Input channels
- 3
- Weights + bias
- 28
- Output-map side
- 8
Parameters and details
Left is the selected operation's input; right is its result. Weights are authored, not trained. Sums use a fixed −0.75…0.75 scale; ReLU and pooling use 0…0.75. Color represents a filter response, not an object's original color.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
What changes shape and what changes values
ReLU replaces negative numbers with zero. A 2×2 maximum or average with stride 2 halves both sides. Stride is the spacing between evaluated centers; padding adds boundary values; dilation spaces kernel taps apart. These are distinct operations even when their output shapes match.
A side length is floor((n+2p−d(k−1)−1)/s)+1 when the kernel fits. With 16 cells, kernel 3, padding 1 and stride 1, output is 16; without padding it is 14. An output cell's possible dependency region is its receptive field. This does not say which pixels actually mattered most.
Where weights come from
A fixed filter has author-chosen weights. Training compares a prediction with a target through a loss function and updates weights to reduce it. Backpropagation differentiates the operation chain; an optimizer uses those derivatives for a step. The next example's single derivative can be checked by hand.
A filter learns from a target map
Select Target map: the task is to reverse the familiar edge filter's direction. Return to the prediction and increase update count. Actual weights and the whole map change, not a separate scalar. Then select the held-out image: fitting remains unchanged while the same filter is tested elsewhere.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…2
Inspect the selected point
What the plot shows
Each point uses the entire training map, not only the selected pixel. The held-out image changes the views, never this curve or the fitted weight.
- Training image
At η=0 the weight stays fixed. At η=1 this single-multiplier task is solved in one update. Values 1…1.5 overshoot but converge with alternating error. New shapes test only this simple filter, not recognition quality.
- Weight before
- 1
- Weight after
- -0.75
- Loss before
- 0.0016
- Loss after
- 0
Parameters and details
Only the 3×3 filter's shared multiplier is learned. Loss is mean squared target-map error divided by 2. The update is normalized by training-response mean square, so η controls the fraction of weight error corrected. The held-out image never enters the gradient.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Check one gradient by hand
Reference and further methods
The filter above shares one parameter across all pixels. This smaller calculation retains one input number so the derivative sign and update direction can be checked by hand. It is an arithmetic reference for the image change already demonstrated.
How a weight learns from error
The target is 1 from input 2: prediction equals 2w. Choose the starting weight, then increase step size η. Two markers on the curve show where the weight moves and whether loss decreases. There are no pixels here: this is one scalar task.
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…4
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…4
Calculation
What the plot shows
The curve gives loss at each possible weight. Before marks the current weight; after marks one step against its gradient. Lower on this same curve means lower loss. Step size changes the distance between points, not the task itself.
- L(w)
- Before step: -0.5; 2
- After step: -0.1; 0.72
The minimum is at w=0.5. At η=0 nothing learns; a larger allowed step can cross the minimum while still reducing loss.
- Loss before
- 2
- Loss after
- 0.72
- Prediction before: 2w
- -1
- Prediction after: 2w′
- -0.2
Parameters and details
One example x=2, y=1 and one parameter w. Updating w is learning; evaluating 2w with fixed w is inference. This exact small problem is not an image-recognition training run.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
What else actual training must check
Reference and further methods
Ordinary inference leaves weights fixed. Training data fits weights, validation helps choose settings, and test data evaluates the selected procedure. The held-out image above is excluded from fitting: it receives the already fitted rule. Two authored patterns demonstrate this mechanism but cannot establish quality on real photographs.
Input normalization must match training. Augmentation creates valid example changes: flipping need not preserve a digit or medical image's meaning. Batch normalization uses different statistics during training and ordinary inference. These conditions concern actual network training and use; they are not hidden modes of our one-weight example.
Practice
Test the idea under another condition. Follow the task above the images; point inspection helps explain the result. This exploration is not scored.
Test the same filter on a held-out image
The new rectangle never entered fitting. Predict the edge-response direction after three updates, then test it. Compare prediction with the target map and select an edge with a nonzero response. Switch to the training image: the fitted weight must not change.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…2
Inspect the selected point
What the plot shows
Each point uses the entire training map, not only the selected pixel. The held-out image changes the views, never this curve or the fitted weight.
- Training image
At η=0 the weight stays fixed. At η=1 this single-multiplier task is solved in one update. Values 1…1.5 overshoot but converge with alternating error. New shapes test only this simple filter, not recognition quality.
- Weight before
- 1
- Weight after
- 1
- Loss before
- 0.0463
- Loss after
- 0.0463
Parameters and details
Only the 3×3 filter's shared multiplier is learned. Loss is mean squared target-map error divided by 2. The update is normalized by training-response mean square, so η controls the fraction of weight error corrected. The held-out image never enters the gradient.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Games
Test your explanation
Question 1 of 5
Left: weighted sum; right: ReLU. Why did some responses disappear?
Exact values are in the point inspector
Blue − · white 0 · red +; scale -0.75…0.75
Exact values are in the point inspector
Grayscale 0…0.75: black to white.
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…2
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…2
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…2
Exact values are in the point inspector
Blue − · white 0 · red +; scale -2…2
Choose one answer. It is checked immediately.
Limitations and references
Limits of this example
Demonstration-layer weights are hand-authored; there is no object recognition or full-network training here. Image learning fits only a shared multiplier of a fixed kernel; held-out pixels never enter fitting. A separate scalar reference explains one quadratic loss.