C08 · Change and measure
Linear filters and convolution
How nine neighboring numbers produce one new value, from averaging to blur and edge detection.
Background if needed
Theory
Multiply each weight by its sample
Begin with a small shape: black cells store 0 and white cells store 1. Take the 3 × 3 square around the selected cell, add its nine values and divide by 9. The middle panel exposes every contribution. Four white cells out of nine, for example, produce 4/9, a gray result.
A kernel is another small table, containing weights. The mean has nine weights of 1 and divisor 9. The Gaussian approximation uses 1–2–1 / 2–4–2 / 1–2–1 and divisor 16, favoring cells near the center. Try grayscale, then RGB. Color uses the same computation separately for three channels; the channel control reveals one of them.
Which contributions produce a pixel?
Select a cell in either image. The left frame marks its neighbors, the middle shows their values, weights and products, and the right shows the result. Start with a black-and-white mean, then try grayscale and color.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Inspect the selected point
What the plot shows
Each curve point is a pixel value on the marked horizontal line. Select another image row to move the graph. Compare edge width and preservation of detail.
- Input
- Output
The mean replaces a pixel with its neighbors' average; the Gaussian kernel favors the center. Sobel measures change, so negative results are valid. For color, the same rule runs independently on the selected R, G or B channel.
Parameters and details
Correlation uses weights as drawn; convolution rotates them by 180°. An asymmetric kernel exposes the difference. Derivative and sharpening preserve negative values: display colours do not alter the array. Weights below are after the chosen rotation, in row order.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Linearity and array boundaries
Sobel uses negative and positive weights: constant values cancel while transitions remain. This measures change rather than a new brightness. Negative results are retained and use a separate display scale. The edges chapter develops both gradient directions and Canny.
In additional settings, correlation applies weights as drawn; convolution first rotates the kernel by 180°. A symmetric mean is unchanged; the asymmetric upper kernel reveals the difference. The displayed calculation already uses the selected convention.
For a fixed linear operator, filtering a sum equals summing filtered inputs and scale passes through. Clipping, rounding and data-dependent weights can break this. Intermediate negative values are therefore preserved.
Zero, repeat and reflection are different assumptions about scene continuation. Our reflection does not duplicate the edge cell. Periodic extension connects opposite borders and appears in Fourier methods. Comparisons need matching boundary rules.
Same filter, fewer computations
Box filtering gives equal weights; Gaussian and binomial filters reduce distant contributions. A separable kernel factors into two vectors: rows, then columns. Direct and factored paths below are computed independently and compared.
Select a result cell: the left view opens the horizontal pass, and the calculation takes its three vertical neighbours with weights 1, 2, 1, then divides by 4. The intermediate is therefore not an extra illustration: its values produce the result. Multiple repetitions display the last such transition.
An integral image obtains a rectangle sum from four prefix values. FFT can help large kernels but needs correct padding and circular-convolution handling. Direct calculation is often cheaper for small kernels; changing computation must not change the operation's meaning.
Two short passes
The intermediate is a horizontal pass, the result includes a vertical pass. Comparison against direct 3 × 3 filtering is actually computed.
Exact values are in the point inspector
Grayscale 0…1: black to white.
- Last vertical pass
- Normalize weights 1 + 2 + 1
Exact values are in the point inspector
Grayscale 0…1: black to white.
Inspect the selected point
What the plot shows
This kernel factors into two vectors. An arbitrary kernel may not.
- Maximum difference
- 0
Parameters and details
The kernel [1,2,1]ᵀ·[1,2,1]/16 factors into horizontal and vertical filters. Each result needs 3+3 contributions instead of 9: less work with matching borders and values. For multiple repetitions the intermediate belongs to the last one; its vertical pass produces the displayed result.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Practice
Test the idea under another condition. Follow the task above the images; point inspection helps explain the result. This exploration is not scored.
Predict a kernel rotation
Select a cell at the shape's edge and check its new value by hand. Compare the mean with Gaussian weights, then Sobel. As an extra task, select an image corner and change the boundary rule.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Inspect the selected point
What the plot shows
Each curve point is a pixel value on the marked horizontal line. Select another image row to move the graph. Compare edge width and preservation of detail.
- Input
- Output
The mean replaces a pixel with its neighbors' average; the Gaussian kernel favors the center. Sobel measures change, so negative results are valid. For color, the same rule runs independently on the selected R, G or B channel.
Parameters and details
Correlation uses weights as drawn; convolution rotates them by 180°. An asymmetric kernel exposes the difference. Derivative and sharpening preserve negative values: display colours do not alter the array. Weights below are after the chosen rotation, in row order.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Your own convolution experiment
In the playground, edit every kernel weight, stride, padding and channel connection using a tiny example or your own photograph.
Open the convolution playgroundGames
Predict a kernel rotation
Choose the kernel from its effect on edges and the thin line.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Compare your result with the reference.
Inspect the selected point
What the plot shows
Each curve point is a pixel value on the marked horizontal line. Select another image row to move the graph. Compare edge width and preservation of detail.
- Input
- Output
The mean replaces a pixel with its neighbors' average; the Gaussian kernel favors the center. Sobel measures change, so negative results are valid. For color, the same rule runs independently on the selected R, G or B channel.
Parameters and details
Correlation uses weights as drawn; convolution rotates them by 180°. An asymmetric kernel exposes the difference. Derivative and sharpening preserve negative values: display colours do not alter the array. Weights below are after the chosen rotation, in row order.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Test your explanation
Question 1 of 5
A 3 × 3 mean acts on a white image. Why does the corner darken with zero padding?
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Blue − · white 0 · red +; scale -1…1
Choose one answer. It is checked immediately.
Limitations and references
Limits of this example
Live 3 × 3 kernels and a separable binomial filter. FFT and integral images are explained accelerations, not claimed as the executed algorithm.
Mathematical reference
- Numbers, functions and grids
- Matrices, operators and bases
- Frequencies, phase and scale
- Numerical computation