C29 · Change and measure
Types of convolution
Move windows over pixels, separate passes, mix channels, and build a larger map from a smaller one.
Background if needed
Theory
1D, 2D, 3D: where the window moves
The rule stays the same: multiply the values inside a window by weights and write their sum to an output cell. A one-dimensional window moves along a sequence; a two-dimensional window moves across rows and columns; a three-dimensional window also moves between volume slices or video frames.
Channels form a separate axis. Three RGB channels do not turn an ordinary 2D convolution into 3D: its window still moves only across and down. The examples below each have one channel; every small square is one sample.
The window moves along one axis.
The window moves across and down.
The window also moves between slices. One depth position is shown here.
One rule, different sample locations
A regular 3×3 filter multiplies nine neighbors by nine weights. The sum rule remains the same here. We change three things separately: window movement, values beyond the boundary, and spacing between samples.
Where nine samples sit
Select a result cell. The nine frames on the left mark exactly the values used in its calculation. Change stride alone, then dilation alone. Compare the sample positions and output size.
Exact values are in the point inspector
Grayscale 0…9: black to white.
Exact values are in the point inspector
Grayscale 0…9: black to white.
Inspect the selected point
What the plot shows
At dilation 2, the window spans 5×5 cells but samples only nine. Stride 2 reduces the number of window positions. Padding permits evaluation nearer the boundary by supplying zeros.
- Window spans
- 3 px
- Weights
- 9
- Output side
- 5
Parameters and details
Select an output cell: exactly nine input samples are highlighted. Dilation spaces samples within a window; stride moves the whole window. A zero border adds numbers, not new camera data.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
A 1×1 window can be useful
A color image holds three numbers at each point; a neural feature map may hold more. A 1×1, or pointwise, convolution takes these values at one location. It creates new channel combinations without blurring the image or observing neighbors by itself.
1×1: mix channels, not neighbors
Select a colored pixel. Its R, G and B values are multiplied by three weights and added into one number. Set one weight to 1 and the others to 0: the output reproduces exactly one channel.
Exact values are in the point inspector
RGB; display range 0…1
Exact values are in the point inspector
Blue − · white 0 · red +; scale -3…3
Inspect the selected point
What the plot shows
One output channel needs three weights. Two new channels would need two independent weight triples. Spatial size stays fixed; channel count can change.
Parameters and details
1×1 mixes the channels at one point, not neighboring pixels. Negative weights are allowed: the output map contains signed numbers, not ready-to-display brightness.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Split one window into two passes
Sometimes one 3×3 square can be replaced by two passes: a narrow 3×1 column, followed by a 1×3 row. A real intermediate map is stored between them. This is spatially separable convolution: the calculation changes while the chosen Gaussian filter's result stays the same.
Down a column, then across a row
Select an output cell. The first pass takes three original pixels in a column; the second takes three already computed values in a row. Switch passes to follow where the numbers came from.
- InputOne map
Exact values are in the point inspector
Grayscale 0…9: black to white.
Exact values are in the point inspector
Grayscale 0…9: black to white.
Inspect the selected point
What the plot shows
The two passes give the same Gaussian smoothing as one square 3×3 kernel. Each pass's three weights are visible in the calculation; the sum is divided by 4. Not every kernel can be separated this way.
Parameters and details
Select a cell. The second pass uses the first pass's smoothed values, not the original pixels.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
All channels, groups, or one at a time
Now we separate channels rather than rows and columns. Dense convolution takes its own window from every input map and adds all contributions. Grouped convolution mixes maps only within each group. Depthwise gives every input map a separate kernel; this example produces one output map from each input.
Depthwise-separable uses two steps: separate spatial filtering, then 1×1 mixing. It uses fewer weights but is not equivalent to an arbitrary dense convolution. Select an output map and follow the input windows contributing to it.
Which channels contribute to one output map
These are four different feature maps, C1–C4, rather than four photograph colors. Select an output cell and change the operation. Dense convolution uses all four maps; two groups use two; depthwise uses one.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Inspect the selected point
What the plot shows
Depthwise does not mix maps. The last variant first filters each map separately, then a 1×1 convolution mixes the results. This constrains the weights differently; it does not guarantee the same result as dense convolution.
Parameters and details
Select an output map and cell. Compare which input layers contribute: all, a pair, or one. The last variant applies each channel's own kernel before 1×1 mixing.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Transposed does not mean inverse
A regular convolution gathers input values into a sum. Its transpose applies the related linear action in the opposite direction: each input value multiplies a kernel and adds a footprint to the output. Overlapping footprints add their contributions.
Overlap can be uneven: one cell receives more contributions than its neighbor. This can create checkerboard patterns. Resize-then-convolve works differently: first repeat nearest pixels on a denser grid, then filter it. The architectures chapter compares these artifacts in more detail.
This is only the selected cell's contribution. Other input cells add their own footprints to the same output. Stride 2 sets the distance between neighboring footprint origins.
One input paints an entire window
In the transposed version, select an input pixel to see its 3×3 footprint on the right. Select an output pixel to inspect every contribution added there. Compare this with resizing before regular convolution.
Exact values are in the point inspector
Grayscale 0…3: black to white.
Exact values are in the point inspector
Grayscale 0…12: black to white.
Inspect the selected point
What the plot shows
Transposed convolution adds footprints. The other operation first creates a denser grid, then moves a regular filter over it. Matching output sizes do not make the operations equivalent. Neither automatically recovers lost detail.
Parameters and details
Select an input cell: its kernel stamps a 3×3 patch on the right. Select an output cell to see which input values add there.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
What if individual samples move?
Dilation spaces the entire sampling grid evenly. Deformable convolution gives each sample its own offset, allowing the window to adapt to a shape. Below is one computed example with manually chosen offsets; a neural network predicts offsets at each image location.
Regular 3×3
Dilation 2
Individual offsets
Frames mark the nine sampled cells. The last example manually follows the bright shape. In a real deformable convolution, offsets can fall between pixels; interpolation then supplies the value.
Reading other names
Reference and further methods
Neural-network convolution usually means cross-correlation without reversing the kernel. True convolution reverses it. Full, same and valid describe boundary and output-size policies, not a different multiplication rule. FFT convolution evaluates the same linear operation through a spectrum; circular boundaries require explicit handling.
Masked convolution zeros disallowed weights, for example to prevent an output from seeing pixels that have not been generated yet. Causal convolution uses only the present and past in a sequence. These constrain available data; neither dilation nor channel separation provides them automatically.
Practice
Test the idea under another condition. Follow the task above the images; point inspection helps explain the result. This exploration is not scored.
A wider window with the same nine weights
Dilation starts at 2, stride at 1, with a zero border. Find an output cell whose samples touch both the shape and background. Compare with dilation 1: which samples no longer contribute? Increase stride: did the weight count change, or only the number of outputs?
Exact values are in the point inspector
Grayscale 0…9: black to white.
Exact values are in the point inspector
Grayscale 0…9: black to white.
Inspect the selected point
What the plot shows
At dilation 2, the window spans 5×5 cells but samples only nine. Stride 2 reduces the number of window positions. Padding permits evaluation nearer the boundary by supplying zeros.
- Window spans
- 5 px
- Weights
- 9
- Output side
- 5
Parameters and details
Select an output cell: exactly nine input samples are highlighted. Dilation spaces samples within a window; stride moves the whole window. A zero border adds numbers, not new camera data.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Your own convolution experiment
In the playground, edit every kernel weight, stride, padding and channel connection using a tiny example or your own photograph.
Open the convolution playgroundGames
Test your explanation
Question 1 of 6
A 7×7 input, 3×3 kernel, stride 2, no padding. How many window positions fit along each side?
Exact values are in the point inspector
Grayscale 0…9: black to white.
Exact values are in the point inspector
Grayscale 0…9: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…9: black to white.
Exact values are in the point inspector
Grayscale 0…9: black to white.
Choose one answer. It is checked immediately.
Limitations and references
Limits of this example
The small tools compute the shown operations without training. The 1D/2D/3D and offset diagrams are static examples with actual values; they do not train 3D kernels or a deformable model. Grouping examples use a fixed number of output maps and omit biases so pixel contributions remain visible.