C17 · Objects and correspondences
Features, alignment and motion
Matching is easy only when a location is distinctive. First inspect measured displacement, then ambiguity.
Theory
Compare patches, not isolated intensities
One pixel is indistinguishable from many others. A patch retains neighborhood structure; SSD sums squared differences from a candidate. Dark displacement-map cells mean low error. Its −3…3 coordinates are not photograph coordinates.
The example creates frame two with a known integer translation and searches a fixed 3×3 patch. Intensities stay constant; outside is zero. Real illumination changes, occlusions and deformation can violate this assumption, so low SSD need not mean a correct match.
Where the patch moved
A 3×3 patch is outlined on the left; the right frame shows a candidate in frame 2. Move the right outline to find a match. The error map contains all offsets from −3 to 3; selecting a cell chooses that candidate.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Inspect the selected point
What the plot shows
A dark cell means a low mean squared difference. A flat patch or repeating texture can have ambiguous minima.
- Minimum error
- 0
- Equal minima
- 4
Parameters and details
Compare the 3×3 frames. Select an error-map cell or move the candidate patch in frame 2. Darker means lower mean squared difference; all equal minima are outlined. True displacement changes the frames, whereas candidate displacement only changes the tested correspondence.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Why a straight edge leaves ambiguity
An image barely changes along a uniform straight edge. Motion along it looks like no motion, so several candidates have equal error: the aperture problem. A corner varies in two directions and is usually more informative.
The structure tensor sums local gradient products. Two small eigenvalues indicate a flat area, one large value an edge, and two substantial values a corner. Harris uses this structure to select features. A detector cannot by itself turn ambiguous evidence into an exact measurement.
Why an edge does not reveal all motion
A 3×3 patch is outlined on the left; the right frame shows a candidate in frame 2. Move the right outline to find a match. The error map contains all offsets from −3 to 3; selecting a cell chooses that candidate.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Inspect the selected point
What the plot shows
Along a straight vertical edge several offsets give equally low error. Choosing the first minimum does not prove the true motion.
- Minimum error
- 0
- Equal minima
- 7
Parameters and details
Compare the 3×3 frames. Select an error-map cell or move the candidate patch in frame 2. Darker means lower mean squared difference; all equal minima are outlined. True displacement changes the frames, whereas candidate displacement only changes the tested correspondence.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Correspondences need verification
Reference and further methods
SIFT and ORB offer different scale, rotation, speed and robustness trade-offs. Repeating patterns cause false matches. RANSAC tests small correspondence subsets against a shared geometric model and seeks consensus, without guaranteeing truth under a wrong model or insufficient evidence.
Optical flow estimates many point displacements using assumptions such as brightness constancy and local coherence. Object motion, camera motion and visible flow are different. A reliable interface must expose ambiguity or missing estimates, not merely a confident arrow everywhere.
Practice
Test the idea under another condition. Follow the task above the images; point inspection helps explain the result. This exploration is not scored.
Find a match in a different direction
The translation is different now. Keep the true displacement fixed: move only the candidate outline in frame 2. Compare the patch patterns first, then verify the choice using the error map.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Inspect the selected point
What the plot shows
Zero error means this patch matches under the stated assumptions. A straight edge can admit several equal solutions: that is an observation limit, not a control defect.
- Minimum error
- 0
- Equal minima
- 2
Parameters and details
Compare the 3×3 frames. Select an error-map cell or move the candidate patch in frame 2. Darker means lower mean squared difference; all equal minima are outlined. True displacement changes the frames, whereas candidate displacement only changes the tested correspondence.
Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.
Changing the example requires JavaScript and Canvas. Computed images remain available above.
Games
Test your explanation
Question 1 of 5
A uniform vertical edge has equal errors at different dy. Why?
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Exact values are in the point inspector
Grayscale 0…1: black to white.
Choose one answer. It is checked immediately.
Limitations and references
Limits of this example
The computation is exhaustive SSD for one patch on two synthetic frames, not full SIFT/ORB, RANSAC or dense flow. Ambiguity is demonstrated by actual equal errors.