C22 · Store and compare

Quality, errors and verification

One score cannot answer every question. First define error and what constitutes a valid reference.

Background if needed

Pixels, arrays and transparency

Theory

A metric begins with a fair comparison

MSE averages squared pixel errors; RMSE restores the signal's units. PSNR uses a declared peak range, here 1. Zero error gives infinite PSNR, not an arbitrary large number. Coordinates, scales and value domains must agree.

A clean synthetic scene provides a genuine reference. An uploaded photo without a known clean original cannot have an invented PSNR. Filter comparisons should lock disturbance, boundaries and parameters; otherwise input changes are mistaken for method differences.

An average hides where damage occurs

Adding a to all 256 pixels gives $MSE=a 2$ . Adding 4a only to a 4×4 patch gives $16 \times (4a) 2 256 = a 2$ . The scores match, yet one shifts the whole image while the other severely damages a small region. A residual map and profile reveal the difference.

SSIM compares local structure under a chosen window and is not a universal judge. IoU and Dice suit masks, but a missing thin channel may need boundary or topology assessment. Learned perceptual metrics depend on a trained model and may disagree with the task or a human.

Same MSE, different damage

On the left every pixel receives a; on the right only a 4×4 patch receives 4a. Select inside the patch, then outside it: the line and profile reveal the difference hidden by the average score.

Amount a Amount a0.1

Uniform error +a16 × 16

Cross-section: row 3

Exact values are in the point inspector

Grayscale 0…1: black to white.

Error +4a in a 4×4 patch16 × 16

Damaged patch · Cross-section: row 3

Exact values are in the point inspector

Grayscale 0…1: black to white.

Inspect the selected point

ViewFit×2×4

What the plot shows

Cross-section along row 3

The image line marks this same row. Select a higher or lower point to move the cross-section; compare values at the same column.

Reference
Everywhere +a
Patch +4a

Selected column: 3

Relocating error can preserve MSE and PSNR while changing what is damaged. Inspect the residual map, not just the number.

MSE: 0.01
RMSE: 0.1
PSNR (range 0–1): 20 dB
Uniform-error MSE: 0.01
Local-error MSE: 0.01

Parameters and details

Shifting the whole image by a and a 4×4 patch by 4a gives equal MSE: patch area is 1/16. Visible errors differ. Metrics use unclipped data, a known aligned reference and a declared 0–1 peak range.

MSE = mean((result − reference)²)PSNR = −10 log₁₀(MSE)

Relocating error can preserve MSE and PSNR while changing what is damaged. Inspect the residual map, not just the number.

Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.

Changing the example requires JavaScript and Canvas. Computed images remain available above.

An artifact suggests which assumption failed

Ringing near edges relates to frequency truncation; halos to excess detail boost; blocking to independent block processing; banding to coarse quantization. Stair steps and moiré can arise from undersampling. Different causes need different fixes, not one quality slider.

Verification includes constants, impulses, ramps, frame edges, missing samples and parameter extremes. Iterations need bounded stopping, finite numbers and checks of the claimed objective. Full-size files also need device-specific memory and timing measurements. A repeatable attractive example is evidence, not proof of universal reliability.

Practice

Test the idea under another condition. Follow the task above the images; point inspection helps explain the result. This exploration is not scored.

Check two equal scores

The error is stronger. Compare both MSE values, then inspect a cross-section through the damaged patch and another below it. One row cannot describe the whole image.

Amount a Amount a0.2

Uniform error +a16 × 16

Cross-section: row 3

Exact values are in the point inspector

Grayscale 0…1: black to white.

Error +4a in a 4×4 patch16 × 16

Damaged patch · Cross-section: row 3

Exact values are in the point inspector

Grayscale 0…1: black to white.

Inspect the selected point

ViewFit×2×4

What the plot shows

Cross-section along row 3

The image line marks this same row. Select a higher or lower point to move the cross-section; compare values at the same column.

Reference
Everywhere +a
Patch +4a

Selected column: 3

Matching curves outside the patch do not imply matching images. Scores, error maps and measurement locations must answer the same task.

MSE: 0.04
RMSE: 0.2
PSNR (range 0–1): 13.9794 dB
Uniform-error MSE: 0.04
Local-error MSE: 0.04

Parameters and details

Shifting the whole image by a and a 4×4 patch by 4a gives equal MSE: patch area is 1/16. Visible errors differ. Metrics use unclipped data, a known aligned reference and a declared 0–1 peak range.

MSE = mean((result − reference)²)PSNR = −10 log₁₀(MSE)

Matching curves outside the patch do not imply matching images. Scores, error maps and measurement locations must answer the same task.

Select an image point. Arrows move it; Home and End reach row edges. Viewing does not change the calculation.

Changing the example requires JavaScript and Canvas. Computed images remain available above.

Games

Test your explanation

Question 1 of 5

Can true PSNR be measured for an uploaded photo without a clean reference?

Choose one answer. It is checked immediately.

Without JavaScript, use the questions below. Open each answer after making your choice.

Can true PSNR be measured for an uploaded photo without a clean reference?

No
Yes, from how attractive it looks
Yes, from file size

Answer and explanation

No. Without a reference the pixel error is unknown.

Two images have equal MSE. What follows?

Their mean squared error is equal
They are equally useful for every task
They must be pixelwise identical

Answer and explanation

Their mean squared error is equal. Error location and significance can still differ.

How should two denoisers be compared fairly?

Same input, disturbance, scale and reference
Give each a fresh random disturbance
Auto-stretch each result independently

Answer and explanation

Same input, disturbance, scale and reference. Differences can then be attributed to processing.

The clean reference is shifted one pixel from the result. Can MSE be compared directly?

No: MSE is never usable for images
Coordinates must be aligned first
Yes: MSE finds correspondences automatically

Answer and explanation

Coordinates must be aligned first. Otherwise positional mismatch confounds processing error: different scene points are compared.

Each result was independently stretched from its own minimum to maximum. What can be hidden?

Only object positions
Nothing: display choices never affect conclusions
Overall brightness and contrast errors

Answer and explanation

Overall brightness and contrast errors. Different numbers can look identical. A fair comparison needs a common display scale.

Limitations and references

Limits of this example

MSE/RMSE/PSNR are computed only against the known synthetic reference. SSIM, perceptual and no-reference metrics are explained, not substituted by MSE.

A metric begins with a fair comparison

An average hides where damage occurs

Same MSE, different damage

An artifact suggests which assumption failed

Check two equal scores

Test your explanation

Limits of this example

Mathematical reference

Sources and further reading