What Does “Close” Mean? Mahalanobis Distance for Visual Defects

9 minutes ago

2 min read

Two image patches can be equally far from “normal”—yet one is much more unusual.

The missing detail is correlation.

Same distance, different meaning

Imagine a simplified patch embedding with two features. In normal images, both have mean zero, variance one, and correlation 0.9: they usually rise and fall together.

Now consider two patches:

A = (1, 1): both features increase together.
B = (1, −1): one increases while the other decreases.

Both sit at Euclidean distance √2 from the mean. But B breaks the relationship normal data usually follows.

Euclidean distance alone misses that distinction.

Let normal variation define the distance

Mahalanobis distance accounts for the covariance of normal features:

dₘ(x, μ) = √[(x − μ)ᵀ Σ⁻¹ (x − μ)]

Here, μ is the normal mean and Σ describes feature variances and correlations. The inverse covariance gives more weight to deviations along directions where normal data varies less. Formula reference.

For our example, the distances become 1.03 for A and 4.47 for B.

Same Euclidean distance. Very different departure from normal structure.

Geometrically, this is whitening: transforming the normal feature cloud into a roughly spherical shape. Euclidean distance after whitening becomes Mahalanobis distance in the original space.

Where this appears in visual inspection

PaDiM models normal patch embeddings with a Gaussian at each spatial position, then measures deviations using Mahalanobis distance.

The May 2026 Mahalanobis PatchCore preprint takes a different approach: it whitens reduced embeddings before searching a memory bank of normal patches. The reference is a nearby stored patch—not simply the normal mean.

There is a catch: poorly estimated covariance can exaggerate tiny, noisy directions. Regularization stabilizes the calculation; representative normal data remains essential. Neither guarantees better detection on every dataset.

Before changing the backbone, I’d ask whether the distance function matches the structure of normal data.