Feature-Space Distance

Measuring how different two sets of images are by embedding both and comparing the distributions — a number that belongs to the encoder as much as to the images, until you divide it by that encoder’s own floor.

What it is

Ask “how different are these two sets of tiles?” and the modern answer is: push both through a fixed encoder, fit a Gaussian to each cloud of features, and measure the distance between the two Gaussians. That is the Fréchet Inception Distance, so named because the original fixed encoder was Inception-v3.

It is worth knowing about here even though the group does not build generative models, because the same machinery answers three questions that do come up:

Are these two cohorts different? — scanner drift, site effects, an old archive versus a new one.
Did this processing step change the images? — stain normalisation, compression, a scanner upgrade.
Are these synthetic tiles realistic? — the use it was invented for.

The appeal is that it needs no labels and no task. It compares distributions of appearance directly, which is exactly what Scanner and Stain Variability is about and what a downstream accuracy comparison can only measure indirectly.

The catch: the number is a property of the encoder

In pathology the Inception network is routinely swapped for a histology foundation model, on the assumption that a domain encoder gives a more meaningful score. Bhat et al. 2026 measured what that substitution does, and the answer is that it changes the result: for one fixed pair of tile sets, the raw distance spans about thirty-fold across six common encoders, and the ordering does not follow embedding dimension.

The reason is structural rather than incidental. The distance is computed entirely inside the encoder’s feature space, so it inherits that space’s scale, geometry and learned invariances — and encoders differ in all three. A raw score is therefore unreadable without naming the encoder, and two papers each reporting a number are not necessarily comparing anything.

The fix, and why it is a ratio

Divide the measured distance by that encoder’s own within-cohort floor: the distance between two disjoint halves of the same cohort, at matched sample size. That floor is what finite-sample estimation alone produces — the reading you get when there is genuinely nothing to find.

ratio = distance(A, B) / distance(half₁ of A, half₂ of A)

Numerator and denominator carry the same per-encoder feature scale, so the ratio cancels it. Subtracting the floor would not — a difference leaves the scale intact. That distinction is the whole trick, and it is the kind of thing that is obvious afterwards and easy to get backwards first.

Measured as coefficient of variation across six encoders on one comparison, this took CV from 1.19 to 0.13 within cohort and 0.99 to 0.42 across cohorts. The second number is the honest one: normalisation reconciles encoders well when the comparison is essentially null and only partly when a real shift is present.

The floor is useful on its own, before any of this. It is a calibration reading — the value of “no difference” in your own data with your own encoder at your own sample size. Any distance not comfortably above it is noise, and nothing in this wiki currently records one.

Two kinds of encoder, and matching one to the question

Once normalised, encoders separate consistently — across cohort drift, stain shift and generative evaluation — into a sensitive group (CONCH, Phikon-v2, Inception-v3) and an invariant group (UNI2-h, Virchow2, Prov-GigaPath).

The split does not follow the training objective: DINOv2 encoders appear in both groups. What seems to drive it is feature scale, training-cohort breadth and augmentation strength — heavy augmentation during pretraining teaches an encoder to ignore precisely the nuisances (colour, focus, scanner) that a drift detector needs to notice.

So this is a design choice, not a quality ranking, and the rule falls out of it:

To detect drift — scanner differences, stain shift, a suspect batch — use a sensitive encoder. Being fooled less is not the goal; noticing is.
To measure something that should be nuisance-invariant, use a strongly augmented one, accepting that it will understate nuisance shift.

A blinded reader study in the same paper is what makes this framing trustworthy: two pathologists’ similarity orderings tracked feature distance at Spearman ρ ≈ 0.96 under encoders from both tiers. The tiers agree on the direction of difference and differ on its magnitude. Neither is disagreeing with a pathologist; they are disagreeing with each other about scale.

Three cautions that come with the metric

It is biased at finite sample size, and the bias is encoder-dependent. Two comparisons are only comparable at matched N — which means the floor has to be computed at the same N as the thing it is normalising.

One number hides fidelity and coverage. A distribution can be wrong by being too narrow (everything realistic, nothing varied) or too broad. Precision and recall over feature manifolds separate the two — precision reads as fidelity, recall as coverage — and reporting the distance alone conceals which failure you have.

Pooling tiles discards slide-level composition. An attention-pooling slide encoder registered per-slide composition that a pooled patch distance could not see, raising the distance by roughly 320-fold on otherwise matched cohorts. If the question is about slides rather than tiles, a tile-pooled distance may be answering a different one.

Why it matters for my work

It turns a qualitative claim in this wiki into a measurable one. Foundation Models in Pathology records that the evidence for foundation-model feature spaces shifting under staining variation “is t-SNE inspection rather than a quantitative shift metric”. This is that metric — with a normalisation that makes it comparable across encoders, and a floor that says what no-shift looks like. The open question there about whether the shift Khan et al. observed actually matters becomes answerable in a defined unit rather than by eye.

The tooling is already on site. Foundation Model Cache holds UNI2-h, Virchow2, Phikon-v2, Prov-GigaPath and CONCH; PinkKidney has four of those wired into one extraction script with features cached per slide. Five of the six encoders in the source paper are extractable here with no new engineering, so computing a local floor is an afternoon rather than a project.

It is a second discriminator for encoder choice. PinkKidney’s encoder comparison saturated on accuracy and was separated instead by representation geometry — a signal-to-batch ratio. The sensitive/invariant axis is a different statistic reaching a compatible conclusion: encoders differ in how much nuisance structure survives in their feature space, and that difference is the thing worth measuring.

It gives Labquality EQA Staining Dataset a use. One block, 66 laboratories, one scanner is a purpose-built stain-variation cohort, and this is a purpose-built stain-variation measurement. The combination would answer, in a number, how far apart 66 real laboratories are in a foundation model’s eyes.

How it connects

Foundation Models in Pathology — the encoders this is computed in, and the page whose open question about a quantitative shift metric this answers.

Scanner and Stain Variability — the phenomenon this measures. That page describes the mechanism; this supplies the instrument, and a floor to read it against.

PinkKidney — the cohort and cached features that make a local floor cheap, and the project whose signal-to-batch statistic is the closest thing here to a prior attempt at the same question.

Foundation Model Cache — where five of the six encoders already sit.

Labquality EQA Staining Dataset — the controlled stain-variation cohort this metric was made for.

Coverage Bias — the same discipline one level up: a number computed over a population is uninterpretable until you say which population, and here the floor is what defines it.

Model Abstention — a sibling reminder that an evaluation metric can be arithmetically fine and still describe something other than what the reader assumes.

Bhat 2026 — HistoFID: Calibrating Fréchet Distance Across Pathology Foundation Models — the paper this page is written from: the thirty-fold spread across encoders, the ratio-to-floor normalisation, and the sensitive/invariant split.

Open questions

What is the within-cohort floor on this group’s own data? Nothing here records one, for any encoder or any cohort. It is the prerequisite for every other use on this page, and it is computable from features already cached.
Would a sensitive encoder detect PinkKidney’s three-scanner batch structure more cleanly than the silhouette statistics already used? Same cohort, same features, different statistic.
Does the ~320-fold tile-versus-slide gap mean pooled patch distances systematically understate cohort differences? If so it matters for exactly the cohort-comparison use this group would have. [unverified]
Is this worth adopting at all, or is it a metric in search of a question here? The honest case for it is drift detection on Memorial WSI Share as it grows — but nothing records whether anyone wants that measured.