Grisi 2026 — CRoMa, a Distributional Robustness Margin for Pathology Foundation Models

A per-sample, scale-free margin asking whether an embedding sits closer to same-biology tiles from another centre than to different-biology tiles from its own — defined for every sample, so robustness becomes a distribution with a tail; measured on 20 tile encoders and 4 slide encoders, no model is uniformly robust, and the margin predicts downstream shortcut learning before any probe is trained.
Author

Grisi C, van der Laak J, Litjens G — Department of Pathology, Radboud University Medical Center, Nijmegen (as printed)

A per-sample, scale-free margin asking whether an embedding sits closer to same-biology tiles from another centre than to different-biology tiles from its own — defined for every sample, so robustness becomes a distribution with a tail; measured on 20 tile encoders and 4 slide encoders, no model is uniformly robust, and the margin predicts downstream shortcut learning before any probe is trained.

Read status: full text (preprint v4) with supplement, plus the code. Read as a pair with Kömen 2026 — PathoROB: Towards Robust Foundation Models for Digital Pathology, the Nature Communications paper whose Robustness Index this one critiques and whose benchmarks it reuses. Read them in that order.

The problem, stated in the paper’s own terms

Self-supervised pretraining rewards any recurring structure, so a pathology encoder learns the acquisition centre — stain, scanner, section thickness — alongside the biology. That is harmless until centre correlates with label in a downstream training set, at which point a linear probe can pass its exam by reading the scanner. PathoROB’s Robustness Index (RI) measures the tendency by typing each tile’s k nearest neighbours: SO neighbours share biology but come from another centre (good — biology organises the space); OS neighbours share the centre but differ in biology (bad — the centre organises it). RI is the pooled fraction SO / (SO + OS).

What is wrong with RI, precisely

Three structural faults, each demonstrated rather than asserted:

  1. It counts, so it discards geometry. Two encoders with identical SO/OS counts inside k can place those neighbours at very different distances. The paper builds a minimal distance-weighted control, MaRI, and finds it barely moves the score (|Δ| ≤ 0.07 on Camelyon, rank ρ = 0.99) — which shows the limiting factor is not the missing distances but the next two faults.
  2. It has no per-sample form. RI exists only as a pooled aggregate, so the distribution of robustness across a cohort — and any vulnerable subset — is invisible.
  3. It silently evaluates different models on different samples. A tile contributes only if its fixed-k neighbourhood happens to contain a typed neighbour. On Camelyon at the shared operating point k = 11, 10–46% of tiles contribute, fewer than a quarter for over half the encoders, because the first typed neighbour occurs at a median rank of ~149 among 20,400 candidates. Supplementary A.4 explains why: k is chosen to maximise biological k-NN accuracy, an objective dominated by same-biology-same-centre neighbours that the metric then discards, so the window is almost by construction too small to reach the neighbours it scores.

CRoMa

For each sample, take the mean cosine distance to its m nearest SO neighbours and to its m nearest OS neighbours (m = 5 throughout, insensitive across 1–20), excluding tiles from the same slide, and form the signed ratio

CRoMa = (d_OS − d_SO) / (d_OS + d_SO)  ∈ (−1, 1).

Positive means biology dominates the local geometry; negative means the centre does. It is defined for every sample by construction, needs no per-model k, and — the property this wiki cares about most — is invariant to global rescaling of an encoder’s distances, so an encoder is not credited for merely spreading its embeddings. Supplementary A.5 shows the margin is the angular offset from the diagonal in the (d_SO, d_OS) plane: a median CRoMa of 0.7 means the nearest same-centre distractors are ~5.7× farther than the nearest cross-centre matches. The nuisance this removes is real — on the slide benchmark, four encoders’ typed-distance radii span a factor of 80.

Three summaries replace the single score: the median (typical case), F(0) (fraction of samples with margin ≤ 0), and LTM₁₀ (mean of the worst decile).

The four buried fields

Benchmarks PathoROB’s three tile sets — Camelyon (breast LN tumour/normal, 2 centres, 20,400 tiles), TCGA-4×4 (4 cancers × 4 centres, 5,760), Tolkach-ESCA (6 oesophageal classes × 3 cohorts, 9,000) — plus PCaBiop, new here: 1,000 PANDA prostate biopsies balanced across benign/cancer and Radboud/Karolinska, and PCaBiop-ISUP (3,000 biopsies, six ISUP groups × two centres)
Scanner / magnification Not restated here — inherited from PathoROB’s protocol and the PANDA release; the paper treats centre as the bundled confounder and says so
Models 20 tile encoders (Virchow/Virchow2, UNI/UNI2-h, CONCH/CONCHv1.5, H-optimus-0/1, H0-mini, Prov-GigaPath, Midnight-12k, Phikon/Phikon-v2, Hibou-B/L, GPFM, MUSK, mSTAR, Prost40M, GenBio-PathFM) + DINOv2-B natural-image control; 4 slide encoders (PRISM, TITAN, Prov-GigaPath, MOOZY). Disclosed pretraining corpora tabulated; 9 are TCGA-exposed, GPFM is Camelyon-exposed
Validation Representation metrics on the full balanced cohorts; downstream via PathoROB’s confounder-biased linear-probe sweep (Cramér’s V from 0 to 1, 20 resampling runs, regularisation chosen on validation), with in-domain and held-out-centre test sets
External validation Of the metric: rankings replicate across three tile benchmarks (pairwise ρ 0.88–0.92) and the downstream association holds on all three. Of the models: PAR (a Middle-Eastern prostate cohort) as the slide-level OOD set

Results that matter

  • Every encoder encodes the centre near-perfectly. Biological k-NN accuracy 0.93–0.99 on Camelyon, centre k-NN accuracy 0.91–1.00. What separates encoders is not whether the centre is present but which signal wins when they compete — and centre decodability rank-predicts the median margin at ρ = −0.94. Biology accuracy predicts it only weakly (ρ = 0.56): CONCH, CONCHv1.5, Hibou-B and Hibou-L all recover biology at ~0.97 and span nearly the whole CRoMa range.
  • Median CRoMa on Camelyon runs from +0.20 (Virchow2, CONCH) to −0.44 (Hibou-L); 13 of 20 encoders are biology-dominant at the median, 7 confounder-dominant. UNI sits at −0.03, Phikon/Phikon-v2 at −0.20/−0.21.
  • No encoder is uniformly robust. Every one has LTM₁₀ < 0 on every benchmark. F(0) ranges from 12.9% (Virchow2) to 99.3% (Hibou-L) on Camelyon. Medians and tails are non-redundant: CONCH matches Virchow2 at the median (0.20) but has more non-positive margins (22.5% vs 12.9%) and a worse tail (−0.20 vs −0.11). Only four encoders are undominated in the median–tail plane across the three tile benchmarks — CONCH, GenBio-PathFM, CONCHv1.5, H-optimus-1 — and none leads on both axes. GenBio-PathFM is on the Pareto frontier in all three.
  • Rankings are stable across benchmarks (ρ ≥ 0.88), but the distributions shift: TCGA-4×4 and Tolkach-ESCA medians are higher (only one and zero confounder-dominant encoders), yet the worst decile stays negative everywhere.
  • Midnight-12k’s TCGA lead is partly overlap — pretrained only on TCGA, it gains 2.5× in typed-distance odds on TCGA cases against ~1.3× for everyone else, yet still leads when TCGA cases are removed; overlap amplifies rather than creates the advantage.
  • Slide encoders are worse. On PCaBiop only PRISM is biology-dominant (0.26); with ISUP-grade labels every slide encoder goes negative (−0.09 to −0.47). MOOZY has the best biological accuracy and a negative margin — accuracy and robustness are different things.
  • The margin predicts shortcut susceptibility. Higher median CRoMa ↔︎ smaller degradation under confounder-biased training: ρ = 0.88 / 0.93 / 0.93 in-domain on the three benchmarks, 0.64–0.85 on held-out centres. The paper’s own reduction, nIPD, normalises degradation by the above-chance headroom available to lose, fixing an APD artefact where a fall from 0.60 to chance and a fall from 0.95 to 0.79 score identically.

Reading it critically

  1. All three metrics rank models the same way (ρ 0.93–0.95), and all three largely recapitulate centre decodability (ρ ≈ −0.94). The paper says this itself. So as a model-ranking statistic CRoMa adds little over a k-NN centre probe; its case rests entirely on the per-sample distribution and the tail — which is a real contribution, but a narrower one than “a better metric” suggests.
  2. The predictive claim is an association across 20 encoders. A Spearman ρ over twenty points, with several encoders sharing training recipes and corpora, is suggestive rather than established; the slide-level version rests on four points and the paper labels it descriptive.
  3. Confounder = centre, and centre bundles everything. Stated as a limitation. A group wanting to know whether scanner or stain batch drives its embeddings gets a single entangled answer unless it labels those factors separately — which the construction permits but the paper never tests.
  4. The diagnostic use for a single model is proposed, not demonstrated — “the most direct extension of this work,” in their words. Everything here compares models; nothing yet says what to do with one model’s low-margin samples.
  5. Pretraining overlap is handled honestly but not eliminated — nine TCGA-exposed encoders, one Camelyon-exposed, MOOZY trained on PANDA; the flags are in the tables.
  6. Preprint, v4. Single first author did all curation, experiments and writing (contributions statement). Competing interests: van der Laak and Litjens declare Philips, ContextVision, Sectra, Novartis relationships outside the work.
  7. A benchmark is not the group’s slides. Balanced cohorts with known centres are exactly what makes the metric computable and exactly what a routine archive lacks; the paper is clear the method needs “a compact, deliberately structured multi-institutional cohort.”

Why it earns space here

  1. It is the missing half of Feature-Space Distance. That page argues an embedding distance “belongs to the encoder until you divide it by that encoder’s own floor.” CRoMa is that division done properly — a ratio of typed distances that cancels the encoder’s scale by construction — and it says what the floor should be measured against: same-biology cross-confounder neighbours.
  2. It gives Encoder Benchmarking a second axis. Every benchmark filed here ranks encoders by task accuracy; this shows accuracy and robustness barely correlate and that the choice is a Pareto problem with a tail. The group’s own encoder comparison has no robustness column.
  3. It is runnable on the group’s cache today. The package takes embeddings and a confounder column; Foundation Model Cache holds the encoders and PinkKidney has a cohort with three scanners recorded per slide. That is the experiment, and it needs no new data.
  4. It corroborates Scanner and Stain Variability with numbers: the centre is decodable from every encoder’s features at 0.91–1.00, and it is the dominance, not the presence, that matters.

Related: Kömen 2026 — PathoROB: Towards Robust Foundation Models for Digital Pathology — the paper it answers; read first. Related: [croma] — the package, evaluated as a tool on its own page. Related: PathoROB — the three tile benchmarks, now a dataset page. Related: Feature-Space Distance, Encoder Benchmarking, Scanner and Stain Variability, Foundation Models in Pathology, External Validation — each updated from this reading; see those pages for the specific claims carried over.