Kömen 2026 — PathoROB: Towards Robust Foundation Models for Digital Pathology

A public benchmark and a Robustness Index showing that all 20 pathology foundation models encode the medical centre — decodable at 88–98% — and that when centre correlates with label a linear probe on Virchow2 still misses 19–99% of tumour patches from the wrong hospital; stain normalisation helps, ComBat raises the index but harms predictions, and nothing eliminates it.

A public benchmark and a Robustness Index showing that all 20 pathology foundation models encode the medical centre — decodable at 88–98% — and that when centre correlates with label a linear probe on Virchow2 still misses 19–99% of tumour patches from the wrong hospital; stain normalisation helps, ComBat raises the index but harms predictions, and nothing eliminates it.

Read status: full text, supplement unread. Read as the first half of a pair with Grisi 2026 — CRoMa, a Distributional Robustness Margin for Pathology Foundation Models, which dissects this paper’s central metric. This note records what PathoROB establishes; the critique of the Robustness Index is on the other note.

What they built

PathoROB: four balanced multi-centre datasets from three public sources — Camelyon (breast lymph node, tumour/normal, 2 centres + 3 OOD), TCGA-UT as TCGA 4×4 (BRCA/COAD/LUAD/LUSC × 4 centres + 4 OOD) and TCGA 2×2 (94 class-pair × centre-pair quartets), and Tolkach ESCA (six oesophageal tissue compartments × 3 centres + 1 OOD cohort) — 99,392 patches, 28 biological classes, 34 medical centres, each cell of biology × centre equally populated, with demographics matched across centres where known. Twenty encoders from Ciga (2022, 11M parameters) to Virchow2, H-optimus-0, UNI2-h, Atlas and RudolfV (the last two proprietary, Aignostics).

Three metrics. The Robustness Index (local): type each patch’s k nearest neighbours as same-biology/other-centre (SO) or other-biology/same-centre (OS) and report SO/(SO+OS), with k chosen to maximise biological k-NN accuracy. Average Performance Drop (output-level): train linear probes on splits where centre and label are increasingly correlated (Cramér’s V 0 → 1), same size and marginals, and measure the fall on in-domain and held-out-centre test sets. Clustering score (global): ARI of k-means clusters against biology minus ARI against centre. Plus patch-retrieval accuracy in centre-heterogeneous databases.

What it found

• The centre is recoverable from every encoder: mean centre prediction accuracy 88–98% by linear probe across three datasets, “a characteristic that is medically useless and potentially harmful.” t-SNE of Phikon-v2 is organised by hospital first; Virchow2 by biology first with hospital second.
• RI ranges 0.446–0.861 averaged over datasets — i.e. 14–55% of local neighbourhoods are defined by the hospital rather than the tissue — and no model is fully robust at any k. Only Virchow2 and Atlas are Pareto-optimal on accuracy versus robustness. Accuracy and RI correlate only modestly (ρ = 0.55).
• Vision–language models are more robust than their accuracy predicts (CONCH, CONCHv1.5, MUSK), which the authors read as caption supervision suppressing non-biological features. Among SSL models, log pretraining WSI count correlates with RI (ρ = 0.69).
• Clever Hans made concrete. At full confounding (V = 1) on Camelyon, tumour-detection accuracy falls from >92% to 53–87% regardless of encoder; on TCGA and Tolkach the drops are −1% to −25% and 0 to −14%. On the in-domain test set, 28% (Atlas, Virchow2) to 94% (Phikon-v2) of RUMC normal patches are called tumour, and 19% (Virchow) to 99% (HIPT) of UMCU tumour patches are missed — Fig. 4 shows a Virchow2 probe calling an unequivocal tumour patch normal because of where it was scanned. On held-out slides the less robust encoders (UNI2-h, Phikon-v2) fail to find the tumour areas; Virchow2 and CONCH still do. APD correlates with RI at ρ = 0.90 (in-domain) and 0.80 (out-of-domain).
• Fine-tuning makes it worse: full and parameter-efficient fine-tuning “exacerbated the issue of Clever Hans learning” (Supp. Notes 7.1–7.2, unread).
• Slide level too. Attention-MIL on four tasks: Camelyon slide AUC falls to 0.47–0.60 at full confounding even for Virchow2 and CONCHv1.5 — worse than patch level; NSCLC subtyping and BRCA HRD prediction degrade consistently; and for BRCA survival, adding three hospitals with different survival distributions lowered C-index for four of six encoders (0.576 → 0.556) despite 67% more training data. That is centre confounding of an outcome label, and it is the failure mode this wiki’s survival pages describe abstractly.
• Clustering and retrieval inherit it: Phikon-v2’s k-means clusters are hospitals, not tissue (clustering score −0.99 on Camelyon); TCGA LUSC-vs-LUAD is unclusterable for every encoder.
• Robustification without retraining. Reinhard stain normalisation raises RI by +19.1% on average and improves probe generalisation by +1.11 points (max +3.28), most under strong confounding, and beats Macenko. ComBat raises RI further (+28.2%, up to +70% for Phikon) and fixes clustering (Phikon-v2 Camelyon −0.99 → +0.61) — yet did not consistently improve, and sometimes worsened, downstream predictions, because biology and centre share linear directions in the embedding and removing one removes the other. DANN helped 12 of 20 encoders, weakly. Nothing eliminates the drop.

Reading it critically

1. The benchmark is the paper’s real product, and the RI is its weakest part. The metric’s fixed-k, pooled, support-dependent construction is taken apart on Grisi 2026 — CRoMa, a Distributional Robustness Margin for Pathology Foundation Models — on Camelyon only 10–46% of tiles even contribute to RI — and the critique is not answered here because it postdates the paper. The findings survive it: all three robustness metrics rank the encoders alike.
2. Twenty encoders, twenty points. Every correlation in the paper (ρ = 0.55, 0.69, 0.80, 0.90) is a Spearman over n = 20 models, with permutation p-values and Fisher intervals reported honestly; the log-WSI scaling result in particular rests on heterogeneous recipes.
3. Confounding is engineered, then read as real-world. The V = 1 splits are a worst case by design. The paper’s own argument that such correlations “are frequently found in histopathology datasets” for rare diseases is plausible and unmeasured here.
4. Balanced subsampling with matched demographics is what makes the index computable — and what makes it unlike any archive. The clean “centre = technical” attribution depends on it; the authors concede that centre-specific biology could be mistaken for artefact under robustification, and that “more complex scenarios reflecting general clinical scenarios” are needed.
5. Supplement unread here, and it carries the fine-tuning result, the feature-space analysis showing centre information “along the directions of greatest variance,” and the ComBat failure analysis.
6. Competing interests are material: four authors are Aignostics employees and three are its co-founders; two of the twenty encoders (Atlas, RudolfV) are Aignostics models, and Atlas is one of the two named Pareto-optimal encoders. Declared in full. One author was at kaiko.ai (Kaiko ViT-B/8 is also evaluated).
7. Two datasets carry non-commercial licences (TCGA-UT CC BY-NC-SA; Tolkach custom non-commercial) — recorded, not investigated.

Why it earns space here

1. It puts numbers on Scanner and Stain Variability’s central claim — 88–98% centre decodability from every encoder — and shows the harm in the form that page’s readers would recognise: an obvious tumour called benign because of the hospital.
2. It tests the fixes this wiki already recommends. Stain Normalisation (Reinhard helps, modestly, most when it matters) and ComBat, which PinkKidney uses for batch correction and which here raises the robustness score while degrading predictions — a specific warning for that project.
3. It is the first filed evidence that fine-tuning increases shortcut learning, which bears directly on the Liver Tru-Cut — Tumour Origin AI comparison of frozen versus adapted encoders.
4. The benchmark is public and small enough to run, and the group has never measured centre or scanner decodability from any encoder on its own material.

Related: Grisi 2026 — CRoMa, a Distributional Robustness Margin for Pathology Foundation Models — the successor metric and the critique. Related: PathoROB — the benchmark as a dataset page, with access and licences. Related: croma — a reimplementation of RI alongside the newer metrics, runnable locally. Related: Scanner and Stain Variability, Stain Normalisation, Foundation Models in Pathology, Encoder Benchmarking, External Validation — updated from this reading.