PathoROB

A public robustness benchmark — 99,392 patches from Camelyon, TCGA and Tolkach ESCA, balanced so every biological class appears equally at every medical centre — which is the one thing a routine archive never is, and the reason no encoder’s centre-robustness has ever been measured on this group’s own material.

A public robustness benchmark — 99,392 patches from Camelyon, TCGA and Tolkach ESCA, balanced so every biological class appears equally at every medical centre — which is the one thing a routine archive never is, and the reason no encoder’s centre-robustness has ever been measured on this group’s own material.

What it is

Released with Kömen 2026 — PathoROB: Towards Robust Foundation Models for Digital Pathology and reused, with one addition, by Grisi 2026 — CRoMa, a Distributional Robustness Margin for Pathology Foundation Models. Four datasets from three public sources, each subsampled so that every combination of biological class and contributing medical centre holds the same number of patches, with cases matched on known demographics across centres:

Dataset Biology Centres Patches (balanced cohort) OOD
Camelyon breast lymph node: tumour / normal (2) RUMC, UMCU (2) 20,400 (5,100 per cell) 3 further centres
TCGA 4×4 BRCA / COAD / LUAD / LUSC (4) 4 5,760 (360 per cell) 4 further centres
TCGA 2×2 94 quartets: 2 classes × 2 centres each, spanning 21 cancer types and 26 centres — 112,800 tile occurrences —
Tolkach ESCA six oesophageal tissue compartments UKK, WNS, CHA (3) 9,000 (500 per cell) 1 cohort

Totals per the paper: 99,392 patches, 28 biological classes, 34 medical centres. The Grisi paper adds a slide-level companion, PCaBiop — 1,000 PANDA prostate biopsies balanced across benign/cancer and Radboud/Karolinska, plus PCaBiop-ISUP (3,000 biopsies, six ISUP groups × two centres) — with the PAR Middle-Eastern cohort as an external set (huggingface.co/datasets/waticlems/pcabiop).

Access. Processed data on Hugging Face under the bifold-pathomics/pathorob collection; code at github.com/bifold-pathomics/PathoROB (BSD-3-Clause, Zenodo 10.5281/zenodo.18961604). The underlying sources set the licences: Camelyon CC0; TCGA-UT CC BY-NC-SA 4.0; Tolkach ESCA a custom non-commercial licence. Recorded as declared.

Why it matters for my work

Because the balance is the point, and the group has nothing like it. A centre-robustness metric needs, for every sample, neighbours that share its biology but not its centre and neighbours that share its centre but not its biology. A routine archive — one hospital, one or two scanners, cases arriving as they arrive — cannot supply the first kind at all. So the question “does our encoder read the scanner?” is not merely unanswered here; it is unaskable on the material to hand without deliberately constructing a cohort of this shape.

The nearest local candidate is PinkKidney: 356 slides, 44 patients, three scanners, with vendor and µm/px recorded per slide. That is a scanner-as-confounder design waiting to be balanced — the same tissue classes across the same three scanners — and it is the one place in the estate where a PathoROB-style robustness measurement could be run on the group’s own slides rather than on someone else’s benchmark.

Two secondary uses. As an external check on any encoder choice: Foundation Model Cache holds most of the twenty encoders both papers rank, and the benchmark is small enough to embed on one workstation. And as the reference point for what “robust” costs: even the best encoders leave 13% of Camelyon tiles in confounder-dominated neighbourhoods, and every encoder’s worst decile is negative on every dataset.

How it connects

Kömen 2026 — PathoROB: Towards Robust Foundation Models for Digital Pathology — the release paper, and the numbers above.

Grisi 2026 — CRoMa, a Distributional Robustness Margin for Pathology Foundation Models — the follow-up that reuses the tile sets, adds PCaBiop, and replaces the benchmark’s own metric.

croma — the package that computes robustness on any embeddings with a confounder column, including on this benchmark.

Scanner and Stain Variability — the phenomenon this benchmark is built to measure.

Encoder Benchmarking — the accuracy benchmarks that page records have no robustness axis; this is the axis.

TCGA — TCGA-UT patches are one of the three sources, and the TCGA-pretrained encoders’ overlap with them is flagged in both papers.

PinkKidney — the local cohort that could be shaped into a scanner-confounder version of this.

Open questions

  • Has anyone downloaded it? No — nothing here has been run; contents are as the papers and the Hugging Face collection describe them. [unverified] beyond that.
  • Could PinkKidney’s three-scanner cohort be balanced into a local robustness benchmark, and how many slides per scanner × class would survive the balancing? Nobody has counted.
  • The benchmark’s confounder is “medical centre,” which bundles scanner, stain and preparation. A local version could separate scanner from the rest — the metric accepts any labelled factor — and that would be a genuinely new measurement rather than a replication.