PinkKidney

Chromophobe versus oncocytoma classification from H&E whole-slide images, using foundation-model tile features and multiple-instance learning — and, as of this review, the project in this wiki with by far the most thorough confound audit and still no external cohort.

Purpose

Separate chromophobe renal cell carcinoma from oncocytoma on H&E slides. That is a deliberately hard target: the two are the classic difficult pair among eosinophilic renal tumours, and the distinction carries real consequence, since oncocytoma is benign. LOT and EVT tiles exist in the project but the primary task is the binary one.

This supersedes the previous version of this page, which recorded Purpose, Data and Methods as “not documented” because it had been written from the Quarto chapter rather than the code. All three are now documented, from a source-level read of the working tree.

Data used

356 slides, 44 patients, single institution. Read from cohort_metadata/cohort_metadata.csv and metadata/splits.csv rather than from any prose:

Field	Value
Patients	44 — 26 chromophobe, 18 oncocytoma
Slides	356 with metadata (343 also carry HistoQC descriptors; 297 of those have a class label)
Tiles	392,428, 236,536 chromophobe / 155,892 oncocytoma
Institution	One. A single batch_id, and institution takes one value
External cohort	None

Slides are identified by caseNN-style pseudonyms throughout; no accession numbers appear in the image names, the tile paths or the split file. See PembeBobrek for the one place where that discipline breaks down, which is a separate repository.

Three scanners, and scanner is perfectly collinear with magnification — this is the single most important design fact and it constrains every analysis below:

Scanner vendor	Slides	Estimated magnification	Pixel size
3DHistech	175	20×	~0.504 µm
Leica Aperio	95	40×	~0.252–0.253 µm
Hamamatsu	86	38×	~0.263 µm

Because each vendor appears at exactly one magnification, the scanner effect and the magnification effect cannot be separated in this cohort by any analysis. Anything attributed to one is equally attributable to the other. scanner_model is blank for all 356 rows, so only vendor is available.

Class is not strongly confounded with scanner, which is worth stating because it is the first thing to suspect: chromophobe patients split 13/7/9 across 3DHistech/Hamamatsu/Aperio and oncocytoma 12/2/4. Both classes appear on all three scanners. Oncocytoma is somewhat concentrated on 3DHistech (12 of 18) but nothing approaching separation.

Methods

QuPath annotation → Groovy tile export → foundation-model feature extraction → MIL training, with phenotype clustering alongside. Verified against python/ and scripts/ rather than the README.

• Tiling. 256×256 tiles at 10× effective magnification. The current Groovy export computes downsample = nativeMagnification / 10.0 per slide, so mixed native magnifications still yield a common effective magnification. This replaced a hardcoded downsample=4 — see the open issue below.
• Splits. create_splits.py uses patient-level StratifiedGroupKFold, 5 folds. Verified empirically, not taken on trust: across all 392,428 rows of splits.csv, zero patients appear in more than one fold, and no patient carries more than one class. Fold sizes range from 50,332 to 100,337 tiles because grouping is by patient and tile counts per patient vary widely.
• Encoders. UNI, UNI2-h, Virchow2, Phikon-v2, H-optimus-0, CONCH and Midnight are supported; results exist for UNI, DINOv2-large, ResNet50-ImageNet and ViT-B/16-ImageNet, plus six UNI preprocessing variants. Features cached one .pt per slide, resumable.
• MIL. ABMIL and CLAM-MB. Patient-level aggregation by majority vote / mean probability.
• Statistics, which are unusually complete for a project at this stage. bootstrap_roc.py, delong_test.py for encoder comparisons, stratified_auc.py, calibration_analysis.py, and train_mil_label_shuffle.py as a negative control.
• Confound tooling. batch_effect_prescreen.py, combat_correction.py, color_effect_analysis.py, magnification_comparison.py, stain_comparison.py, plus HistoQC and GrandQC wrappers for tile QC.
• Interpretation. Leiden phenotype clustering with UMAP stored as AnnData, and attention and phenotype maps exported back to QuPath as GeoJSON overlays.

Current state / open questions

The headline result, and why it survives scrutiny

UNI features reach AUC 1.000 with zero variance across all five folds, at slide and patient level, on both ABMIL and CLAM-MB. ImageNet-pretrained and general-purpose encoders do not: DINOv2-large 0.971, ViT-B/16 0.945–0.970, ResNet50 0.908–0.910.

A perfect AUC normally means a leak, and the first duty is to look for one. Here the project’s own controls make that reading hard to sustain, and they are more complete than in either published paper reviewed in this wiki this month:

• Label-shuffle control passes. Shuffled labels converge to AUC 0.51, 0.53 and 0.61 across three seeds — chance. If the pipeline leaked labels, this would not happen.
• Colour is not the mechanism. UNI in grayscale still scores 1.000. So does grayscale + z-score, Macenko + grayscale, and Macenko + grayscale + z-score.
• Colour-quartile stratification passes. Splitting slides into luma quartiles and scoring within each, AUC stays at 1.000 / 1.000 / 0.999 / 1.000. A brightness shortcut would collapse inside a brightness stratum. It does not.
• ComBat behaves as it should. On UNI features the magnification silhouette drops from 0.097 to −0.014 while the class silhouette rises from 0.191 to 0.205, with a per-scanner AUC drop of 0.0. Batch structure removed, class structure preserved.
• The batch-effect detector itself is validated. synthetic_two_site_test.py injects a known effect into 50 of 91 descriptors at 0.5 SD and confirms recovery at kNN accuracy 0.994, ARI 0.977. A confound detector with a positive control is rare and worth keeping.

There is a genuine tension in the evidence, and it should be recorded rather than resolved: a colour shortcut does exist. Logistic regression on 39 HistoQC colour descriptors alone gives a pooled AUC of 0.700 (95% CI 0.634–0.765) for the class task, and tile-level mean RGB differs by class at p ≈ 1e-10 to 1e-12. So the shortcut is available — and the grayscale result says the model does not need it. Both are true.

What actually limits this result

Four things, none of which the controls above address.

1. n = 44 patients. This is the binding constraint. The honest statement is “no ranking errors were observed in 44 patients”, not “AUC = 1.0” — a perfect AUC at 26 versus 18 has a confidence interval reaching well below 0.9. The 392,428 tiles are not the sample size; the patients are.

2. The metric is saturated, so the preprocessing experiment cannot answer its own question. All six UNI variants — raw, ComBat, grayscale, grayscale+z-score, Macenko+grayscale, Macenko+grayscale+z-score — score exactly 1.0000. That means the comparison between stain normalisation strategies, which is what those runs were for, has no resolution left. Ranking Macenko against grayscale against ComBat is impossible at this ceiling. Answering that question needs a harder setting: a weaker encoder where the variants can separate (ResNet50 already shows movement — 0.908 raw versus 0.891 ComBat), a held-out scanner, or a metric with more headroom than AUC, such as calibration error or tile-level agreement.

3. Scanner and magnification are collinear, as above. No amount of analysis on this cohort separates them.

4. No external validation, and one institution. This is the project’s largest gap and it is the one the internal controls cannot substitute for. See External Validation.

The magnification confound, still open

CLAUDE.md records it as an active issue and the code agrees. The older tile export used a hardcoded downsample=4, which on a 20× slide yields 5× tiles and on a 40× slide yields 10× — so tiles clustered by scanner rather than by morphology. The Groovy script now computes the downsample per slide, and the Python side is fixed, but re-tiling from QuPath is still pending. Until it is done, the tiles underlying the results above were produced by the old path. After re-export the whole chain needs rerunning and the AUCs comparing against the current ones.

The scale of what is being corrected for is measurable: from 91 HistoQC slide descriptors, magnification is recoverable at kNN accuracy 0.968 (per-class recall 0.994 / 0.929 / 0.952, k-means ARI 0.516), with 67% of descriptors significant after Bonferroni and 47 showing large eta-squared. At the raw-descriptor level the batch effect overwhelms the class signal — silhouette 0.121 for magnification against 0.016 for class, a signal-to-batch ratio of 0.13.

That ratio inverts in UNI’s feature space, which is the most interesting single number in the project:

Encoder	Class silhouette	Magnification silhouette	Signal-to-batch
UNI	0.191	0.097	1.787
ResNet50-ImageNet	0.064	0.070	0.800
DINOv2-large	0.072	0.082	0.782
ViT-B/16-ImageNet	0.059	0.139	0.393

UNI is the only encoder tested whose class signal exceeds its batch signal. That is a concrete, locally measured argument for pathology-specific pretraining, and it explains the AUC gap better than the AUCs themselves do.

Smaller things

• 138 files are uncommitted in the working tree at review time, so the committed state and the state that produced the newest results may differ.
• 46 of the 343 HistoQC-processed slides have no class label, and 356 appear in cohort_metadata.csv against 343 in HistoQC — the shortfall is not explained anywhere.
• No test suite, linter or build step; validation is by re-running a stage and inspecting outputs. Reasonable for research code, worth knowing before anyone depends on it.

Open questions for the next review: whether re-tiling changes anything; whether a second institution or a held-out scanner can be obtained, which would convert this from a strong internal result into an externally supported one; whether the 44 patients were consecutive or selected for diagnostic clarity, which is not recorded anywhere and would materially affect how the AUC reads [unverified]; and whether the preprocessing comparison should be rerun on a weaker encoder where it can actually resolve.

Related: PembeBobrek — the Turkish guide repository and deploy chain for this project, which also carries a data-exposure finding that needs attention.

Related: Foundation Models in Pathology — this project supplies the wiki’s first locally measured comparison of a pathology foundation model against ImageNet encoders, on the batch robustness axis rather than accuracy alone.

Related: Scanner and Stain Variability — a three-scanner cohort with the effect quantified rather than assumed, and the collinearity that limits what can be concluded from it.

Related: Stain Normalisation — six normalisation variants were run here, and the saturated metric is why they cannot be ranked.

Related: Patch Extraction — the downsample bug is a worked example of why tiling parameters must be recorded in µm/px or effective magnification, never as a fixed level index.

Related: External Validation — the missing piece, and the reason the internal controls above cannot close the argument.

Related: Weakly Supervised Learning — ABMIL and CLAM-MB are the MIL methods that page describes, applied here to a slide-level label.

Derived from: working tree read 2026-07-26 at commit eff18f0 (2026-05-06) — CLAUDE.md, python/*.py, scripts/*.groovy, classifiers/classes.json, and the committed outputs cohort_metadata/cohort_metadata.csv, metadata/splits.csv, results/*/{abmil,clam_mb}_results.json, results/encoder_comparison/comparison.csv, results/uni/abmil_shuffled/*/summary.json, color_analysis/{report.md,color_only_lr_auc.json,color_quartile_stratified_auc.csv}, batch_effect_prescreen/{scanner_clustering.json,synthetic_two_site_test.json,combat/combat_verdicts.json}, magnification_analysis/comparison/summary.csv and stain_analysis/comparison/summary.csv.