Foundation Model Cache
The local model library on the J: drive — around 370 GB across 115 model directories mirroring HuggingFace paths — and, more usefully, what each class of model in it could do for a project already running here.
What it is
A drive-backed library of pretrained models laid out so that J:\<org>\<model> corresponds exactly to huggingface.co/<org>/<model>, which makes every directory resolvable to its model card and paper without guessing. Read directly from the drive on 2026-07-28: 115 model directories, roughly 370 GB of weights. The drive holds ~809 GB in total; the rest is tooling — CellProfiler, cuDNN, TensorRT, FastPathology — not models.
It is close to a complete sweep of the published pathology model landscape, and it is the group’s largest computational asset. The only prior record of it is the local_models.txt directory dump noted on Drafts Helpers, which lists names but says nothing about what anything does.
Why it matters for my work
An entire class of model here is unused, and it replaces a step the group hand-builds
Every image pipeline in this wiki stops at the same place: run tiles through an encoder, then train an attention-MIL head to pool them into a slide-level answer — the pattern in Weakly Supervised Learning. Six models in this cache do that pooling as a pretrained step, so the aggregator is learned from hundreds of thousands of slides instead of from the few hundred in a local cohort:
| Model | What it is |
|---|---|
| TITAN (Mahmood Lab) | Whole-slide encoder pretrained on 335,645 WSIs with vision-language alignment |
| PRISM (Paige) | Slide encoder over Virchow tile embeddings, pretrained against clinical report text — and generative, producing a text description of the slide |
| CHIEF (Harvard DBMI) | Slide-level foundation model for cancer diagnosis and prognosis |
| Prov-GigaPath | Whole-slide model trained on real-world hospital data |
| mSTAR | Multimodal slide-level model |
| EAGLE | H&E computational-biomarker model |
This is the most actionable thing on the drive. The slide-level aggregator is the part of the group’s pipeline trained on the least data, so it is the part most likely to be limiting — and there are six pretrained replacements sitting locally. PinkKidney already caches tile features per slide, which is exactly the input TITAN and PRISM consume, so testing one is a head-swap rather than a new pipeline.
And the published literature has not answered it, which is worth knowing before assuming it has. The 2026 benchmark includes both TITAN and Prov-GigaPath, and TITAN finishes mid-table — but its protocol extracts one feature vector per 224×224 tile and averages them, and it ran TITAN on eight patch-level tasks where no slide exists at all. That is a tile-encoder shape, so the slide-level models were almost certainly evaluated through their tile encoders rather than their aggregators. The paper never states which weights it loaded. [unverified] on the mechanism; the consequence is firm — that result is not evidence against pretrained aggregation, and the head-swap test above is still the only way to find out.
The tumour-origin question is already answered, and the answer is on this drive
Liver Tru-Cut — Tumour Origin AI asks where a liver core biopsy’s tumour came from. TOAD is the published model for precisely that. The cache contains a STATUS.md dated 2026-05-08 recording the outcome of checking it: TOAD’s trained classifier head was never published. The repository ships training code, and its “encoder” is a truncated ResNet50 seeded with generic ImageNet weights, not anything TOAD-trained; the only access to the real model is a one-slide-at-a-time web demo. So TOAD cannot serve as an off-the-shelf comparator, and that question does not need asking again. Recording it here because a negative result that stays in a STATUS.md on a drive gets rediscovered the expensive way.
Capability classes, mapped to what is already running
Tile encoders — UNI, UNI2-h, Virchow, Virchow2, H-optimus-0 and -1, Phikon, Phikon-v2, Midnight, OpenMidnight, H0-mini, Hibou-B/L, Path Foundation, PathOrchestra, EXAONE-Path 1.5 and 2.0, plus general backbones (DINOv2-large, CLIP). Seven are already wired into one extraction script in PinkKidney; kaiko-qupath builds on Midnight.
Which of them to reach for is now a settled-enough question to record. The 32-model benchmark in sources/papers/bareja-2026-foundation-model-benchmark.md puts Virchow2, UNI, UNI2, H-optimus-0 and Prov-GigaPath in the top five on both TCGA and non-TCGA tasks — and all five are on this drive. The finding that matters more than the ordering: on slide-level tasks outside TCGA those five differ by no statistically significant margin on any metric. So the shortlist is real, the ranking within it is not, and the choice among them should be made on feature dimension, inference cost and licence rather than on a leaderboard. The same benchmark found that pathology-specific encoders did not significantly beat general vision models on any tier, and that bigger architectures and bigger pretraining corpora bought nothing outside TCGA — which removes “it’s the newest and largest” as a reason to prefer anything here. See Encoder Benchmarking.
Vision-language models — CONCH, CONCH v1.5 (a ViT-L restored from the UNI checkpoint), MUSK, PLIP, QuiltNet, BiomedCLIP, MedSigLIP, MedGemma, Pathology-LLaVA, and ALPaCA / Llama-slideQA, which pairs Llama-3.1-8B with CONCH for slide-level question answering and is the largest single item on the drive at 107 GB. These classify from a text prompt with no training run at all, which is a different economics from everything else here, and they are the natural substrate for LLM Report-Quality Checking and Agentic Slide Navigation.
Temper the expectation on the classification side, though: the 2026 benchmark found pathology-specific vision-language models significantly worse than vision-only encoders on TCGA, CPTAC and external tasks — with the notable exception of out-of-domain slide-level tasks, where no model category differed significantly from any other. Individual wins still happen and they cluster by tissue: CONCH ranked first on a lung staging task and top-three on two lung histology tasks, and PLIP second on an MGMT methylation task. So these earn their place here for the zero-training economics and for lung work, not as general-purpose classifiers.
Cell and nucleus models — CellViT-Hibou-L, InstanSeg, NucSegAI, the SAM and SAM2 family, the MONAI nuclei bundles, and KongNet, a nuclei detection-and-classification model from the Warwick TIA Centre submitted to the 2025 MIDOG challenge — that is the mitosis domain-generalisation benchmark, so it is aimed squarely at the problem Mitotic Count describes and feeds Cell Detection and Counting and Ki-67 Proliferation Index.
Multiplex and spatial proteomics — two models, and they are the ones with no counterpart anywhere else in the group’s stack. KRONOS is a panel-agnostic encoder self-supervised on 47 million single-marker patches across 175 protein markers, 16 tissue types and 8 imaging platforms, built with marker-identity embeddings so it takes arbitrary named channels natively — which is exactly the input problem Spatial Proteomics identifies. MIPHEI-ViT goes the other way and predicts a 16-channel multiplex immunofluorescence image from ordinary H&E, so a retrospective H&E cohort can be given a synthetic marker panel without new staining.
Multi-stain — MADELEINE learns a slide representation across several stains of the same block, which is the setting of HER2 Intratumoral Heterogeneity (H&E plus IHC plus ISH) and a learned alternative to the geometric approach in Cross-Stain Registration.
Prognostic models — Polaris Breast ROR is a Weibull survival model for invasive disease-free survival that takes clinical covariates alongside slide features from its companion foundation model. It is the only model here that outputs a time-to-event prediction rather than a class, which puts it on the same ground as Survival Analysis Workflow and makes it subject to everything Concordance Index and Competing Risks say about evaluating such models.
Organ- and task-specific classifiers — the SPIDER breast, colorectal, skin and thorax models behind spider-qupath; neuroFM, a ViT-large trained on neuropathology at 20× and the obvious thing to point at the material behind WHO CNS — Who Cites Who; GastritisMIL; a Kather-100k colorectal tissue classifier; and a special-stain classifier that identifies 14 histological stain types — useful less as a model than as a cohort-curation tool, since it answers “what stain is this slide” over an archive where that was never recorded.
Quality control and tissue detection — the GrandQC tissue-detection weights are already here, which matters because GrandQC-QuPath spends most of its effort on environment plumbing to obtain exactly these. Also cx-tissue-seg, HEST tissue-seg and the TIAToolbox weights, all relevant to WSI Quality Control and to the tissue-detection work in stepbystep.
Text — PathologyBERT, a masked language model pretrained on breast pathology specimens. Relevant to Report Text Extraction, with the caveat that matters most here: it is English and breast-specific, while the group’s report corpus is Turkish and renal. It is a baseline to beat rather than a tool to adopt.
How it connects
Foundation Models in Pathology — the concept these are instances of; this page is the inventory of which ones are actually on hand.
Encoder Benchmarking — how to choose between the tile encoders above, and why the published comparison narrows the shortlist without picking a winner from it.
PinkKidney — has seven tile encoders in one harness and cached per-slide features, so it is the cheapest place to test any slide-level model here.
Liver Tru-Cut — Tumour Origin AI — the tumour-origin question, and the drive already records why TOAD cannot answer it off the shelf.
Spatial Proteomics — KRONOS and MIPHEI-ViT are the two models that make this modality reachable without new instrumentation.
Weakly Supervised Learning — the MIL aggregator that six slide-level models here would replace.
kaiko-qupath and spider-qupath — the two projects already consuming models from this drive.
GrandQC-QuPath — its model weights are already local.
Open questions
- Which slide-level model to try first, and on what. PinkKidney is the obvious testbed because the features are already cached and the cohort is well controlled — but its accuracy is already saturated, so the more informative target may be Liver Tru-Cut — Tumour Origin AI, where small biopsies and limited tissue are exactly the regime a pretrained aggregator should help.
- No pinned revisions are recorded. These were downloaded at unknown commits and several upstream repositories have moved since. Any reproducibility claim about an embedding depends on the exact revision, and that is currently only whatever was current on the download date. A manifest of repo ID, revision and local path would fix this and would also give stain-normalisation-qupath the machine-readable model list it currently fails to parse out of a directory dump.
Nothing records which of these has ever actually been run here.Answered for one cohort, and comprehensively. Liver Tru-Cut — Tumour Origin AI ran roughly twenty of these encoders over 509 liver biopsies and kept the results: H-optimus-1, MedSigLIP, Hibou-B, UNI2-h and PathOrchestra lead a cross-scanner transfer leaderboard; TITAN, CONCH, PLIP, QuiltNet, MedGemma, MUSK, Prov-GigaPath and CLIP-L all have prediction files. GrandQC was used as a tile quality filter. CHIEF is the exception that proves the point — its environment exists and a smoke test loaded its 18-class tumour-origin head successfully, but it was never run on the cohort. So the library is largely exercised rather than aspirational, which is a better answer than expected. BiomedCLIP is confirmed not run, deferred for an incompatible architecture.- Is the drive backed up? At 370 GB this is a substantial re-download.
[unverified]