Wang 2026 — Pathology-CoT and the Pathology-o3 Agent

Expert pathologists’ viewing behaviour, captured from viewer logs and paired with AI-drafted rationales they correct, used as training supervision for an agent that decides where to look on a whole-slide image.

Expert pathologists’ viewing behaviour, captured from viewer logs and paired with AI-drafted rationales they correct rather than author, used as training supervision for an agent that decides where to look on a whole-slide image — plus a released results file that lets the headline external-validation numbers be recomputed, and they do not fully reconcile.

Read status. This is the first paper in sources/papers/ read in full rather than from an abstract, and one of very few anywhere whose central results can be recomputed from released artefacts. Everything below labelled verified was re-derived from pathology-o3/external_validation_results/results.csv in the authors’ own repository. The recomputation was independently attacked by a separate reviewer instructed only to refute it (see “The external validation does not fully reconcile”). Numbers not so labelled are as printed in the paper.

The gap it names

Pathology VLMs are trained on static, preselected images. Diagnosis is not static: a pathologist pans, changes magnification, returns. The paper calls this the analysis–navigation gap — models are good at reading a field someone else chose, and have no training signal at all for choosing the field. The knowledge of where to look is tacit, learned by apprenticeship, and therefore absent from every text corpus a VLM was pretrained on.

That framing is the paper’s most portable contribution and it is independent of whether the agent works.

Three contributions

1. The AI Session Recorder. Pathology viewers already log every viewport change. Those logs are unusable directly — a single slide generates ~257 viewport events, and treating each as a 1,024×1,024 patch would produce >500,000 visual tokens. The recorder compresses the stream into two discrete commands, using heuristics borrowed from the fact that a real microscope has discrete objectives:

• <inspect> — viewport held over an area >1 s, or continuous panning >2 s. A broad look.
• <peek> — a rapid zoom to native resolution; captures the central 1,024×1,024 region.

Actions are then filtered to drop low-magnification overviews, merged at IoU >0.8, pruned toward the most specific view, and binned to the nearest standard objective (a 12× view becomes <10×-inspect>). Output is a command plus a normalised ROI box.

2. Review-over-authoring for rationales. For each ROI a VLM (Gemini-2.5-Pro) is given the task description, the low-magnification context, a high-magnification centre crop and, where it exists, the pathologist’s transcribed speech, and drafts a rationale in two parts — why zoom here and what is found. The pathologist then accepts, edits or rejects it in a lightweight interface.

Reported at 5.6× faster than authoring from scratch in Methods; the results text says “roughly 5–6× faster than typing and 3–4×” faster than dictation; the abstract rounds this to “sixfold”. Three slightly different figures for one measurement.

3. Pathology-o3. A two-stage agent: a YOLOv8 behaviour predictor proposes ROIs, then a VLM reasoning module analyses each crop and synthesises a diagnosis. Deliberately decoupled — unlike the training data, the agent is never asked to justify choosing an ROI; that is the predictor’s job.

The four fields this repo always wants

Field	Value
Dataset	Training: 137 WSIs from 25 CRC lymph node cases, 8 Stanford pathologists (4 attendings, 2 fellows, 2 residents), 10.6 h, 921 sessions, 5,222 conversational rounds, 1,521 extracted ROI boxes. Second cohort: DROID skin, 31 of 99 WSIs recorded, 6 pathologists, 4.2 h, 175 sessions, 1,110 rounds.
Scanner / magnification	Training slides all Leica Aperio, 40×, 0.25 µm/pixel — a single scanner at a single resolution.
Validation strategy	Fivefold cross-validation for the behaviour predictor, split at patient level (explicitly, to prevent leakage). Figs 2, 4, 5 merge predictions from the five test folds.
External validation	Yes, and a real one — LNCO2 (Sweden), different continent, different scanners (Leica Aperio and Hamamatsu), different magnifications (20× and 40×). Predictor trained only on Stanford data, no fine-tuning, fold-0 model only.

The external validation is genuinely strong in design: it varies scanner, magnification, country and patient population at once, and the model was not adapted. Most papers in External Validation’s sense do far less. That makes it worth reading the numbers carefully rather than dismissing them.

Results as printed

Internal CRC lymph node set: 84.5% precision, 100.0% recall, 75.4% accuracy. Next best, OpenAI o3: 46.7% / 87.5% / 57.8%. Non-CoT VLMs performed worse.

Averaged across backbones, adding behavioural guidance raised precision by 17.0% and recall by 11.4% — the claim that matters most, because it says the gain comes from the navigation policy rather than from any one VLM.

External LNCO2: 69.4% accuracy, 62.9% precision, 97.6% recall.

Cost: US$0.02 per slide with Gemini-2.5-Flash, US$0.12 with Gemini-2.5-Pro.

Bootstrap 95% CIs from 1,000 iterations; comparison against the next-best model by two-sided paired t-test on bootstrap differences, P < 0.05.

The external validation does not fully reconcile

This section is the reason the paper note is long. All of it was recomputed from the authors’ released results.csv (321 rows) and then handed to an independent reviewer whose only instruction was to refute it. It survived, with one of my own claims corrected in the process.

The released file contains per-slide predictions for four models plus ground truth. Cohort composition, verified: 321 slides from 22 patients, 41 positive and 280 negative.

The agent returned no prediction on 73 of 321 slides (22.7%). Verified. All 73 are ground-truth negative. The mechanism is visible in the released code: think_ln_classify.py raises FileNotFoundError("No ROI images found...") when the behaviour predictor proposed no ROI for a slide, and the caller catches it and returns None. So a slide the navigator skipped is never seen by the reasoning module at all. Parse failures elsewhere default to NEGATIVE, so the blanks are specifically never-analysed slides. They are not evenly spread — one patient has 14 of 28 slides blank, another 8 of 11.

Recomputing on that file:

Convention for the 73 non-answers	Accuracy	Precision	Recall
Dropped	89.9	62.5	97.6
Counted as errors, unclassed	69.5	62.5	97.6
Treated as POSITIVE	69.5	29.2	97.6
Treated as NEGATIVE	92.2	62.5	97.6
Paper reports	69.4	62.9	97.6

Three things follow, and they are separable.

1. Accuracy and precision use different denominators. The reported 69.4% is 223/321 — the non-answers counted as errors. The reported precision is only reachable on the 248 answered slides. No single convention produces both. The one self-consistent alternative — non-answers scored POSITIVE, giving 69.5% and 29.2% — is ruled out by the paper’s own sentence that Pathology-o3’s precision was “more than double that of the next-best model (Gemini at 23.5%)”; 29.2% is not, 62.5% is.

2. The reported precision of 62.9% is not attainable. Recall of 97.6% forces TP = 40 out of 41 positives, so precision must be 40/(40+FP), which steps 63.5 → 62.5 → 61.5. There is no integer confusion matrix on a 41-positive cohort giving 62.9% with 97.6% recall. This is an arithmetic slip independent of the denominator question, and it is small — 62.5 is the value the file supports.

3. The convention is not applied consistently across the panel. GPT-4o also abstains, on 33 slides, and its three reported numbers reproduce only with those 33 dropped (N = 288). So within one figure, one model’s non-answers are penalised and another’s are excluded.

The independent reviewer also found a plain transcription error in the same figure: Fig. 3a labels Gemini-2.5-Pro’s recall 7.8% where the released file gives 87.8% — a lost leading digit. Eight of the ten other numbers in that panel reproduce exactly, which is what establishes the file as the source.

What this does and does not mean. It does not mean the agent is bad, and it does not mean the external validation was not done. The honest reading of the released file is that on a genuinely out-of-distribution cohort the agent found every metastasis it looked at but declined to look at 23% of slides, and the two headline numbers each describe a different subset of that behaviour. The clinically interesting quantity — what would happen on those 73 slides if the navigator were forced to propose a region — is unmeasured, and it bounds precision somewhere in [29.2%, 62.5%].

It is also worth stating plainly that none of this would be checkable had the authors not released per-slide predictions. Almost no paper in this folder can be audited at this level. That release is a genuine methodological virtue and it is what surfaced the problem; the appropriate response is to use the finding, not to treat the paper as less trustworthy than the ones that cannot be checked at all.

What the repository actually contains

Cloned at 5ae3d04, the later of the repository’s two commits, 27 July 2026. Details on Pathology-CoT. In short: no licence file anywhere, so the code is technically all-rights-reserved despite the paper describing an “open-source methodology”; the released conversation data is the colorectal cohort although Data availability describes the skin cohort, and it is a subset — 155 session folders against the 921 GI sessions reported; every one of the 741 ROI coordinate slots in those conversations reads [coordinates not found], so the boxes survive only as overlays drawn on the shipped images and not as numbers; and three issues are open with none closed, two asking where the rest of the dataset is.

The behaviour-predictor weights are not missing — they are excluded from git (130 MB, over GitHub’s file limit) and linked from the README on Google Drive with three download routes. My first pass through this repository recorded them as unavailable, from reading the first 60 lines of the README and a shallow clone. Both were my errors, corrected here.

Methodological notes and reservations

1. 22 patients, 321 slides, slide-level bootstrap. Multiple lymph node slides come from the same colectomy — roughly 14.6 slides per patient. Bootstrap resampling at slide level treats them as independent, so the reported CIs are narrower than the data supports. Resampling at patient level is the standard correction. Not mentioned in Methods. This is the same clustering error External Validation warns about, in its most common form.
2. Prevalence is 12.8% and accuracy is reported. On a cohort that imbalanced, “always negative” scores 87.2% accuracy — higher than every model in the table including the agent’s own 69.4%. The paper’s own Methods section explains this trap clearly and then reports accuracy as a headline anyway. See Class Imbalance and Accuracy.
3. Non-answers are a feature reported as nothing. An agent that declines rather than guesses is arguably safer clinically. The problem is that abstention was neither designed, disclosed nor measured — it emerged from a FileNotFoundError. See Model Abstention.
4. Single scanner, single resolution in training. Every training slide was Leica Aperio 40× at 0.25 µm/pixel. The <peek> action is defined as a crop at native resolution, so its physical field of view is fixed by that µm/pixel and changes on a 20× scanner. The paper argues task-specific detectors are more robust to this than end-to-end models; that argument is plausible and untested. [unverified]
5. AI anchoring bias, acknowledged. Rationales are AI-drafted and pathologist-corrected; the paper says ~80% needed no edit. That is presented as efficiency, but a high accept rate is also what anchoring looks like. The authors name the risk and say internal audits suggest independence; those audits are not shown. [unverified]
6. Behaviour “hits” use IoU > 0.3 or containment. Deliberately generous, justified by pathologists preferring different magnifications. Reasonable, but it means behaviour efficiency and completeness are not comparable to detection metrics elsewhere.
7. The reasoning module is not ablated against a no-navigation control on the external cohort. The +17.0%/+11.4% backbone-averaged gain is internal.

Why it is relevant here

Three specific reasons, in order of directness.

It is the strongest existing answer to what Reading-Behaviour Capture is for. That page was written as a pre-adoption test for instruments that record viewing, and its open question is whether the group intends to run a reading-behaviour study at all. This paper supplies a second answer: recorded viewing behaviour is not only a measurement of readers, it is training supervision. And the two properties that page insists on — image-space coordinates with the zoom factor, and preserved visit order — are exactly the properties the AI Session Recorder consumes. An instrument that fails those tests cannot feed this pipeline either.

The group already owns a compliant instrument. The focus/ package in QuPath Patoloji Atlası Extension samples in image-pixel coordinates with the viewer’s downsample factor per tick and keeps sequences. That is the same class of data nuclei.io produced here. The distance between what the group records and what this paper trained on is smaller than it looks.

The abstention finding is directly reusable. LLM Report-Quality Checking and Report Text Extraction both involve models that can fail to return anything, and neither records what happens to those cases in an evaluation.

Open questions

• Would the focus/ recordings from QuPath Patoloji Atlası Extension discretise into <inspect>/<peek> commands with the paper’s heuristics? The thresholds (1 s hold, 2 s pan, IoU 0.8) are stated, so this is testable on existing data without collecting anything new.
• What is the group’s equivalent of the 73 non-answers? Nothing in this wiki records a coverage figure for any model or extraction step.
• Is LNCO2 worth applying for as a ready-made external cohort? See AIDA Data Hub.
• Does the arXiv preprint (2510.04587, Oct 2025) report the same LNCO2 numbers, or did they change between preprint and acceptance? Not checked. [unverified]