Agentic Slide Navigation

Letting a model choose where to look on a whole-slide image and at what magnification, instead of handing it regions someone else picked — which turns the scarce training signal from labelled tissue into recorded expert behaviour.

What it is

Nearly every model in this wiki is handed its input. Patch Extraction cuts the slide into tiles on a fixed grid; Weakly Supervised Learning pools those tiles under one slide label; Foundation Models in Pathology encodes whichever tiles arrive. The selection is a preprocessing decision made once, by a script, uniformly across the slide.

A pathologist does not work that way. They form an impression at low power, choose somewhere to go, change magnification, look, revise, and return. Agentic navigation is the attempt to model that loop: the system decides where to look next and at what power, then reasons about what it sees, then decides again.

The distinction that matters is not “agent” versus “model” — it is who chooses the field of view. Grid tiling chooses exhaustively and blindly. Attention-based MIL chooses after seeing everything, by weighting tiles it has already encoded. Agentic navigation chooses before, and therefore never encodes most of the slide at all.

Why the choice is forced, and what it costs

At 40× a slide is tens of gigapixels. Feeding all of it to a vision–language model is not expensive, it is impossible: one recorded reading session generates ~257 viewport events, and treating each as a 1,024×1,024 patch would produce over 500,000 visual tokens for a single slide. Something has to select, and the question is only whether the selector is a grid, an attention head, or a learned policy.

The cost of choosing first is that everything not visited is invisible. A grid is exhaustive and stupid; a policy is selective and can be wrong. That is a real trade, not a free improvement, and it shows up as a specific failure mode — see Model Abstention for the case where the navigator proposed nothing at all and the slide was silently never analysed.

The data problem, which is the actual problem

A navigation policy needs supervision, and the supervision does not exist in any corpus. Where to look is tacit, procedural knowledge, learned at a microscope beside someone more senior. It is not in textbooks, not in reports, not on the internet — which is why a general VLM asked to pick regions performs poorly at it even when it reads the chosen regions well. The published framing for this is the analysis–navigation gap: models trained on preselected images are good at analysis and have no training signal for navigation.

Two answers exist, and only one is currently evidenced:

Let a strong VLM choose. Prompt a capable model to nominate regions. Cheap, needs no new data, and reported to diverge from expert paths and select regions of low diagnostic utility. [unverified] as to magnitude beyond a single comparison.

Record what experts actually did. Pathology viewers already log every viewport change with a timestamp and a zoom level. Discretise that stream into a small vocabulary of actions, pair each action with a rationale, and train on it. This converts routine diagnostic work — not extra annotation sessions — into supervision, which is why it scales at all.

The second is what Reading-Behaviour Capture describes the recording contract for, and the connection is the most useful thing on this page: an instrument that satisfies that page’s two properties is already producing navigation training data, whether or not anyone intended it to. Positions in image space with the zoom factor give you the ROI box; preserved visit order gives you the sequence. An instrument that fails those tests cannot feed this, no matter how good its heatmaps look.

What the vocabulary looks like

The one published example discretises continuous navigation into two actions, using the fact that a real microscope has discrete objectives rather than a continuous zoom:

a broad look — viewport held over a region for more than about a second, or panned steadily for a couple of seconds, binned to the nearest standard objective;
a quick high-power check — a rapid zoom to native resolution, capturing a fixed-size central crop.

Overlapping actions are merged and low-power overviews dropped. The thresholds are arbitrary but concrete, which makes them testable against any other recording. Nothing about them is specific to the viewer they were derived from.

Why it matters for my work

The group may already hold the raw material. The focus/ package in QuPath Patoloji Atlası Extension samples in image-pixel coordinates with the viewer’s exact downsample factor per tick and preserves sequences. That is the same class of data the published pipeline consumed. Running published discretisation thresholds over recordings that already exist costs a day and no new data collection — the most actionable item this topic produced, and it is recorded as an open question on Reading-Behaviour Capture rather than as work anyone has scheduled.

It reframes what a reading-behaviour study is for. Interobserver Agreement records that the instrument has never been exercised, partly because the payoff was framed as measuring readers. A second payoff is training data, and it accrues from ordinary diagnostic work rather than from a study anyone has to run.

It changes where scanner metadata bites. A high-power action defined as “a crop at native resolution” has a physical field of view fixed by µm/pixel, so the same action means different tissue on a 20× and a 40× scan — the Patch Extraction argument applied to actions instead of tiles. Any policy learned on one scanner inherits that scale. See Scanner and Stain Variability.

The evaluation vocabulary is different and needs its own care. Navigation is scored by whether proposed regions match expert-visited ones — efficiency (precision of proposals) and completeness (recall of expert regions) — under a deliberately loose hit criterion, because two pathologists examining the same feature at different powers produce boxes with low overlap. Those numbers are not comparable to detection metrics, and reporting them beside diagnostic accuracy invites exactly that confusion.

How it connects

Reading-Behaviour Capture — the recording contract that decides whether a viewing log can become training data at all; this page is the strongest reason that contract matters.

QuPath Patoloji Atlası Extension — the in-house instrument that already satisfies it, and the reason the question here is practical rather than hypothetical.

Pathology-CoT — the reference implementation, read at source level, including the parts that do not work.

Model Abstention — the specific failure mode of choosing before looking: when the navigator proposes nothing, the slide is never analysed, and the resulting empty prediction quietly reshapes every metric.

Weakly Supervised Learning — the incumbent answer to the same problem. MIL also finds the diagnostic region without region labels, but by weighting tiles after encoding all of them; the two can compose, with a navigator acting as a hard-attention filter ahead of a MIL pipeline.

Patch Extraction — what agentic selection replaces, and the page that already documents why a tile without a recorded magnification is not a measurement.

Foundation Models in Pathology — the analysis half of the gap. A stronger tile encoder makes the reading better and the navigation no better, which is the asymmetry that motivates this whole approach.

Mitotic Count — the clearest clinical case for why navigation is the task: the count depends on which hotspot was chosen, so a model that cannot choose a field cannot do the job at all.

External Validation — navigation policies are learned from one institution’s slides on one scanner, and are exactly the kind of thing that transfers poorly; testing the policy separately from the reasoning is what makes the failure visible.

Wang 2026 — Pathology-CoT and the Pathology-o3 Agent — the paper behind this page, and the one whose released per-slide predictions made its own external-validation numbers checkable.

Open questions

Do the group’s existing focus/ recordings discretise cleanly under published thresholds? The thresholds are stated, so this is testable on data already on disk.
How many recorded sessions would be needed before a policy is worth training? The one published example used 10.6 hours across 8 pathologists to extract ~1,500 regions. Whether that is a floor or an excess is unknown. [unverified]
Is navigation worth learning for any task the group actually has? It suits search-and-find problems — metastasis in nodes, hotspot location. It is a poor fit for exhaustive screening or margin tracing, where a single parallel proposal step cannot substitute for systematic coverage.
Does a navigation policy encode a pathologist rather than a diagnosis? Recorded behaviour differs by seniority, and one published cohort found more than 60% of recorded regions sat on normal structures — screening behaviour, not abnormality detection. What a policy trained on mixed seniority converges to is unrecorded. [unverified]