Slide Metadata Extraction

Getting scanner, magnification, µm/pixel and scan date out of slide files — the one prerequisite behind most of this wiki’s open questions, and the difference between a value that was measured and one that was computed.

Getting scanner, magnification, µm/pixel and scan date out of slide files — the one prerequisite behind most of this wiki’s open questions, and the difference between a value that was measured and one that was computed.

What it is

Every WSI carries provenance in its own header: which scanner, at what objective power, at what microns per pixel, on what date. Extracting it is a few lines of code and is usually treated as too trivial to write down. This page exists because it is not trivial, and because the group has now run two extractors over the same slides and got two different answers.

The fields worth capturing, in roughly the order they matter here:

Microns per pixel (openslide.mpp-x / mpp-y). The only spatial quantity with physical meaning. Mitotic Count needs it to count per defined area, and Patch Extraction needs it to tile at a fixed magnification instead of a fixed divisor.

Objective power (openslide.objective-power, Aperio AppMag). The scanner’s own declared magnification. Note this is a declaration, not a measurement — see the trap below.

Vendor and scanner identity (openslide.vendor, Aperio ScanScope ID, ScannerType). What Scanner and Stain Variability needs to attribute a batch effect to anything.

Scan date and time, which live inside Aperio’s ImageDescription as a |-delimited Key = Value string. Scanner Log Analysis is built on these.

Whether a label or macro image is present at all — the one field that turns De-identification from a claim into a check.

Why it matters for my work

This is Known gap 1 in Wiki — Front Door: no scanner, magnification or cohort size is recorded for most AI projects here. The method half of that gap is now answered, even though the cohort inventory itself is still missing.

Two routes were run over the same 62 control slides, and only one worked. Both live in Digital Pathology Drafts. The Python/OpenSlide route (helpers/svs_info_extraction/extract_svs_metadata.py, and the extractor behind stepbystep/output/metadata/) returned vendor, ScanScope ID, AppMag, MPP, ICC profile and pyramid structure for every slide. The QuPath/Groovy route (metadata-qupath) returned none of those fields for any slide, because server.getMetadata() hands back an ImageServerMetadata object and the script iterates it with a two-argument { key, value -> } closure, which is a type error. The whole scanner block fails on every image, in every format.

The comparison is exact rather than approximate: the two output tables cover the same 62 distinct slide names, intersection 62, with nothing on either side that is not on the other.

The failure reports itself as success. metadata-qupath’s own committed summary reads:

Successfully processed: 107 · Failed: 107 · Success rate: 100%

because the per-image error is caught by an inner handler that appends to an error list without aborting, while the success counter increments unconditionally. Anyone reading the summary sees a clean run. This is the same shape as the fail-open behaviour De-identification tracks — an operation that does not happen and a log that says it did — and it is worth noticing that the two appeared independently in the same estate.

The trap: a computed value wearing a measurement’s name

This is the transferable part, and it has already propagated into this wiki.

metadata-qupath cannot read AppMag, so it computes a substitute: estimated_magnification = round(0.25 / pixel_width_um * 40). On the control cohort that yields 38× for 55 slides, 40× for 47, and 20× for 5.

On those same slides, the OpenSlide route read the scanner’s own AppMag tag: 40× for 101 slides, 20× for 5. The 55 slides labelled “38×” are Aperio ScanScope units declaring 40×.

The tell needs no code at all: no scanner emits 38 as an objective power. It is not a lens. Its appearance in a magnification column is arithmetic — 0.25 / 0.2633 × 40 = 37.98 — and nothing else.

Then the derived value got promoted. PinkKidney takes estimated_magnification and assigns scanner_vendor from it through a lookup — {20: "3DHistech", 38: "Hamamatsu", 40: "Leica Aperio"} — filling a column that is never read from any slide. Its cohort_metadata.csv has 356 rows and exactly three (vendor, magnification) pairs, with zero exceptions, because the correspondence is definitional rather than observed. Two consequences, and they differ in strength:

• Certain: that table cannot be evidence that vendor and magnification are confounded in the cohort. They agree by construction. The real situation is one measured variable — pixel size — presented as three.
• Open: whether the vendor labels are nonetheless correct. If the lab does run 3DHistech at 20× and Aperio at 40×, the lookup produces right answers by a wrong method. But the control cohort above gives a reason to actually check: the pixel size PinkKidney attributes to “Hamamatsu at 38×”, ~0.263 µm, belongs at the same institution to Aperio ScanScope units declaring 40×. [unverified] — one run of extract_svs_metadata.py against a single slide from that bucket settles it.

The general rule: a metadata column should record where it came from. A field derived from another field is a modelling assumption, and once it is named scanner_vendor nobody downstream can tell.

Ordering, and why it is forced

De-identification establishes that redaction strips date, time and ScanScope ID while leaving MPP and AppMag intact. So extraction has to run before redaction or the scan timestamps are gone permanently. This is not a preference; it is the only order that preserves both.

How it connects

Patch Extraction — needs µm/pixel to tile at a fixed magnification; the fixed-divisor bug it records is what happens when this metadata is missing or wrong.

Scanner and Stain Variability — cannot attribute a batch effect to a scanner unless the scanner identity was actually read rather than inferred from pixel size.

Mitotic Count — area-calibrated counting depends on µm/pixel, so losing this metadata costs a diagnostic capability, not just provenance.

De-identification — sets the ordering constraint, and the has_label_image field is what would let an anonymisation step be verified rather than trusted.

Scanner Log Analysis — consumes the scan dates that only this step can recover.

metadata-qupath — the QuPath implementation, and the worked example of every failure mode on this page.

Open questions

• The cohort inventory still does not exist. The tooling is proven on 62 control slides; the slides that matter are on Memorial WSI Share and have never been crawled. The remaining work is a shell loop and a CSV writer, not a design problem — extract_svs_metadata.py handles one file per invocation, prints JSON to stdout, and has no exception handling, so a corrupt slide aborts the run. Wrapping it is the whole job.
• Which magnification figures already in this wiki are derived rather than measured? At least the 38× on PinkKidney, Scanner and Stain Variability, Patch Extraction and PembeBobrek trace to the formula above. Others may too.
• Is there any case where the scanner’s declared AppMag is itself wrong, so the computed value is the more honest one? Worth knowing before treating the tag as ground truth — the argument above is that 38 is impossible, not that 40 is verified. [unverified]
• Nothing checks that an extraction actually populated the fields it claims to. A one-line assertion that the scanner column is non-empty would have caught the Groovy failure on the first run instead of after 107 slides.

Derived from: sources/projects/metadata-qupath.md, sources/projects/digital-pathology-drafts.md; source read 2026-07-26 — metadata-qupath/scripts/QuPath_v06_Compatible.groovy, metadata-qupath/cohort_metadata/cohort_metadata_v06.csv and detailed_summary_v06.txt, helpers/svs_info_extraction/extract_svs_metadata.py, stepbystep/output/metadata/svs_metadata_20250614_100514.csv, PinkKidney/python/add_batch_metadata.py and cohort_metadata/cohort_metadata.csv.