Video-Based WSI Mosaicing

Building a whole slide image from a microscope camera instead of a scanner — the frame selection, registration and calibration decisions that separate a usable mosaic from a picture that only looks like one.

What it is

A slide scanner moves a calibrated stage in a known pattern and knows where every field sits. Video-based mosaicing removes the stage: someone moves the slide by hand — or a motorised stage does — while a camera records, and the software recovers the spatial arrangement afterwards from image content alone. The output is a stitched mosaic, tiled into a pyramid and served like a real WSI.

The appeal is obvious in a department that has microscopes and cameras but limited scanner time, and it is the only route to a zoomable image for a laboratory with no scanner at all. The cost is that everything a scanner knows by construction — position, magnification, physical pixel size — now has to be either estimated or supplied, and each of those is a place to be wrong quietly.

Two implementations exist here: Video to WSI (upload a recording, no live capture) and DIY WSI (live camera acquisition, desktop app, larger analysis stack). They share no code and do not reference each other, which makes their agreements unusually informative.

The pipeline, and the decision at each stage

1. Frame selection. A video has far more frames than fields. Selecting the sharpest frame per position is the goal, and the traps are stopping early (sampling only the start of the video) and losing scan order. Video to WSI samples the whole video, preserves chronology, and replaces a stationary field when a sharper frame for the same position arrives. DIY WSI uses motion-based adaptive sampling, reported to cut processing 60–70%.

2. Registration — where both projects landed in the same place. This is the substantive finding.

The obvious approach is to estimate a rich transform between consecutive frames — a homography (8 degrees of freedom) or a similarity (4 DOF, allowing rotation and scale). Both projects implement that machinery. Both then disable it by default in favour of pure translation.

Video to WSI estimates a partial affine with RANSAC, then collapses it to the median inlier displacement — a pure translation matrix.
DIY WSI keeps affine and homography code but ships force_translation_only: True and phase_correlation_priority: True, with a test asserting that matches carrying rotation or scale are rejected.

The reason recorded in both is the same: accumulated drift. Under a microscope the stage genuinely moves in X and Y; apparent rotation and scale between adjacent fields are mostly estimation error, and a transform chain that is free to absorb them compounds the error across hundreds of frames until the mosaic bends. Constraining the model to the motion that physically occurs is more robust than fitting the motion the data appears to show.

Two independent codebases reaching that conclusion is a stronger argument than either making it alone. The generalisable form: match the transform model to the physical degrees of freedom, not to what the estimator can fit.

A third implementation now corroborates it from outside the group, and adds the step both local projects lack. Micro-Manager’s TileAligner — read at source 2026-08-23 — refines a stage-acquired tile grid by phase correlation on the overlap strips, exactly the primitive both projects converged on. With a motorised stage the translation-only model is not a robustness choice but true by construction, which is the cleanest possible confirmation of the rule above. And it then does what neither local pipeline does: global optimisation over the whole grid (BFS over pairwise shifts) instead of chaining consecutive frames — which is the structural answer to the accumulated-drift problem both projects worked around by constraining the transform. A chained mosaic controls drift by refusing degrees of freedom; a globally optimised one controls it by letting every overlap vote.

3. Blending. Overlaps must be merged without visible seams or lost detail. Both keep the locally sharper pixel rather than averaging — averaging two fields of differing focus produces a result blurrier than the better one. Video to WSI additionally feathers exposure-corrected overlaps.

4. Coverage. Hand scanning leaves gaps that only become visible after stitching. DIY WSI tracks coverage against a 70% default overlap and navigates to uncovered regions during acquisition, which is the only point at which a gap is cheap to fix.

5. Disconnected regions. When a scan jumps, the honest output is separate regions rather than an invented spatial relationship. Video to WSI packs them separately. A mosaic that looks continuous but joins unrelated fields is worse than one that visibly does not.

6. Export. Pyramidal TIFF, OME-TIFF, and DZI tiles for DZI Web Publication. Memory is the binding constraint at scale, which is why DIY WSI composites from disk tiles rather than holding the mosaic in RAM.

Why it matters for my work

It is the cheapest possible route to a zoomable slide, and the department already has the inputs. It is also the natural teaching-material pipeline: a mosaic exported as DZI feeds Patoloji Atlası by exactly the same path a scanned slide does.

The honest framing is the one Video to WSI’s own README uses — not a medical device, not validated for primary diagnosis, not a replacement for a calibrated scanner. This is a route to teaching images and research material, and the section below is why that boundary is real rather than legal boilerplate.

The calibration gap — the thing to fix first

A scanner records physical pixel size in its metadata. A video mosaic does not, unless someone computes it, and both projects default it to 0.5 µm/px.

DIY WSI implements the correct calculation — specimen µm/px = sensor pitch ÷ (magnification × adapter) — and its own test gives 0.155 µm/px for the Leica Flexacam i5 at 20×. That function is called only from tests. The consumer, cell density per mm², defaults to 0.5.
Video to WSI takes microns_per_pixel = 0.5 in its analysis module and computes tissue area in mm² from it. Its README states plainly that the upload workflow captures no magnification or pixel calibration.

The default is roughly 3.2× too large for that hardware, and since area goes as the square, any per-mm² figure derived from it is out by around 10×. Neither project is currently shipping that to users — DIY WSI’s analysis stack is not wired to its API — so this is a gap to close before wiring, not a live defect.

The wider point is the one already recorded on Mitotic Count: counting anything per mm² requires knowing µm/pixel, and a plausible-looking default is more dangerous than a missing value, because a missing value stops the calculation while a default silently completes it. Any quantitative readout from a mosaic — cell density, tissue area, and certainly a mitotic count — depends on a calibration these systems do not yet require.

A mosaic with no recorded µm/pixel is a picture. With one, it is a measurement.

How it connects

Whole Slide Imaging — what this approximates; that page assumes a scanner produced the file, along with the metadata this approach has to reconstruct.

Mitotic Count — the sharpest consequence of the calibration gap: area-based counting is impossible without µm/pixel, and wrong rather than absent if a default supplies it.

DZI Web Publication — the shared output path, and the same tiling step the atlas tooling uses.

WSI Quality Control — focus, blur and artefact screening apply at least as much here, where acquisition is manual and failures are the operator’s rather than the scanner’s.

Patoloji Atlası — the plausible destination for mosaics built this way.

Scanner and Stain Variability — a camera and adapter are one more source of appearance variation, on top of scanner and stain; a mosaic and a scanned slide of the same tissue will not look alike.

DIY WSI and Video to WSI — the two implementations behind every specific claim here.

Micro-Manager — the stage-driven end of the same spectrum: the standard microscope-control platform whose tiled-acquisition path independently uses the same phase-correlation primitive, adds global optimisation, and measures its own stitching error — the reference point for what this method gives up by not having a stage.

Open questions

Should physical pixel size be a required input rather than a defaulted one? Both projects would answer the calibration problem outright by refusing to compute a density without it. [unverified] whether that is planned in either.
Has a mosaic ever been compared against a scanned WSI of the same slide? That is the experiment that would establish what these systems are good for, and nothing in either repository records it. [unverified]
Is stitching accuracy measured anywhere, or only judged by eye? No registration-error metric appears in either project’s tests. [unverified] Narrowed 2026-08-23, and the narrowing explains the gap rather than closing it. Micro-Manager’s aligner reports accepted/total pair counts and the RMS deviation of aligned tile origins from their nominal grid positions — a real stitching-error metric, computable because a stage supplies a nominal position to deviate from. A hand-moved slide has no nominal grid, so the metric is a property of having a stage, not of better software. What remains open for the local projects is therefore harder than first stated: a hand-acquired mosaic needs an external reference — a stage-micrometer slide, or a scanned WSI of the same section — before its stitching error is measurable at all, which is the same held-out-reference logic Cross-Stain Registration arrived at for registration quality.
If a motorised stage ever arrives, the build-versus-adopt decision inverts. With a stage, Micro-Manager covers grid acquisition, phase-correlation refinement with global optimisation, a measured alignment error, and pyramidal OME-BigTIFF export — everything this page describes except one thing: its pyramidal writer’s contract is grayscale-only, which excludes H&E brightfield as it stands. Whether any microscope in the department has a motorised stage is recorded nowhere in this wiki, and that single fact decides whether this entire method remains the only option or becomes the fallback. [unverified]
Z-stack and focus fusion are unimplemented in both. For thick or folded tissue that is a real limit, and it is the same missing z-axis that Mitotic Count flags for reading mitoses on a flat image.