Video to WSI

Upload a microscope video, get a focus-aware mosaic viewable in OpenSeadragon or exported as pyramidal TIFF — with an unusually disciplined statement of what it deliberately does not do.

Purpose

Turn a hand-recorded microscope scan into something that behaves like a whole slide image, without a slide scanner. See Video-Based WSI Mosaicing for the method; this page is the implementation.

The README is the most disciplined one in the estate and is worth reading as a model. It opens by stating the project “is not a medical device, has not been validated for primary diagnosis, and does not replace a calibrated whole-slide scanner”, then carries an explicit Deliberate boundaries section listing what is not implemented. Spot-checking those claims against the code, they hold — the disabled SVS/DICOM exporter really does refuse, with the message that “a renamed TIFF or a single-frame DICOM is not” a conformant replacement. Documentation that states its own limits and then honours them in code is rarer than it should be.

Data used

Microscope videos, uploaded. Nothing patient-identifiable is committed: *.MOV, *.mp4 and *.avi are gitignored, and the two large local recordings in the working directory (77 MB and 41 MB) are untracked. Tracked image material is three small test frames plus one stitched output.

Methods

FastAPI backend, Next.js 16 / React 19 frontend with OpenSeadragon 5 and a Fabric.js overlay. Python side uses OpenCV (headless), pyvips, tifffile; job state persists in SQLite and CPU-heavy work runs off the event loop. Five test modules.

The pipeline, verified in backend/processor.py:

  1. Frame selection — samples the whole video rather than stopping at the first N candidates, preserves scan chronology, and replaces a stationary field when a meaningfully sharper frame arrives.
  2. Feature detection — SIFT (5000 features), AKAZE, or ORB, as a real fallback chain.
  3. Registration — estimateAffinePartial2D with RANSAC to find inliers, then the estimate is collapsed to translation only: the median inlier displacement becomes a pure translation matrix. A frame is rejected if the translation exceeds 1.25× the largest image dimension. The rationale given is avoiding accumulated perspective, scale and rotation distortion — and DIY WSI arrived at the same conclusion independently, which is the most interesting thing this pair of repositories jointly says.
  4. Blending — keeps locally sharper pixels and feathers exposure-corrected overlaps.
  5. Disconnected scans — packed as separate regions rather than inventing a spatial relationship between unrelated fields. A good instinct: the alternative is a mosaic that looks continuous and is fiction.
  6. Output — full source resolution preserved in the canonical stitched.png (only the registration working copy is downsampled), lossless PNG Deep Zoom tiles, validated tiled pyramidal TIFF and OME-TIFF.

Optional non-diagnostic image description via Google Gen AI, inactive without GENAI_API_KEY.

Current state / open questions

Active; last commit 2026-07-25, working tree clean.

No spatial calibration is captured, and the README says so. The upload workflow does not record objective magnification or physical pixel size, so OME-TIFF omits physical size unless microns_per_pixel is supplied explicitly through the Python exporter. The exporter handles a supplied value correctly, including rejecting non-positive input.

But the analysis path defaults to 0.5 µm/px. backend/pathology_analysis.py takes microns_per_pixel: float = 0.5 and computes tissue_area_mm2 from it. Any density or area figure produced without an explicitly supplied calibration is therefore expressed in mm² derived from a placeholder. DIY WSI has the identical default in a different file, reached independently. See Video-Based WSI Mosaicing for why this is the single most consequential gap in both projects.

No LICENSE file. The repository declares no licence anywhere — unlike DIY WSI, whose README at least claims MIT. An unlicensed public repository is “all rights reserved” by default, which is probably not the intent for a research and teaching tool. [unverified] as to intent; the repository owner is the one who can settle it.

Open questions

  • Add a licence, or state deliberately that there is none.
  • Should microns_per_pixel be a required upload field rather than an optional exporter argument? It is the difference between a picture and a measurement.
  • The two sibling projects share no code and do not reference each other. Is that deliberate — a clean-room second attempt — or drift? Nothing in either repository says. [unverified]
  • Nineteen top-level markdown files including PHASE1–PHASE4 completion reports and two separate test summaries. Historical development notes rather than documentation; worth folding into docs/ as DIY WSI has done.

Related: DIY WSI — the sibling that does live camera acquisition and the full analysis stack; these two solve overlapping problems with no shared code. Video-Based WSI Mosaicing — the method page both projects justified. DZI Web Publication — the DZI output path, shared with the atlas tooling. Whole Slide Imaging — what this approximates without a scanner.

Derived from: repository working tree read 2026-07-26 — README.md, backend/processor.py, backend/pathology_analysis.py, backend/advanced_output.py, backend/requirements.txt, frontend/package.json, .gitignore, plus git ls-files and git log at commit 0b290c1.