stepbystep

A numbered, self-teaching pipeline from installing OpenSlide to extracting patches — the largest and most recently worked-on codebase in the drafts umbrella, and the only one that documents a decision it got wrong and then corrected.

Purpose

This is the group’s own route into computational pathology, written as an ordered curriculum rather than a library: guides and scripts numbered 00 through 23, so the reading order and the execution order are the same thing. 00_LEARNING_PATH_GUIDE.md starts at installation; by 19_patch_size_optimizer.py and 23_interactive_patch_extraction.py it has reached the decisions described in Patch Extraction.

It is the most active thing in Digital Pathology Drafts — five commits, the last on 2025-11-01, which is also the umbrella’s own last commit. Where the other drafts are paused, this is where work stopped.

Data used

An internal control-H&E slide collection from 2025 on a network share, referenced by name throughout the guides. This is the right kind of material for the questions being asked — the same tissue stained and scanned under controlled conditions — and it is also why the tuning results below are collection-specific rather than general.

TissueSelector additionally carries a live QuPath project of roughly thirty-six slides with thumbnails and per-slide data. That project is correctly excluded from version control by the repository’s own .gitignore (/data, *.qpdata), so the slide material stays local.

Methods

The tissue-detection decision, and the correction. 21_CORRECTED_FINDINGS_SUMMARY.md is the most valuable file here, because it records a reversal. The initial assumption was that higher tissue coverage meant better detection, making threshold 250 (41.4% coverage) look optimal — a “4.7× improvement”. Objective scoring reversed it: threshold 200, at 8.8% coverage, scored 0.505 against threshold 250’s 0.400, because patch quality was 0.89 versus 0.25. Higher coverage was pulling in background.

Unusually, the correction was actually propagated. Both 15_tissue_detection_tester.py and 17_optimized_tissue_detector.py now carry the revised recommendation in their own headers and defaults; only the cosmetic rename the document asked for (optimized_ → validated_) was never done. A correction that reaches the code rather than only the write-up is rare enough to note as a strength.

Two reservations about that result, which the document does not raise:

The winning configuration scores 0.00 on spatial quality — the worst possible value on one of the three components — while threshold 250 scores 0.20 there. The composite is carried by patch quality at 40% weight. A 0.505-versus-0.400 margin on a hand-weighted composite where the winner is zero on a component is a weaker result than “validated optimal” suggests.
It was tuned on one collection, and the document says so plainly (“optimal for H&E slides in your collection”). By the time the threshold reaches qupath_tissue_detector.py it is described as “our proven threshold 200 method … optimized for clinical pathology workflows”. That drift from tuned here to proven clinically is the thing to watch; it is the same pattern as quoting an upstream vendor’s accuracy as if locally measured.
Threshold 200 was already the default before any of this ran. 02_simple_pathml.py declares def detect_tissue_simple(self, level=2, threshold=200). So the validation did not discover 200; it compared the incumbent against two alternatives and kept the incumbent. That is a perfectly respectable result and a much weaker claim than “validated optimal”, and the difference matters because the number is now carried downstream as evidence. Related: the quality score is frozen as a literal — self.quality_score = 0.505 in the QuPath bridge — and echoed for every slide processed, rather than recomputed, so it describes one historical experiment and not the slide in front of you.

The detection itself is plain mean-grayscale thresholding (grayscale = np.mean(slide_array, axis=2); tissue_mask = grayscale < threshold), not Otsu. Otsu appears only as a comparison arm in the tester and was never adopted — worth knowing, because “tissue detection” in WSI Quality Control usually implies something adaptive, and this is a fixed global cut.

The TissueSelector duplication is real but not yet drift. TODO.md records an rsync of stepbystep/qupath_integration/ into TissueSelector/. That path no longer exists — the folder is now 30_qupath_integration — so the recorded command is stale. Comparing the two trees file by file: 13 of 14 files are byte-identical, and the fourteenth differs only by a trailing # Legacy import comment. So the copy is effectively in sync today.

But the copy cannot run, because it sits at a different depth. python_bridge/qupath_tissue_detector.py computes stepbystep_dir = Path(__file__).parent.parent.parent. Inside stepbystep that resolves to the stepbystep root, where simple_pathml_tools.py and 02_simple_pathml.py both live. Inside TissueSelector, the same expression resolves one level higher — to the umbrella root, which contains neither file. The primary import fails, the fallback spec_from_file_location(... '02_simple_pathml.py') also misses, and the except calls sys.exit(1). Verified by resolving both paths on disk: the stepbystep target exists, the TissueSelector target does not.

So duplicating by copy produced a directory that looks identical to a working one and is not, for a reason invisible in a file-by-file diff. [unverified] whether it happens to work when launched with a working directory that makes the module importable anyway — that would mask the problem rather than fix it.

And the copy is not what TissueSelector actually runs. Its real entry point is scripts/TissueSelectionWorkflow.groovy (2285 lines), a hand-rolled detector that never calls the copied Python bridge at all — it does its own pixel-loop HSV/RGB/LAB thresholding, with an optional second Python path that writes an ad-hoc OpenCV script to a temp file at runtime. So there are three independent tissue-detection implementations across the two folders, and the one that ships is the one nobody documented.

Two of its filter primitives are stubs wired into live decisions, which is worse than being unimplemented:

private BufferedImage applyGaussianBlur(BufferedImage image, double sigma) {
    return image   // Would implement proper convolution
}
private double calculateSolidity(ROI roi) {
    return 0.8     // Would implement proper convex hull calculation
}

calculateSolidity is called in a real filtering branch — if (solidity < config.minSolidity || circularity < config.minCircularity) return false — so the solidity filter compares a constant against a user-set threshold. It never measures shape: depending on where minSolidity sits relative to 0.8 it silently accepts everything or rejects everything, while presenting as a tunable morphology control. The blur setting likewise does nothing. Neither announces itself.

This is also why the README-versus-status contradiction here resolves the way it does: README.md markets “Validated Tissue Detection for Clinical Pathology Workflow”, while the folder’s own STATUS.md says “Current State: INCOMPLETE… Core functionality broken”. The status file is the accurate one.

Current state / open questions

Decide whether TissueSelector is a copy or a project. If it is a copy, the depth assumption has to go (resolve the module by search, or make it a package). If it is a project, it should stop importing from stepbystep at all. The current arrangement is the worst of both.
The documented execution order breaks at its fourth step. EXECUTION_ORDER.md prescribes running 04_workflow_example.py, which begins from simple_pathml import SimpleSlideData, SimpleAIModelLoader. There is no simple_pathml.py — only 02_simple_pathml.py and simple_pathml_tools.py — so it raises ModuleNotFoundError immediately. Verified by checking all three filenames on disk. The root README.md Quick Start has the mirror-image problem: it shows from 02_simple_pathml import SimpleSlideData, which is not valid Python at all because an identifier cannot start with a digit — and the repository’s own 08_IMPORT_GUIDE.md documents that exact line as a SyntaxError to avoid. The curriculum warns against the mistake its own front page makes.
The numbering is a reading order, not a dependency order. 17_optimized_tissue_detector.py never consumes 16_threshold_validator.py’s output; each hardcodes the threshold independently. Worth knowing before treating the sequence as a pipeline.
The numbered scheme has unnumbered duplicates. metadata_extractor.py and simple_pathml_tools.py sit alongside 11_metadata_extractor.py and 12_simple_pathml_tools.py. Since the numbering is the documentation, a second unnumbered copy defeats it, and it is not recorded which is canonical.
Re-check threshold 200 against the spatial-quality zero before it is used outside the collection it was tuned on. That is a small experiment and it is the one that would either confirm the correction or reveal that the composite metric needs a different weighting.

Related: Patch Extraction — the decisions this curriculum ends at, and which it documents better than most published work does. Related: WSI Quality Control — tissue detection is the first QC gate; the threshold finding belongs to that discussion. Related: QuPath Annotation Workflow — where the Groovy side of this integration lands. Related: Digital Pathology Drafts — the umbrella; TissueSelector is a submodule of it while this is a plain directory, which is why the two drifted apart structurally.

Derived from: repository source read 2026-07-27 — 21_CORRECTED_FINDINGS_SUMMARY.md, 15_tissue_detection_tester.py, 17_optimized_tissue_detector.py, 30_qupath_integration/python_bridge/qupath_tissue_detector.py, the TissueSelector copy of the same, TODO.md, directory listings of both trees with per-file hash comparison; git log for currency; .gitignore in TissueSelector for the exclusion of slide data.