Digital Pathology Drafts

The private umbrella repository the QuPath prototypes live in — a working scratchpad of some fifty folders that is best understood as three different kinds of thing sharing one git history, and whose submodule wiring does not reproduce on a fresh clone.

Purpose

This is where the group’s digital-pathology experiments accumulate before any of them becomes a real project. It is the parent of spider-qupath, kaiko-qupath, metadata-qupath and anonym-qupath, each of which has its own page because each turned out to have findings worth recording separately. This page exists to describe the container itself: what is in it, what state it is in, and which of its properties are traps for anyone trying to use it.

It is a private repository (GitHub API, read 2026-07-27: isPrivate: true, no licence, created 2025-03-01, last push 2025-11-01, roughly 55 MB). That single fact settles several questions that would otherwise look alarming — see Data used.

The repository is not one project but three kinds of content mixed together:

1. Own draft repositories, wired in as submodules — the four QuPath prototypes above, plus the Turkish teaching books, the Sectra integrations, and the reporting and bibliometrics work.
2. Working code committed as ordinary directories — about thirty folders whose files live directly in this repository’s history rather than in a submodule of their own.
3. Reference collections that contain no original code at all — a downloaded paper plus a flattened text dump of somebody else’s repository, kept for reading.

Telling these apart matters, because only the first kind has an independent home to go back to.

Data used

No cohort, and by design no imaging data: .gitignore excludes *.svs, *.pt, *.pth, *.mp4, *.png and *.zip. That exclusion holds — git ls-files returns no whole-slide image, no video and no model weights anywhere in the tree. The large binaries that do sit in the working directory (a 1.9 GB slide and a 155 MB screen recording under the ASDP hands-on folder; 742 MB of tiles and 212 MB of weights under the ESDIP academy folder) are local-only and have never been committed.

Sixteen publisher PDFs are committed, about 35 MB, spread across the reference collections and a general pdfs/ folder. In a private repository these are personal reference copies rather than redistribution, which is why this is recorded here as a fact rather than raised as a problem. It would become one if the repository were ever made public, and that is the reason to record it now rather than discover it later. Nothing from any of them is copied into sources/papers/, which publishes.

Two folders in the tree contain identifiable clinical material and must not be published, mirrored, or made public with the repository. One holds photographs of slide racks in which the labels are fully legible — institution, accession number, block and part numbers, processing dates — and those images are tracked in git. The accession format is sequential and dated, so it re-links on its own, and the 2D barcodes in frame encode the same values; by the standard in De-identification that makes them re-identification keys rather than illustrations. The other holds macroscopic specimen photographs with the cassette identifier legible in frame, and is untracked. A third folder holds Turkish macroscopy and diagnosis narrative as a two-column table — no name or national-identifier fields, but free text with embedded accession numbers and dates, which is the usual residual risk in Report Text Extraction. None of this material is reproduced anywhere in this wiki, by value or by image.

Methods

The submodule wiring is broken in a way that a fresh clone will hit. .gitmodules declares 25 submodule paths, but only 22 gitlinks exist in the index. RwithFileMaker, GrandQC-QuPath and imagebank/imagebank_tmp are declared and unregistered — the first two show as plain untracked directories in git status. A git clone --recursive therefore does not reproduce this working tree: those folders simply will not arrive, and git submodule update --init has nothing in the index to act on. Anyone setting this up on a second machine, which is exactly what a drafts umbrella is for, gets a silently incomplete checkout.

The submodule pointers that do exist are stale. The umbrella’s own last commit is 2025-11-01, but RwithFileMaker upstream was pushed 2026-04-22, shiny-server 2025-12-13, and MemorialPath/ecosystem 2026-04-24. The recorded commits are months behind the repositories they point at, so the umbrella is a snapshot of an older state rather than a current index.

Provenance was checked rather than assumed on the one folder that invites a wrong guess. petebankhead-qupath-budding-scripts is named after somebody else, and it is exactly what the name says: sbalci/qupath-budding-scripts is a genuine GitHub fork of petebankhead/qupath-budding-scripts, MIT, licence intact. Credit is correctly attributed and there is nothing to flag.

The reference collections are not vendored code, which is worth stating because they look like it. GNCnn, slide2vec and liverquant each contain a publisher PDF plus a .txt dump of the upstream repository’s source, not a copy of the upstream tree. That distinction matters: a vendored source tree stripped of its licence would be a real problem, and reading the folders rather than their names is what showed they are reading material instead.

They are still the weakest form of what they are trying to be. A flattened .txt dump of a repository goes stale, loses commit history, and cannot be updated; an upstream URL and a DOI do the same job better and stay current.

Two working tools in the tree are worth naming because wiki pages now rest on them. macroCalibrate/interactive_ruler_calibrator.py is an interactive OpenCV tool that recovers scale from a ruler in frame — two clicks spanning a known distance give pixels-per-millimetre, the whole image is then resampled to a fixed target scale, and scale_factors.csv records the factor and the before/after dimensions per file. It resamples rather than merely recording the scale, which makes a comparable series at the cost of an irreversible resize; the method and its traps are written up in Macroscopic Specimen Photography. The pdfs/ folder is not a grab-bag but a coherent reading list on stain colour — Ruifrok’s colour deconvolution, Landini’s revisit of it, Bautista’s calibration-slide standardisation, Gray’s stain quantification for laboratory QC, Clarke and Treanor’s review, the Karolinska physical-calibration trial and the UK NEQAS 247-lab variability study. That cluster is what Colour Calibration is built from.

Two folders in the batch are completely empty: uscap_2026_abstracts and TileExporter, zero files each. The first is the output directory of a script described below; the second has never contained anything.

One root script is worth flagging rather than judging. download_uscap_abstracts.py (untracked) drives a headed Chromium session with a spoofed desktop Chrome user-agent across 1768 Elsevier article URLs, and its own docstring says it “bypasses Elsevier 403s” before merging every result into a single PDF. Its output directory is empty, so it has not been run to completion. Two things are simply true and are recorded without adjudication: whether that retrieval is within the publisher’s access terms is a question for the user and, if it matters, an institutional library, not something to settle here; and the link file is named uscap2026_pdf_links.txt while its first entries are 2023, 2024 and 2025 article identifiers, so the name overstates what it holds.

Current state / open questions

Active as a scratchpad, dormant as a repository — nine months since the last commit, while at least three of its submodules have moved on without it. That gap is the honest summary of its state: it is a place things are kept, not a place things are currently integrated.

• The clone problem is the one worth fixing, because it is invisible until someone tries. Either register the three declared-but-missing submodules or remove them from .gitmodules; the current state claims a structure the index does not contain.
• The tracked slide-label photographs are the one item needing a decision. They are in git history, not just the working tree, so removing them later means rewriting history. The repository being private makes this containable now and awkward later. Flagged here, decided by the user.
• Six committed DICOM annotation files have never had their headers read. They sit in the ecosystem submodule beside seven real whole-slide images that are correctly gitignored, and their JSON sidecars carry only anonymised exam identifiers — which is reassuring about the sidecars and says nothing about the DICOM. DICOM headers routinely carry PatientName, PatientID and InstitutionName tags entirely independently of the filename, so an anonymised filename is not evidence of an anonymised file. Reading those six headers is a five-minute job and it is the highest-value unresolved item in this repository.
• Third-party material is referenced rather than copied — decided 2026-07-27. Keep the upstream URL and citation; drop the local copy unless it is needed to run something, in which case keep it untracked as GrandQC-QuPath already does. This also points at the maintained version rather than a frozen snapshot, which is the practical reason to prefer it.
• Whether this should remain one repository at all is the open question underneath the other two. Three kinds of content with different lifetimes — durable submodules, working code, and read-only reference material — are sharing one history, and the reference material is what makes the repository large while the submodules are what make it fragile.

Related: spider-qupath, kaiko-qupath, metadata-qupath, anonym-qupath — the four prototypes that live here as submodules; each was read separately because each had findings this page would have buried. Related: QuPath Tooling — the hub that indexes the QuPath cluster; the folders here that did not earn their own page are listed there with their reasons. Related: De-identification — the standard the slide-label photographs fail, and the reason they are described here by category rather than by value.

Derived from: repository and GitHub API read 2026-07-27 — .gitmodules, .gitignore, README.md, TODO.md, requirements.txt, download_uscap_abstracts.py, uscap2026_pdf_links.txt; git ls-files --stage, git status, git log, git rev-list for index and currency state; gh repo view for visibility, licence and push dates on the umbrella and 22 submodule repositories; directory listings and file-type inventories across all 54 folders.