Corrigendum to the MultiSurv paper note

Four corrections to vale-silva-2021-multisurv.md, found on re-checking the same session it was written. Read this file alongside it — the append-only rule on sources/papers/ means the original cannot be edited, so this is the appended correction rather than a rewrite.

Two of the four were errors in my favour-of-the-paper direction, which is the more embarrassing direction, and one turned a hand-waved claim into a measured one.

1. “Baselines beaten on unimodal data” is wrong

The note says: “Baselines beaten on unimodal data: CPH, RSF, DeepSurv, CoxTime, DeepHit, MTLR, Nnet-survival”, and later “beaten fairly on unimodal data”.

Actually, computed from figures_and_tables/results.csv, MultiSurv’s Ctd rank against the seven baselines is:

Modality MultiSurv Ctd Rank Beaten by
mRNA 0.758 1 / 8 —
miRNA 0.702 1 / 8 —
CNV 0.617 1 / 8 —
Clinical 0.809 2 / 8 Nnet-survival (0.810)
DNAm 0.736 6 / 8 DeepSurv 0.759, MTLR 0.751, CoxTime 0.747, CPH 0.739, DeepHit 0.737

On clinical data Nnet-survival beats it on all three metrics, not just Ctd — Ctd 0.810 vs 0.809, IBS 0.132 vs 0.143, INBLL 0.404 vs 0.439. On DNAm it is sixth of eight, behind plain Cox regression.

So MultiSurv wins clearly on three of five modalities, essentially ties on clinical, and loses on DNAm. The paper’s own wording — best results in “nearly all” configurations — is accurate; my summary flattened it into a stronger claim than the authors made. Correct version: the non-PH architecture is competitive across modalities and clearly best on three of five, but it is not a clean sweep, and on methylation data a 1972 model beats it.

2. The per-cancer median is now measured, not asserted

The note says criticism 1 rests on “a median well below the pan-cancer figure” — stated without a number, because I had only the min and max.

Actually the per-entity table is stored in the output cells of figures_and_tables/figure-evaluation_by_cancer_type.ipynb, for the best model (clinical + mRNA). Nineteen entities have enough test patients to be scored:

  • Median Ctd 0.774, IQR 0.658–0.848, range 0.554 (LUSC) – 0.988 (THCA).
  • 12 of 19 entities fall below the pan-cancer figure of 0.822.

The claim holds and is now quantified: the pan-cancer number sits above the median entity and above nearly two-thirds of them. Two further caveats the same output cells reveal — the per-entity test sets are tiny (26 to 108 patients), so the per-cancer confidence intervals are very wide (CESC 0.721–0.983), and 14 of the 33 cancer types could not be scored at all because their test sets were too small, with the notebook logging skipped bootstraps for “no admissable pairs” on entities down to 5 test patients.

3. WSI patches are centre-cropped, not downsampled, and “50 per slide” is not a recorded value

The note says patches were generated “at OpenSlide level 0 (512 px, downsampled to 299)” and that “50 patches were generated per slide offline”.

Actually src/utils.py:105,117,123 applies torchvision.transforms.CenterCrop(299) — no resize appears anywhere in the pipeline. So the model sees a 299 px window cut out of the 512 px level-0 patch, which is a smaller field of view than “downsampled to 299” implies. The image submodel therefore sees even less tissue per patch than the note states, which strengthens criticism 8 rather than weakening it.

And the 50-patches figure is not recorded: src/scripts/wsi_patcher.py’s --n_patches option defaults to None, and preprocess_wsi.ipynb cell 75 introduces 50 as an illustration — “For example, 50 patches per slide will yield…”. The 5 patches sampled per forward pass is solid (n_wsi_patches=5 in the evaluation notebook, plus the wsi5patches299px training-log directory names); the denominator it is sampled from is not. Criticism 7’s “5 of ~50” should read “5 of an unrecorded number”.

4. Colour handling — the note overstates the absence, and misses a real defect

The note says src/transforms.py has “no colour handling beyond dataset mean/std”.

Actually both halves of that are wrong, and the truth is more interesting:

  • There is colour augmentation, but only at training time and only in utils.py, not transforms.py: ColorJitter(brightness=64/255, contrast=0.5, saturation=0.25, hue=0.04). Validation and test compositions omit it. This is a partial, deliberate defence against stain variation — worth crediting. It is augmentation, not normalisation, so the substance of criticism 9 (no stain normalisation, site signal learnable) stands.
  • There is no mean/std normalisation at all. grep for Normalize across src/ returns nothing, and transforms.ToTensor only divides by 255. So patches enter an ImageNet-pretrained ResNeXt-50 as raw [0,1] RGB, without the ImageNet channel normalisation those weights were trained under. That is a genuine defect, not just an omission: it degrades what the frozen pretrained layers can contribute, and it is a plausible contributor to the image submodel’s near-chance performance. preprocess_wsi.ipynb even has a “Compute patch mean and std” section, so the statistics were computed and then apparently never applied.

This adds an eleventh criticism rather than removing one.

Where these corrections landed

  • Correction 1 → sources/projects/multisurv.md, and nowhere on the wiki, because no wiki page made a baseline-sweep claim.
  • Correction 2 → wiki/concepts/concordance-index.md, which now carries the measured median.
  • Corrections 3 and 4 → wiki/methods/patch-extraction.md and wiki/datasets/tcga.md.

No published wiki page ever carried any of the four errors; they were confined to the source note. The figures the wiki does cite — 0.809, 0.822, 0.787, 0.569 — were checked against results.csv and are correct.