WHO CNS — Who Cites Who

A bibliometric analysis of every reference cited across four editions of the WHO Classification of Tumours of the Central Nervous System — and the most methodologically careful project in this repository.

Purpose

Ask what evidence the WHO CNS classification is actually built on, and how that evidence base changed across the 2000, 2007, 2016 and 2021 editions. Published as a Quarto book at whociteswho.patoloji.dev.

The corpus is the classification books’ own bibliographies, extracted by OCR — not a literature search. That distinction matters and is corrected on Citation Network Analysis, which previously described this strand as applying the same OpenAlex machinery as the group’s other bibliometrics. OpenAlex and Lens.org are used here for enrichment of already-extracted references, not to define the corpus.

Data used

10,950 citation instances across 6,654 unique references. No patient data of any kind.

The 1979 and 1993 editions have their own descriptive chapters but are deliberately outside the citation cohort, and the reason is documented rather than assumed: both are classification guides and atlases rather than scientific reviews. The 1979 edition says so itself, quoted from its page 19 — “the literature references have intentionally been kept to a minimum”. Excluding data and writing down why is the right handling and worth copying.

Enrichment comes from PubMed, a Lens.org persistent public collection (244211, snapshotted to file), and an OpenAlex REST top-up for residual PMIDs Lens did not return.

Methods

Python extracts and resolves references, R (analysis/global_stats.R) computes roughly 50 dynamic variables, and Quarto renders the book. make render runs the whole chain: rebuild stale derived tables → recompute statistics → render → post-render hook copying output/ into the sibling Bibliometrics — Who Cites Who site repository. A pre-render staleness guard prints a loud non-blocking warning if any derived table is older than its inputs.

Scale: 1,918 tracked files, 317 scripts, 38 analysis chapters, 130 audit files.

The identifier and denominator policy

This is the part worth reading even if the subject does not interest you.

Every reference carries exactly one umbrella identifier by strict precedence — PMID › DOI › REFNNN — where REFNNN is a temporary placeholder resolved upward by manual curation. No reference is left unclassified, and Missing/Other must be zero.

From that follows a rule the project applies consistently: identity and volume statistics use all three identifier classes, while metadata-dependent analyses (MeSH, journals, authors, impact) use whatever subset has the data and declare their coverage against the full 6,654 cohort. References are never silently dropped.

The manuscript then reconciles four distinct population denominators that are not interchangeable — raw extracted references before de-duplication, citation instances, unique articles per edition, and the smaller per-chapter population — and states which one every reported percentage uses. See Coverage Bias for why this matters generally.

Confounder handling

The resolution rate varies by edition and by publication era, because PubMed indexed older literature retrospectively and unevenly. That gradient inflates anything computed on resolved references only. Rather than asserting it does not matter, the project re-estimates:

  • Coverage-corrected expansion ratio reported alongside the raw one; the expansion is attenuated but its direction survives.
  • Rarefaction to equal sample size for richness comparisons — and this changed a conclusion. Distinct-journal richness does not increase across editions and slightly decreases once the larger 2021 bibliography’s size advantage is removed, consistent with rising journal concentration (Gini). Author richness genuinely roughly doubles at equal sample. Two findings that look alike raw and are opposite once rarefied.
  • Cross-edition identity on a canonical key (author + year + title tokens) rather than PMID-exact, because a PMID-exact intersection systematically undercounts the shared canon when resolution coverage is edition-dependent. The PMID-exact figure is reported alongside as a conservative lower bound, and turnover likewise carries canonical-primary / PMID-exact-sensitivity.

Its own adversarial audit

audit/AUDIT_REPORT.md (2026-06-04) records a multi-agent audit — 11 auditors plus adversarial verification, every finding independently reproduced by a different skeptic agent before inclusion. 56 findings, 0 blockers, 6 major, 1 dismissed as a false positive. Reference verification: 67/67. A separate peer_review/METHODOLOGICAL_AUDIT_2026-07.md re-derived key statistics from raw data rather than reading the prose.

Current state / open questions

Very active; last commit 2026-07-12. Working tree carries one modified script and four untracked files from an in-progress Web of Science retrieval strand.

Two of the three major audit findings are fixed, verified against the current tree rather than taken from the report:

  • Pipeline reproducibility hazard — citations.csv is curated and not reproducible by consolidate_v2.py; re-running it regressed 10,950 → 10,557 rows and dropped the DOI-recovered citations. Now closed properly: a frozen citations.curated.csv is auto-restored after any consolidate run “so make can never drop those 452 citations”, and mesh-json is deliberately not a prerequisite of data. The count moved 426 → 452 between the June audit and now, consistent with continuing curation.
  • Cluster-count contradiction (19 vs 20) — the hardcoded “19” strings are gone; n_clusters is computed live and interpolated everywhere.

The third has no evidence of being closed. The June audit’s finding that the coverage-gradient correction is applied to volume metrics but not to the MeSH-driven headline findings — “the one place the manuscript’s own gold standard isn’t met” — has no coverage-equalised MeSH sensitivity variant in the current chapters, and the July audit covered different ground. [unverified] whether it was addressed elsewhere or is still open.

Open questions

  • Close the MeSH coverage-equalisation gap, or record a decision not to.
  • A [P1 — real] schema conflict remains logged in peer_review/OPEN_TASKS_2026-07.md: rebuild_cross_edition_ids.py writes unresolved_manifest.csv in the superseded schema. It is not wired into the Makefile so make render cannot trigger it — the conflict only fires if run by hand. Deleting or renaming it would remove the trap entirely.
  • No LICENSE file, matching DIY WSI, Video to WSI and Pathology Atlas Pipeline. Four repositories in a row. For a project intended for publication this one matters most. [unverified] as to intent.
  • The Web of Science strand is uncommitted work in progress; no page claim is made about it.
  • 304 references remain unresolved REF: placeholders, down from 432 in April. The manifest is now regenerated from the live master rather than being a snapshot, so the count cannot drift.

Related: Bibliometrics — Who Cites Who — the sibling page; that one covers the wider bibliometrics estate and the published-site repository, this one the WHO CNS analysis itself. Citation Network Analysis — the method page, corrected by this reading. Coverage Bias — the concept page this project’s denominator discipline justified. OpenAlex Pathology Metadata — one of three enrichment sources here, not the corpus.

Derived from: repository working tree read 2026-07-26 — README.md, Makefile, DATA_PROVENANCE.md, analysis/materials_methods.qmd, analysis/cluster_analysis.qmd, analysis/data_description_WHO1979.qmd, analysis/data_description_WHO1993.qmd, audit/AUDIT_REPORT.md, peer_review/METHODOLOGICAL_AUDIT_2026-07.md, peer_review/OPEN_TASKS_2026-07.md, plus git ls-files/git log/git status at commit 692b375.