Concordance Index
The standard discrimination metric for survival models — and the one that silently rewards a model for knowing which disease the patient has.
What it is
The concordance index asks: across all pairs of patients whose order of events is known, how often does the model rank them correctly? A patient who dies earlier should have been given the higher predicted risk. Score 1.0 is perfect ordering, 0.5 is a coin flip. It is the survival analogue of the area under the ROC curve, and it handles censoring by simply refusing to score pairs whose order cannot be determined.
Three things about it are worth holding onto.
Harrell’s C and Antolini’s Ctd are not the same metric. Harrell’s C assumes one risk score per patient that does not change over time — natural for a Cox model, where the hazard ratio is constant by assumption. Antolini’s time-dependent Ctd compares patients using the predicted survival probability at the time the earlier event happened, so it stays meaningful when a model’s predicted risk ordering changes across follow-up. Any non-proportional model needs the time-dependent version; reporting Harrell’s C for one is a category error. The two coincide when hazards really are proportional, which is a useful sanity check.
Concordance across heterogeneous groups measures the groups. This is the failure mode that matters most and it is almost never stated. The index pools every comparable pair, including pairs of patients with entirely different diseases. If a cohort mixes cancers with wildly different baseline prognosis, a model that does nothing but identify the tumour type will score extremely well, because ranking a thyroid carcinoma above a pancreatic adenocarcinoma is correct and trivial. The number then describes the cohort’s heterogeneity, not the model’s ability to separate patients who face the same clinical decision.
MultiSurv is a clean worked example, and its own numbers quantify the effect. It reports a pan-cancer Ctd of 0.822 across 33 TCGA cancer types — and the cancer type is literally one of its ten clinical input features, given a 33→17 embedding. Evaluated within each disease, the same model has a median Ctd of 0.774 (IQR 0.658–0.848, range 0.554 in lung squamous to 0.988 in thyroid), and 12 of the 19 entities with enough test patients to score fall below the pan-cancer figure. So the headline sits above the median disease and above roughly two-thirds of them. The two numbers answer different questions, and only the per-disease one corresponds to a decision anybody makes.
A second lesson hides in that breakdown: the per-entity test sets ran from 26 to 108 patients, so the per-cancer confidence intervals are very wide (cervical carcinoma 0.721–0.983), and 14 of the 33 entities could not be scored at all for want of test patients. Stratifying an evaluation is the right thing to do and it costs precision — which is an argument for planning the test set around the strata you intend to report, not an argument against stratifying.
A second, independent worked example — and it is not a big-cohort or pan-cancer artefact. SurvivalNet, from a different group four years earlier and on molecular rather than pan-cancer data, does the same thing at the scale of a single organ. Its shipped demo cohort is 560 TCGA glioma patients, and the Subtypes field in that file splits them into three molecular classes whose outcomes are not remotely comparable: IDHmut-codel (n=145) has a 12.4% event rate, IDHmut-non-codel (n=209) 20.6%, and IDHwt (n=203) 68.0%. That spread is the whole prognostic range of glioma. The variables defining those classes are themselves inputs — IDH1 and IDH2 mutation, 1p and 19q arm-level copy number, and three glioblastoma histology indicators — and the repository’s own committed attribution output confirms the model leans on that split: of 399 features ranked by mean risk gradient, the strongest protective ones are CDKN2A, 10q, 10p, SMARCA4, PTEN, IDH1 and IDH2, and a glioblastoma histology flag sits second from the top of the risk-increasing end. The model’s own explanation of itself is a subtype classifier.
One detail worth keeping, because it sharpens rather than softens the point: the model gets there through the glioblastoma-associated markers, not through the codeletion definition. 1p and 19q rank 280th and 373rd of 399. So it cleanly separates the large contrast — IDHwt against the rest — and largely ignores the finer 12.4%-versus-20.6% distinction. A pooled concordance index rewards it for the separation it did find, and cannot show you the one it missed.
The lesson this second case adds: “one cancer type” is not a stratum. Pooling looks safe when the cohort is one organ and one histological family, and it is not — a molecularly defined subgroup with a fivefold difference in event rate does the same free work for the metric that thyroid-versus-pancreas does. Whenever a strong prognostic classifier is available as an input, ask what the concordance index would be within its levels before quoting the pooled number.
The same trap applies to any pooled analysis: multi-site, multi-stage, or mixed-histology cohorts all inflate concordance for free. The fix is to report it stratified, or to compute it within strata and pool the estimates rather than the patients.
Discrimination is not calibration, and the c-index cannot see the difference. Concordance depends only on ranking, so a model can order every patient perfectly while predicting survival probabilities that are wildly wrong in absolute terms — and score 1.0. Anything used to counsel a patient needs a calibration metric alongside: the integrated Brier score (mean squared distance between predicted probability and observed status, integrated over follow-up) or the integrated negative binomial log-likelihood. Reporting both is still uncommon enough to be worth noticing when a paper does it.
A related note on how models produce survival curves at all. The Cox family predicts a relative risk and recovers absolute survival from a separately estimated baseline hazard. A discrete-time formulation instead chops follow-up into intervals and predicts a conditional survival probability for each one, multiplying them into a curve. That drops the proportional hazards assumption entirely and makes calibration metrics natural to compute, at the cost of choosing an interval grid. MultiSurv’s is a compact, readable implementation of this idea (src/loss.py, ~50 lines) and is the most reusable thing in that repository.
“C-index” names a family, not a function — check the tie rule before trusting a number. This is an implementation-level point that the textbook definition hides, and it was measured rather than assumed. SurvivalNet ships its own c-index, and testing it against a reference Harrell’s C on its own 560-patient cohort gives three results worth generalising:
- On continuous risk scores — what a model actually emits — it agrees with the reference to within a constant 0.0006. For its intended use it is fine.
- That small offset is not noise. It comes from pairs where a censoring time exactly equals an event time: they enter the denominator, but no branch can ever award them credit. Real cohorts recorded in whole months or years produce many such pairs.
- The consequential difference is that tied predictions get no half credit. Textbook Harrell’s C awards 0.5 when two patients receive the same risk score; this implementation awards 0. A constant-risk null model therefore scores 0.0003, not 0.5, and discretising risk scores to two levels pulls the value 0.071 below the reference. (The residual 0.0003 rather than exactly zero comes from one branch that does award full credit for equal risks — the case where both patients had events at the same time — which is the only way a constant-risk model can score at all.)
So the “0.5 is chance” intuition is a property of a particular tie convention, not of the metric. Any model whose outputs tie — a tree ensemble with few leaves, a risk-group classifier, a discretised score — will be scored differently by two implementations that both call themselves the c-index. When reporting one, name the implementation.
One thing SurvivalNet gets right by construction, worth noting because it is the mirror of the Harrell-versus-Antolini point above: it predicts a single time-invariant risk score per patient, so Harrell’s C genuinely is the correct choice there. The category error only appears when a non-proportional model is scored with a time-independent index.
Why it matters for my work
jsurvival offers regularised regression, survival trees and random survival forests, and Survival Analysis Workflow already records that no agreed evaluation protocol exists for them. This page is the missing half of that open question: the c-index is what those models will be reported with, and the version, the stratification, and the accompanying calibration metric all need deciding before rather than after.
The stratification point applies directly to the group’s own designs. Pancreas Cohorts is a federated family of sub-studies sharing one schema — a concordance index computed across pooled sub-cohorts with different case mix would be inflated the same way MultiSurv’s is. So would any outcome model built across the mixed-entity material in Memorial LIS Extracts.
There is also a reporting discipline here worth importing wholesale: report the metric with a confidence interval, say what the interval covers, and say what it does not. A bootstrap over the test set captures sampling variability in the test set only — not variability from retraining the model, which for a deep model is often larger and is usually unreported.
How it connects
Survival Analysis Workflow — this is the evaluation step that workflow currently stops short of; step 7 ends at plotting and the ML options below it have no stated protocol.
jsurvival — the module that will actually compute these numbers, and where a stratified c-index would have to be implemented if the group wants one.
External Validation — a c-index from the development cohort is the number most likely to fall on an external one, because part of it is cohort composition rather than model skill.
Diagnostic Test Accuracy — the same discrimination-versus-calibration split, one step earlier: AUC is to sensitivity and specificity what the c-index is to a calibration curve.
Competing Risks — when a competing event is common the quantity being discriminated needs defining first; concordance on cause-specific death answers a different question than concordance on all-cause death.
Class Imbalance and Accuracy — the same underlying lesson in the classification setting: a single pooled summary statistic can be driven by the composition of the sample rather than by the model.
Multimodal Fusion — the setting where this bites hardest, because a fused model’s apparent gain is compared against a unimodal baseline using exactly this metric.
MultiSurv — where the ~50-line discrete-time survival loss described above actually lives, if the group ever wants to implement one.
SurvivalNet — the second worked example of the heterogeneity trap and the source of the tie-convention lesson; also the case where Harrell’s C is correctly chosen.
Gradient-Based Feature Attribution — the complement to this metric: concordance says how well a model ranks patients, attribution says what it used to do it, and in both worked examples here the inflation was invisible to the first and obvious in the second.
Open questions
Does jsurvival report Harrell’s C, Antolini’s Ctd, or both?Answered from source, 2026-07-26. The variant was documented all along — in the.a.yaml, which is not where anyone had looked. ClinicoPath jamovi Module’sconcordanceindexanalysis is titled “Concordance Index (Harrell’s C-index)” and offers two methods: Harrell’s (default) and Uno’s IPCW, plus a separate time-dependent concordance option. jsurvival’s ownmultisurvivalalso reports a C-index with IPCW weighting.Two things follow. Antolini’s Ctd is not offered under that name anywhere, so the Harrell-versus-Antolini category error above is not solved by picking a menu item — the time-dependent option is the nearest available answer and what it computes is worth checking before relying on it.
[unverified]And Harrell is the default, which is the wrong default for exactly the non-PH models (tree-based, AFT, time-varying) the umbrella provides, so the choice has to be made deliberately per analysis.The tie convention remains unrecorded. [unverified] This is now the whole of the original question that survives, and it is the cheaper half: it decides whether tree-based and risk-group outputs, which produce tied predictions by construction, are scored on the same scale as Cox models. It only became askable once the SurvivalNet implementation was measured.
Is a stratified or within-strata concordance available anywhere in the group’s tooling? If not, it is a small addition with an outsized effect on honesty in pooled analyses. The SurvivalNet case sets the trigger for when it is needed: whenever a strong prognostic classifier is available as an input feature, which for the group’s own material means stage, grade, or entity in almost any pooled cohort.
For any planned outcome model here, is the reporting target discrimination, calibration, or clinical utility? Decision Curve Analysis answers the third and none of the three substitutes for the others.