Hard Example Mining
Focusing training on the examples a model currently gets wrong, instead of letting an easy majority dominate the gradient.
What it is
In a segmentation task most pixels are trivial — plain background, or the middle of an obvious cell. Averaged over an image, those easy pixels swamp the handful of genuinely difficult ones at cell boundaries or in crowded regions, and the gradient ends up mostly reflecting work the model has already mastered.
Hard example mining fixes the balance by concentrating the loss where the model is failing. The common variants:
- OHEM (online hard example mining) — within each batch, keep only the pixels the model currently gets wrong and compute the loss on those. “Wrong” is defined either by a confidence threshold, or by taking the top-k highest-loss pixels. A floor on how many are kept prevents the loss collapsing to almost nothing once the model gets good.
- Focal loss — instead of selecting a subset, smoothly down-weight easy examples so hard ones dominate. Continuous rather than hard-thresholded.
- Region-level mining — select difficult regions rather than pixels, on the argument that difficulty is a property of a neighbourhood.
Two failure modes are worth holding onto. Mislabelled data is, by construction, “hard” — so mining amplifies label noise, which matters when annotations are dot clicks made quickly by several observers. And aggressive mining destabilises early training, when nearly everything is hard; hence the floors and warm-up schedules that most implementations carry.
Why it matters for my work
It is the standard answer to the imbalance that dominates pathology image tasks, where the object of interest occupies a small fraction of the pixels — the spatial version of the problem described in Class Imbalance and Accuracy.
Çakı et al., Med Image Anal 2026;113:104155 is a variation worth understanding, because it changes what “hard” means. Conventional OHEM defines difficulty pixel by pixel. That paper argues difficulty in crowded tissue is architectural — the model has the wrong arrangement of cells, not merely the wrong pixels — and so selects hard regions by the topological discrepancy between true and predicted cell distributions, then applies ordinary pixel-level OHEM inside them. Mining at two levels: region, then pixel.
That reframing is the transferable idea. “Hard” is whatever your evaluation metric cares about, and if the metric is architectural then a pixel-wise notion of hardness is measuring the wrong thing.
How it connects
Persistent Homology — supplies the region-level difficulty signal in the source paper, replacing pixel loss as the criterion.
Class Imbalance and Accuracy — the underlying problem; hard example mining is a training-time response to it, where reporting the right metric is the evaluation-time response.
Cell Detection and Counting — the task where the crowded-region failure mode appears.
Interobserver Agreement — mining amplifies label noise, so annotation reliability sets a practical ceiling on how aggressively it can be pushed.
Çakı 2026 — Topology-Guided Hard Example Mining (TG-HEM) — where this page’s structural definition of “hard” comes from: the cell-detection loss is reweighted by the topological discrepancy between true and predicted cell arrangements, so difficulty is a property of the arrangement rather than of a pixel.
Open questions
- With dot annotations from several pathologists, how much of what gets mined as “hard” is actually inter-annotator disagreement? Nothing in the source paper’s abstract addresses this.
[unverified] - Does the group’s own IHC quantification work use any form of mining, or plain averaged loss? Not documented.
[unverified]