Cognitive Bias in AI-Assisted Diagnosis

Putting a model beside a pathologist changes how the pathologist decides, not only what they see; the measured effect is that advice agreeing with a wrong first impression gets adopted, and that under time pressure the reader stops checking whether it agrees at all.

What it is

A cognitive bias is a systematic deviation from rational judgement — systematic being the word that matters, because it means the error does not average out over cases. Two of them are the named hazards when a decision-support system sits beside a reader, and they pull in opposite directions:

Automation bias is treating the machine’s output as authoritative and stopping the search. Its two failure modes are omission (missing what the model did not flag) and commission (adopting what the model wrongly flagged). The commission error has a clean operational definition — a negative consultation, where a reader who was right before seeing the model is wrong after.

Confirmation bias is the opposite posture: weighing evidence by how well it fits what you already believe. Its dangerous form here is false confirmation — an initially wrong human judgement reinforced by wrong machine advice. Two errors that happen to point the same way produce more confidence than either alone, which is the source of the title of the experiment below.

The two are not alternatives to be chosen between. The most useful thing measured so far is that they trade off against each other as conditions change, so an intervention that suppresses one can uncover the other.

How thin the evidence is, and why that is the first thing to say

sources/papers/rosbach-2026-cognitive-bias-review.md searched ACM, IEEE and PubMed to 19 August 2025 for studies that actually operationalised a cognitive bias during expert–machine diagnostic interaction. The corpus was 24 studies — 8 primary, 16 reviews — and of the primary studies, exactly one came from pathology. For each of the 12 biases it identified, the review had to write its own hypothetical pathology examples, because no empirical ones existed.

That is the state of the field, and it sets the register for this page. Everything quantitative below comes from a single experiment on 28 readers, reported twice. Treat the numbers as one existence proof with a coefficient, not as an established effect size.

Three further provenance facts belong with them. The review itself was read abstract only — it is paywalled with no open deposit, so its twelve biases and their definitions are unread here, and only the corpus counts above are quoted from it. And every figure below is taken from the November 2024 arXiv v1 preprints, not from the versions of record at CHI ’25 and BVM 2025; the CHI camera-ready is two pages longer than the preprint, so it contains material this reading does not cover, and any number here could in principle have changed in revision.

Third: neither of the two things that could check these numbers is publicly reachable (established 2026-08-31). The preprints’ only code and data pointer is a review-time anonymous.4open.science link — AB-TP-6440 for the automation-bias paper, 2W0R-A180 for the confirmation-bias one — and both now return HTTP 410 repository_expired. That is the designed end state of such a link rather than a withdrawal by the authors: a fabricated identifier returns 404 repo_not_found, so the 410 confirms these repositories existed and lapsed. No de-anonymised replacement was located in the group’s GitHub organisation (DeepMicroscopy, ~60 repositories), in repository search, or under author accounts — a search boundary, not a proof of absence. The versions of record are closed: Unpaywall reports oa_status: closed for the BVM chapter with no repository copy, Europe PMC has no record of it, and the Springer page serves only the abstract. arXiv carries v1 only — there is no post-acceptance revision in which the pointer could have been updated. The practical consequence is that the released dataset, the per-patch table, the participant demographics and the analysis code described in both preprints cannot be inspected from here, and any open question phrased as “computable from the released data” is not.

The one experiment, and what it measured

sources/papers/rosbach-2025-confirmation-bias-tcp.md and sources/papers/rosbach-2025-automation-bias-tcp.md are the same study — 28 pathology experts, 20 H&E patches, estimating tumour cell percentage; an unaided round, a two-week washout, then the identical images with a real FCOS detector’s prediction and a prototype-based explanation; half of each round under a ten-second countdown. Two papers, two dependent variables, one dataset. Reading them as independent replications is the mistake to avoid.

The task was chosen well. TCP is quick, routine, quantitative, unstandardised, badly reproducible, and it gates molecular testing through an assay-specific tumour-fraction threshold — so it is both bias-prone and consequential.

What the AI did to accuracy. Mean absolute deviation from ground truth fell from 13.55 (unaided, no timer) and 15.29 (unaided, timer) to 11.74 and 12.23 with AI. On average, help helped.

Confirmation bias, measured. Restricting to cases where the AI-aided answer was wrong by more than 5 points and human and machine erred in the same direction, the distance from the final answer to the AI’s number grows with the distance between the AI’s number and the reader’s own first answer at slope 0.61 (p < .001), with an intercept indistinguishable from zero — when the model’s number matches yours, your final answer is the model’s number. Splitting on whether advice was congruent (within 15 points of the baseline estimate) or not:

Predictor of the final answer	Congruent AI (n=78)	Incongruent AI (n=139)
Reader’s own baseline estimate	0.25 (p = 0.09)	0.47 (p < .001)
AI prediction	0.60 (p < .001)	0.43 (p < .001)

When the model agreed, the reader’s own prior estimate no longer significantly predicted their final answer. Confidence and compliance moved together: mean confidence 3.87 vs 3.24 on a 5-point scale, mean judge-advisor statistic 0.55 vs 0.49.

That p = 0.09 is the load-bearing number on this page and it is not a sentence the paper writes, so it was checked rather than trusted. The table it comes from had to be reconstructed from the preprint’s text layer, where column alignment is exactly the kind of thing that goes wrong silently. The assignment was confirmed arithmetically: in a linear mixed model the p-value is determined by the coefficient and its standard error, and recomputing t = coeff/SE reproduces every printed p-value under this reading — 0.25/0.15 → t = 1.67, p ≈ 0.097, printed as 0.09; 0.60/0.17 → t = 3.53, p < .001; both incongruent-condition rows t > 7. The competing assignment, with the two standard errors swapped, gives p ≈ 0.14 where the table prints 0.09, and is therefore excluded. The same recomputation reproduces the intercepts of the three time-pressure models (0.53, 0.21, 0.85), so the whole table reconstruction stands on arithmetic rather than on column positions.

Automation bias, measured. Binarising TCP at the 25% direct-sequencing threshold, of 560 AI-assisted assessments only 67 involved adopting advice that contradicted the reader’s own baseline — 29 corrections and 38 new errors. The negative-consultation rate is 38/560 ≈ 7%, consistent with the 6–11% reported outside pathology.

The handoff, which is the finding worth carrying. Under time pressure, confirmation bias got weaker (slope 0.65 → 0.56) while overall reliance on the model rose (JAS 0.49 → 0.55, t(27) = −2.80, p = 0.005) and the damage done by each negative consultation roughly doubled (mean absolute deviation 19.42 → 27.79). The authors’ reading — offered as interpretation, not measurement — is that automation bias eclipses confirmation bias when there is no time to deliberate: the reader stops asking whether the advice agrees with them and simply takes it.

The most robust conclusion of the pair is the negative one. Only 67 of 560 assessments involved following the model against one’s own judgement at all. These readers were not credulous. They were selective, and the selectivity is the bias.

The result that is easy to miss: two metrics, opposite signs

On the same data, scored two ways:

Continuous (mean absolute deviation): the AI helped, significantly.
Thresholded (correct side of the 25% cutoff): among cases where the AI was followed against prior judgement, it produced 38 new errors against 29 corrections — net harm.

Both are correct. The continuous gain comes from many small refinements; the threshold crossings are rare and asymmetric. But nobody acts on a tumour percentage — they act on whether the sample clears the assay threshold. On the metric that matches the clinical action, the tally was negative. Any evaluation of a decision aid for a thresholded biomarker that reports only the continuous accuracy gain can therefore report a success that the decision it feeds does not share. See Biomarker Cut Points.

How to measure it, if this group runs a reader study

The design requirements are cheap and they are all load-bearing. Without them none of the analysis above is possible, and a study that only records agreement-with-AI cannot tell a corrected error from a reinforced one.

Collect the unaided baseline first, with a washout. Two weeks in the source study. This is the single non-negotiable element: every measurement above is a function of the difference between an unaided and an aided answer on the same case by the same reader.
Count both consultation directions. Positive (model corrected the reader) and negative (model broke a correct answer), separately. An accuracy delta hides the tally.
Declare the denominator. A negative consultation is only possible when the reader was right and the model was wrong. Reporting 38/560 uses every assessment as the denominator; the rate per opportunity is higher and is the quantity a reader intuitively wants. The same argument this wiki makes on Model Abstention.
Record confidence per case. One extra click, and it carried a real result — congruent advice raised confidence as well as compliance.
Use the judge-advisor statistic for reliance: 0 = ignored the advisor, 1 = adopted it wholly, 0.5 = equal weight. Cheap, interpretable, comparable across studies.
Do not treat repeated measures as independent. The automation-bias paper’s own ANOVA appears to have done exactly that — see that note’s point 1. That note proposed settling it against the released analysis script; as of 2026-08-31 the script is unreachable (above), and the version of record, which would settle it the other way if the camera-ready prints F(1, 27), is paywalled. The inference needs neither: with 28 subjects, F(1, 108) is the error df of a between-subjects 2×2 on 28 × 4 = 112 observations, while the same paper’s paired t-tests on the same data are t(27). That internal inconsistency is self-contained evidence and it now stays [unverified], which raises rather than lowers the caution owed to the reported p < .01. The one remaining way to resolve it is Springer access to pp. 129–134.

Why it matters for my work

Every AI-assistance result recorded in this wiki is an accuracy statement. Bertram’s computer-assisted mitotic count improved interobserver reproducibility; a triage model is 80% precise. None of them measured what the assistance did to the reader’s own process, and the review above establishes that essentially nobody else has either. That is a gap this group is unusually well placed to fill, because it already has the two things such a study needs: readers, and tasks with the right shape.

The tasks with the right shape are the quantitative, threshold-gated ones — the same ones the group already works on. HER2 scoring, Ki-67 indices and tumour cell percentage are all continuous estimates cut at a decision boundary, all irreproducible between observers, and all candidates for exactly the split described above.

There is one immediate, free design consequence: show the model’s answer after the reader commits, not beside it. The mechanism measured here needs the reader’s prior view and the model’s view to be present simultaneously. Sequential presentation does not eliminate anchoring, but it makes the reader’s own answer recoverable, which is what turns a deployment into a measurable one.

How it connects

Rosbach 2026 — Cognitive Biases in AI-Assisted Medical Decision Making — establishes the denominator this page opens with: one primary pathology study in the whole indexed literature. Read it for the landscape, not for findings; the body is behind a paywall and unread.

Rosbach 2025 — Confirmation Bias and Time Pressure in AI-Assisted Tumour Cell Percentage Estimation — the confirmation-bias half of the single experiment, and the fuller method description of the two.

Rosbach 2025 — Automation Bias in AI-Assisted Tumour Cell Percentage Estimation under Time Pressure — the automation-bias half of the same experiment; the 7% negative-consultation rate and the two-metrics-opposite-signs result come from here.

Clusmann 2026 — Safety and Security of Large Language Models in Healthcare (Nature Review) — reaches the same two named hazards from LLM safety rather than from imaging decision support, and independently reports that oversight “fades into passive acceptance under time pressure”. Two literatures, two methods, one convergent finding — which is the strongest evidence on this page, because it is the only thing here not resting on the single 28-reader experiment.

Interobserver Agreement — the standard justification for putting a model beside a reader is that it raises agreement. This page is the reason to ask what else moved: agreement can rise because two readers converged on the model’s answer, which is a worse outcome than disagreement if the model was wrong.

Biomarker Cut Points — the threshold is what converts a continuous estimate into a decision, and it is where the continuous and thresholded views of the same result diverge in sign.

Model Abstention — the same denominator discipline: a rate is uninterpretable without the population it was computed over, and both pages exist because that gets skipped.

Diagnostic Test Accuracy — the positive/negative consultation split is a confusion-matrix-shaped way of scoring a decision aid, and it is more informative than the accuracy delta usually reported.

Laboratory Workload Measurement — the time-pressure results are the reason to care about workload as an experimental variable, but the source study’s ten-second countdown is explicitly not how clinical time pressure works; that page holds the right construct.

Agentic Slide Navigation — the other place in this wiki where a model shapes where a pathologist looks. There the influence is the point; here it is the hazard, and the two should be read against each other.

LLM Report-Quality Checking — an LLM scoring a report against a checklist is a decision-support system with a reader downstream, so the same failure modes apply to it.

Abe 2026 — Screening Efficiency Over Experience in Digital Cytology — eye-tracking study of 100 cytotechnologists showing that professional experience does not correlate with diagnostic accuracy (\(r = 0.189\)), while low-power field pop-out efficiency is the sole independent predictor (\(p = .045\)), directly addressing whether expertise protects against diagnostic error.

Open questions

Everything quantitative here rests on 28 readers in one experiment. There is no replication. The first useful contribution this group could make is a second measurement, on a different task, with the same design.
Does the effect survive a sequential interface — reader commits, then sees the model? The source design contains the unaided condition needed to test it but never compares presentation orders.
Does model calibration change the picture? Every result here comes from a detector that was accurate on about half the slides with no confidence signal shown. A model that says how sure it is might be resisted differently.
Does expertise protect? The Rosbach TCP cohort was heavily experienced (53.5% with \(>15\) years) but did not report a stratified analysis. In digital cytology, [[abe-2026-screening-efficiency-digital-cytology|Abe et al. 2026]] evaluated 100 practitioners across 1–40 years of experience and demonstrated that experience does not significantly correlate with diagnostic accuracy (\(r = 0.189, p = .0596\)); seniority primarily predicts cautious habits (longer inspection of clinical requisitions), whereas diagnostic performance is governed by low-power target detection efficiency and selective noise neglect.
What are the other nine of the review’s twelve biases? Not recoverable without the full text.
Does the presence of an explanation cause or reduce the effect? The source study always showed prototype-based explanations and has no bare-number arm, so the two are confounded throughout.