Rosbach 2025 — Automation Bias in AI-Assisted Tumour Cell Percentage Estimation under Time Pressure

The same 28-pathologist experiment as the confirmation-bias paper, re-analysed for the opposite failure: AI assistance improved average accuracy, and in 7% of assessments it converted a correct answer into a wrong one — a rate unchanged by time pressure, though the damage per incident roughly doubled.
Author

Rosbach E, Ganz J, Ammeling J, Riener A, Aubreville M. Technische Hochschule Ingolstadt, Ingolstadt, Germany (Rosbach, Ganz, Ammeling, Riener); Flensburg University of Applied Sciences, Flensburg, Germany (Aubreville)

Doi

10.1007/978-3-658-47422-5_27

The same 28-pathologist experiment as the confirmation-bias paper, re-analysed for the opposite failure: AI assistance improved average accuracy, and in 7% of assessments it converted a correct answer into a wrong one — a rate unchanged by time pressure, though the damage per incident roughly doubled.

Read status: full text, from the arXiv v1 preprint rather than the Springer chapter.

This is the same experiment as Rosbach 2025 — Confirmation Bias and Time Pressure in AI-Assisted Tumour Cell Percentage Estimation. Identical participants (31 → 28), identical 20 H&E patches from the same three public datasets, identical FCOS detector trained on BreCaHad, identical ten-second countdown, identical two-week washout, and several identical statistics — the JAS pair 0.49/0.55 and its t(27) = −2.80 appear in both papers. Do not read the 7% here and the 0.61 slope there as two studies agreeing. They are one dataset, two dependent variables, five authors versus twenty-seven, and two venues four weeks apart.

What is different from the companion paper

Only the question and the dependent variable. Confirmation bias asks whose answer wins when the machine agrees with you. Automation bias asks how often do you follow the machine when it is wrong and you were right — the commission error, following Goddard’s framing. The authors argue, persuasively, that measuring it as negative consultations isolates automation bias from every other bias, because it counts only the cases where AI exposure destroyed a correct answer.

To count that on a continuous estimate you need a correctness rule, and here is where the design choice sits: TCP estimates are binarised at 25%, the tumour-DNA threshold for direct sequencing. An estimate is “correct” if it lands on the same side of 25% as the ground truth.

  • Negative consultation = baseline correct → AI prediction wrong → AI-aided answer wrong.
  • Positive consultation = baseline wrong → AI-aided answer correct.

Results

AI helped, on average. Mean absolute deviation from ground truth |Dev_GT|:

No time pressure With time pressure
Baseline (unaided) 13.55 ± 4.81 15.29 ± 5.00
AI-aided 11.74 12.23

Two-way ANOVA on |Dev_GT|: main effect of AI F(1, 108) = 8.32, p < .01; time pressure p = 0.19; interaction p = 0.46. Within the unaided baseline alone, time pressure significantly worsened performance (one-tailed paired t, t(27) = −1.76, p = 0.045); the same trend under AI assistance was not significant.

Reliance rose under time pressure. JAS 0.49 ± 0.13 → 0.55 ± 0.12, t(27) = −2.80, p < .01 — the identical result reported in the companion paper.

The automation-bias count. Of 560 AI-aided estimates (28 × 20), participants adopted AI advice that contradicted their own baseline, across the 25% threshold, in only 67 cases:

  • 29 positive consultations — the AI corrected a previously wrong answer.
  • 38 negative consultations — the AI destroyed a previously correct answer.

38/560 ≈ 7%, which the authors call automation bias “in its purest form” and which sits inside Goddard’s reported 6–11% range for negative-consultation acceptance.

Time pressure did not change the rate but appears to change the damage (descriptive only — normality failed after filtering, so no test was run):

Within the 38 negative consultations No TP (n=19) With TP (n=19)
Frequency 19/280 ≈ 7% 19/280 ≈ 7%
Dev_GT
JAS 0.58 ± 0.32 0.65 ± 0.25

H1 fully accepted; H2 partially accepted — severity yes, frequency no.

The result the paper does not state, and it is the most useful one

The two metrics disagree in sign on the same data. Counted across the 25% threshold, the AI’s net effect on the cases where it was followed against prior judgement was harmful: 38 new errors against 29 corrections. Measured as mean absolute deviation, the AI’s net effect was beneficial: |Dev_GT| fell by roughly 2–3 percentage points in both time conditions, a significant main effect.

Both are true, and they are not in conflict — the continuous improvement comes from many small refinements while the threshold crossings are rare and asymmetric. But it means the answer to “did the AI help?” is determined by whether you score the estimate or the decision it feeds. For TCP that is not academic: nobody acts on a tumour percentage, they act on whether the sample is above the assay threshold. On the metric that matches the clinical action, the tally was negative. The paper reports both numbers and draws no line between them; this is the line.

Reading it critically

  1. The ANOVA’s degrees of freedom do not match a repeated-measures design. The paper says “two-way repeated measures ANOVA” and reports F(1, 108). With 28 subjects in a 2×2 within-subject design, the error term for a within-subject main effect should be around df = 27. But 28 subjects × 4 conditions = 112 observations, and 112 − 3 factor df − 1 = 108 — exactly the error df of a between-subjects two-way ANOVA that treats all 112 condition-means as independent. If that is what was run, the four measurements from each participant were not treated as repeated, the effective sample is inflated roughly fourfold, and p < .01 for the AI main effect is anticonservative. The paired t-tests elsewhere in the paper are correctly specified with t(27), which makes the ANOVA the odd one out. [unverified] — this is inferred from the reported df, not from code. It is checkable against the released analysis scripts, and it is the first thing to check.
  2. The 7% denominator is every AI-aided assessment, not every opportunity. A negative consultation is only possible when the participant was initially correct and the AI was wrong on the threshold. Most of the 560 were not such cases. The rate per opportunity — the quantity a reader intuitively wants, “when the AI was wrong and I was right, how often did I follow it?” — is necessarily higher and is not reported. The comparison to Goddard’s 6–11% is only valid if that literature used the same denominator, which is not established here. This wiki has the same problem recorded on Model Abstention: the denominator is a choice, and it belongs beside the metric.
  3. Everything rests on one threshold at 25%. A case estimated at 24% against a ground truth of 26% is scored as a full error; a case at 5% against 60% and a case at 24% against 26% count the same. Different assays use different minimum tumour fractions, so a different threshold would produce a different 7%. The sensitivity of the headline number to that choice is not examined. See Biomarker Cut Points.
  4. The severity result is 19 observations against 19, untested, with standard deviations larger than the difference. 19.42 ± 17.37 versus 27.79 ± 20.71 overlap heavily. The authors are appropriately careful — they state that normality assumptions failed and confine themselves to descriptive statistics — but “time pressure exacerbates the severity of automation bias” is a stronger sentence than these numbers support, and it is the sentence in the abstract.
  5. The headline finding is arguably the negative one. Only 67 of 560 assessments involved following the AI against one’s own prior view at all. The paper says it plainly — pathologists “were largely unwilling to adopt model recommendations that contradicted their initial judgments, regardless of the correctness of the system’s output” — and concludes automation bias may not be the primary bias here. That is consistent with the companion paper’s confirmation-bias result and is the more robust conclusion of the pair: these readers were not credulous; they were selective, and selectivity is the bias.
  6. Sample size, the artificial ten-second timer, the omission of clinical context, and the dataset-annotation ground truth are shared with the companion paper and are limitations there too. The authors list them.

What it would change here

  • If this group runs an AI-assisted scoring study, count both consultation directions. Reporting only “accuracy improved” would have hidden the 38-against-29 tally entirely.
  • Report the metric that matches the clinical action. For any thresholded biomarker — TCP for sequencing eligibility, Ki-67, HER2 — a continuous accuracy gain and a decision-level loss can coexist, and only one of them reaches the patient.
  • The negative-consultation rate is a cheap, reusable safety metric for any decision-support deployment where an unaided baseline exists.

How it connects

Open questions

  • What error df does the released analysis script actually produce? Settles point 1 either way. The anonymised repository is said to contain it.
  • What is the negative-consultation rate per opportunity? Computable from the released data.
  • How does the 7% move if the threshold is 10%, 20% or 50% rather than 25%?
  • Does the Springer chapter differ from the preprint? Not checked. [unverified]