Healthcare AI Safety Gap · Don't break patients
    Peer-reviewed / official13 Jul 2026·Spain / Intl

    Physicians struggle to override AI's mistakes, study finds

    PLOS Digital Health: doctors kept trusting incorrect AI patient classifications even after outcome data proved them wrong.

    78% vs 37%
    prescribing-rate gap for a drug that did nothing at all, driven purely by the AI's (wrong) label

    What the research found

    Researchers at the University of the Basque Country had 223 self-reported physicians decide whether to give a fictitious drug to patients an AI system had (wrongly) sorted into "more" or "less" treatment-sensitive groups. In a second experiment the drug did nothing at all — recovery rates were identical with or without it — yet physicians still prescribed it to 78% of patients labelled high-sensitivity versus 37% of those labelled low-sensitivity, and judged it effective. Given direct, patient-by-patient outcome evidence that contradicted the AI, most doctors still couldn't revise their judgement.

    Why it matters for providers

    Confirms that "keep a human in the loop" only works if the human can actually catch the AI when it's wrong — and this shows that's much harder than it sounds, even with disconfirming outcome evidence in hand.

    Original source
    PLOS Digital Health
    Read the full report ↗
    Peer-reviewed or official source — the most reliable tier.