Healthcare AI Safety Gap · Don't break patients
    Peer-reviewed / official13 Jul 2026·Spain / Intl

    Physicians struggle to override AI's mistakes, study finds

    PLOS Digital Health: doctors kept trusting incorrect AI patient classifications even after outcome data proved them wrong.

    78% vs 37%
    prescribing-rate gap for a drug that did nothing at all, driven purely by the AI's (wrong) label

    What the research found

    Researchers at the University of the Basque Country had 223 self-reported physicians decide whether to give a fictitious drug to patients an AI system had (wrongly) sorted into "more" or "less" treatment-sensitive groups. In a second experiment the drug did nothing at all (recovery rates were identical with or without it), yet physicians still prescribed it to 78% of patients labelled high-sensitivity versus 37% of those labelled low-sensitivity, and judged it effective. Given direct, patient-by-patient outcome evidence that contradicted the AI, most doctors still couldn't revise their judgement.

    Why it matters for providers

    Confirms that "keep a human in the loop" only works if the human can actually catch the AI when it's wrong, and this shows that's much harder than it sounds, even with disconfirming outcome evidence in hand.

    Original source
    PLOS Digital Health
    Read the full report ↗
    Peer-reviewed or official source, the most reliable tier.