71% → 38%
mean accuracy after one false sentence
What the research found
MedMisBench tested 11 leading models — including GPT-5, Gemini and Claude — on roughly 11,000 real medical questions. When a single plausible-but-false sentence was injected alongside a question the model had answered correctly, mean accuracy collapsed from 71% to 38%, and more than half of correct answers flipped to wrong. The models had the knowledge; the context steered them away from it.
Why it matters for providers
Real patients and clinicians rarely give clean prompts. This shows any AI can be pushed to a confidently wrong answer by one misleading detail — the exact condition of everyday use.
Original source
Oxford et al. — MedMisBench (arXiv / bioRxiv preprint)
A reputable institutional report or news outlet.