What the research found
Researchers narrowed 40 patient questions about asthma in pregnancy to 19 and put each once to six consumer AI models, including GPT-5.5, Claude Sonnet 5, Gemini 3.5 Flash, DeepSeek-V3, Qwen and Doubao, with clinicians rating the 114 answers for quality, readability and safety. After adjudication, 37 answers (32.5%) had at least one safety concern and 4 (3.5%) were rated high-risk unsafe advice; quality varied significantly between models, and none wrote at a median reading level of grade 8 or below. The authors say the results do not establish safety for unsupervised patient use.
Pregnant patients with asthma are a group where a confident but wrong answer about medication carries risk for two people, so chatbot advice needs a clinician's check.