Research
Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
Published in Nature Medicine
Press
ARS TechnicaAmericans ask AI for health care. Hospitals think the answer is more chatbots.
Article cites our research which warns about the risks in AI chatbots giving medical advice.
De StandaardDr ChatGPT doesn’t help you any better than Dr Google, and that’s not because of the AI models’ ‘knowledge.’
New study led by Andrew warns of the risks in AI chatbots giving medical advice.
New York TimesHealth advice from AI chatbots is frequently wrong, study shows
New study led by Andrew warns of the risks in AI chatbots giving medical advice.
ReutersAI no better than other methods for patients seeking medical advice, study shows
New study led by Andrew warns of the risks in AI chatbots giving medical advice.
The RegisterAI chatbots are no better at medical advice than a search engine
A new study led by OII researchers warns of the risks in AI chatbots giving medical advice.
New York TimesFrustrated by the Medical System, Patients Turn to A.I
Chatbots are cheap, always available, superficially empathetic — and sometimes wrong. Some have concluded they’re a risk worth taking. Article references upcoming study led by Andrew.
de CorrespondentThe claims about increasingly smart AI models?
More vibe than science. Luc comments.
NBC NewsAI Revolution – NBC News discuss latest OII study exploring AI evaluation
The NBC Morning News programme discuss the findings from Andrew's latest study which finds weaknesses in how AI systems are evaluated.
The RegisterAI benchmarks are a bad joke - and LLM makers are the ones laughing
Covers our research finding that many AI benchmarks do not measure the right things.
GizmodoAI Capabilities May Be Overhyped on Bogus Benchmarks, Study Finds
Covers our Measuring What Matters study on the construct validity of AI benchmarks.
NBC NewsAI’s capabilities may be exaggerated by flawed tests, according to new study
Researchers behind a new study say that the methods used to evaluate AI systems’ capabilities routinely oversell AI performance and lack scientific rigour.
The GuardianExperts find flaws in hundreds of tests that check AI safety and effectiveness
Scientists say almost all have weaknesses in at least one area that can ‘undermine validity of resulting claims’ with commentary and latest research findings from Andrew.
BMA The DoctorBot-ched advice – ‘disturbing’ results in AI study
Rebecca and Andrew comments on our study showing that LLM chatbots can perform worse when interacting with humans than when assessed using benchmarks.
TechCrunchPeople struggle to get useful health advice from chatbots, study finds
Coverage of Andrew's study showing that people using AI chatbots for medical self-diagnosis did not make better decisions than people using traditional sources.