Research
Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
Published in Nature Medicine
Attributing and situating knowledge cannot be left to language models
Published in Nature Machine Intelligence
‘We can see a savage’: a case study of the colonial gaze in generative AI algorithms
Published in AI & Society
Gender trouble in language models: an empirical audit guided by gender performativity theory
Presented at ACM FAccT
A scaling law to model the effectiveness of identification techniques
Published in Nature Communications
Anonymization: The imperfect science of using data while preserving privacy
Published in Science Advances
Press
BBC NewsThe friendlier the AI chatbot the more inaccurate it is, study suggests
New study led by Lujain and Sofia finds that friendlier chatbots make more mistakes.
MashableStudy: Friendly AI chatbots may be less accurate
How does a friendlier chatbot respond to a falsehood about the moon landings?
NatureFriendlier LLMs tell users what they want to hear - even when it is wrong
A large language model that is trained to respond in a warm manner is more likely to give incorrect information and reinforce conspiracy beliefs.
The TelegraphWhy you don’t want your AI chatbot to be nice to you
Systems trained to sound friendlier are up to 30 per cent less accurate, study finds
The GuardianFriendly AI chatbots more likely to support conspiracy theories, study finds
Chatbots programmed to respond warmly even cast doubts on Apollo moon landings and fate of Hitler, researchers say.
The VergeFriendly chatbots make more mistakes
The researchers found AI chatbots trained to be warmer were significantly more likely to make factual errors and agree with false beliefs than the originals.
ScienceUK Biobank faces questions about data security after latest breach
Experts say the lapse highlights that even new measures to control access did not safeguard deidentified patient information; Q&A with Luc.
El PaísChinese website sells medical information of 500,000 volunteers from UK database
Luc comments on the UK Biobank data leak, with medical data put for sale on a Chinese website.
The GuardianWhat is the UK Biobank project and what are the privacy concerns around it?
Volunteers’ data has enabled medical breakthroughs, but there are questions over how that data is protected. Luc comments.
The TimesWhat is UK Biobank? Medical database that’s a victim of own success
The resource at the centre of a data breach in China is a treasure trove for scientists and has been responsible for a series of medical breakthroughs. Luc comments on Biobank data leaks.
The GuardianNow you can break up with big tech at a bar: ‘cybersecurity disguised as a party’
Luc comments on why people are turning away from big tech.
ARS TechnicaAmericans ask AI for health care. Hospitals think the answer is more chatbots.
Article cites our research which warns about the risks in AI chatbots giving medical advice.
De StandaardDr ChatGPT doesn’t help you any better than Dr Google, and that’s not because of the AI models’ ‘knowledge.’
New study led by Andrew warns of the risks in AI chatbots giving medical advice.
New York TimesHealth advice from AI chatbots is frequently wrong, study shows
New study led by Andrew warns of the risks in AI chatbots giving medical advice.
ReutersAI no better than other methods for patients seeking medical advice, study shows
New study led by Andrew warns of the risks in AI chatbots giving medical advice.
The RegisterAI chatbots are no better at medical advice than a search engine
A new study led by OII researchers warns of the risks in AI chatbots giving medical advice.
New York TimesFrustrated by the Medical System, Patients Turn to A.I
Chatbots are cheap, always available, superficially empathetic — and sometimes wrong. Some have concluded they’re a risk worth taking. Article references upcoming study led by Andrew.
de CorrespondentThe claims about increasingly smart AI models?
More vibe than science. Luc comments.
NBC NewsAI Revolution – NBC News discuss latest OII study exploring AI evaluation
The NBC Morning News programme discuss the findings from Andrew's latest study which finds weaknesses in how AI systems are evaluated.
The RegisterAI benchmarks are a bad joke - and LLM makers are the ones laughing
Covers our research finding that many AI benchmarks do not measure the right things.
GizmodoAI Capabilities May Be Overhyped on Bogus Benchmarks, Study Finds
Covers our Measuring What Matters study on the construct validity of AI benchmarks.
NBC NewsAI’s capabilities may be exaggerated by flawed tests, according to new study
Researchers behind a new study say that the methods used to evaluate AI systems’ capabilities routinely oversell AI performance and lack scientific rigour.
The GuardianExperts find flaws in hundreds of tests that check AI safety and effectiveness
Scientists say almost all have weaknesses in at least one area that can ‘undermine validity of resulting claims’ with commentary and latest research findings from Andrew.
Tech Policy PressWhy We Shouldn’t Trust Facial Recognition’s Glowing Test Scores
Failures in facial recognition technology are far from uncommon, and numerous examples continue to be reported in the press. Despite these repeated failures, the technology is rapidly being integrated into our daily lives, Juliette, Teo, and Luc write.
The Daily TelegraphChatGPT is driving people mad
In a recent research paper, academics at the Oxford Internet Institute found that AI systems producing “warmer” answers were also more receptive to conspiracy theories.
BMA The DoctorBot-ched advice – ‘disturbing’ results in AI study
Rebecca and Andrew comments on our study showing that LLM chatbots can perform worse when interacting with humans than when assessed using benchmarks.
Oxford Internet InstituteDo language models have an issue with gender?
Feature piece by Sofia about our study on how to best evaluate if language models perpetuate gender stereotypes.
TechCrunchPeople struggle to get useful health advice from chatbots, study finds
Coverage of Andrew's study showing that people using AI chatbots for medical self-diagnosis did not make better decisions than people using traditional sources.
Le SoirUne équipe de l’UCLouvain découvre une faille dans le RGPD : « Rester anonyme sur internet est presqu’impossible »
Coverage of our research showing that data considered anonymous can still identify web users.
University of OxfordPioneering new mathematical model could help protect privacy and ensure safer use of AI
University coverage of our scaling law for identification techniques and privacy risks.