A scaling law to model the effectiveness of identification techniques
Why do AI identification systems perform well in lab tests but fail in practice? We propose a scaling law to forecast the privacy risks posed by identification techniques, and to support independent accountability efforts for AI-based biometric systems.
Authors
Luc Rocher , Julien M. Hendrickx, Yves-Alexandre de Montjoye
Published
2025
Our method provides a scientific framework for evaluating identification techniques, in particular to expose how they may fail when deployed at scale. It could, for instance, be used to examine the accuracy of advertising trackers and browser fingerprinting, two techniques that harvest small pieces of information such as time zone or browser settings to identify and track users, often without meaningful consent.
We developed this approach to assess the real-world risks of re-identification in data release, and to critically evaluate identification techniques being deployed in high-stakes environments. In hospitals, humanitarian aid delivery, and border control, identification errors carry severe consequences. Yet these systems are often deployed with insufficient scrutiny of how they perform beyond controlled test conditions.
The method uses Bayesian statistics to estimate how identifiable individuals are on a small scale and extrapolate to larger populations. We find that many identification techniques perform impressively in small-scale studies but degrade significantly under real-world conditions, where populations are larger and more diverse. This gap between lab performance and deployment reality is often underestimated or ignored.
Voice recognition in banking, iris scanning in humanitarian contexts, and facial recognition in law enforcement are being deployed at scale, often with little public debate about their necessity or proportionality. While our method can help organisations identify weaknesses before full-scale deployment, it should not be seen as a pathway to making invasive surveillance systems acceptable.
Our scaling law offers, for the first time, a principled model to evaluate how identification techniques perform at scale. This understanding is essential for assessing the risks of re-identification and ensuring compliance with data protection legislation, as well as also for informing broader debates about the increasing deployment of identification techniques.