Computational Privacy Group - Imperial College London

As AI systems become increasingly powerful, their integration into everyday life will bring substantial benefits - but also raise pressing concerns around privacy, safety, and beyond. The Computational Privacy Group aims to provide leadership, in the UK and internationally, in the privacy-preserving, safe, and ethical use of AI systems, ranging from synthetic data and LLMs to agentic systems.

We primarily consider an adversarial perspective to identify and quantify vulnerabilities, which we believe to be a critical foundation for developing safe, secure and privacy-preserving systems. Our research has studied the limits of anonymization, demonstrated how machine learning models can leak sensitive data, and identified safety vulnerabilities in AI systems.

While technical in nature, our work has had significant public policy implications, informing for instance the International AI Safety Report, reports of the United Nations, FTC, the European Commission as well as in briefs to the U.S. Supreme Court.

Research Areas

Privacy & confidentiality

Personal data is increasingly used to train AI models, power intelligent agents, and generate synthetic data, as well as to enable privacy-preserving mechanisms for statistical releases. As these systems are deployed across sensitive domains like healthcare and finance, understanding and mitigating the privacy risks they pose become critical.

We have studied membership inference attacks (MIAs) against synthetic data [1,2,3], image classifiers [4], and LLMs [5,6], as well as employed evolutionary search to uncover weaknesses in query-based systems [7,8]. More recently, we have also studied more fundamental memorization in ML models, quantifying how individual training examples affect memorization of other samples [9], and have developed more effective ways to quantify and estimate privacy leakage [4].

Security & safety

We take an adversarial approach to explore the many safety and security challenges that come with AI systems. Our security research covers multiple angles of AI system vulnerabilities, with an eye toward future risks as AI capabilities expand.

To audit safety risks of AI systems, we have used strong adversaries to develop and analyse threats such as jailbreaks and prompt injection attacks [10]. We also studied the safety in perceptual hashing algorithms for client-side scanning [11,12] and examined how LLMs memorize adversarially crafted training data [13]. A major focus is the emerging security challenges of agentic AI systems. We investigate new attack vectors and scenarios for misuse introduced by developments like the Model Context Protocol (MCP), which enables AI assistants to connect with external data sources and tools. We are particularly interested in how these risks evolve as agents become more capable and deploy at scale.

Societal impact of AI

Our research is grounded in the belief that technical advances in AI and data science must be matched by careful consideration of their societal implications. We hence regularly investigate how modern technologies influence broader systems of accountability, fairness, and trust.

For example, we have studied the potential for collusion in algorithmic markets [14] and examined how copyright traps can be used to detect unauthorized use of protected content in AI training [13]. By exploring these intersections of technology and society, we aim to inform responsible development and contribute to ongoing policy and regulatory discussions.

News and Events

Best paper award at SaTML 2025

Apr 15, 2025

We were lucky enough to win the Best paper Award at SaTML 2025 with our Systemization of Knowledge (SoK) on Membership Inference Attacks (MIAs) against LLMs.

Yves-Alexandre de Montjoye presenting at ELSA Workshop

Mar 17, 2025

Yves-Alexandre de Montjoye will be presenting at the ELSA Workshop on Privacy-Preserving Machine Learning taking place on 17-21 March, 2025. The workshop brings together researchers and practitioners to discuss recent developments in privacy-preserving machine learning techniques …

Yves-Alexandre de Montjoye at Dagstuhl Seminar

Mar 14, 2025

Yves-Alexandre de Montjoye was invited to give a talk at the Dagstuhl Seminar "PETs and AI: Privacy Washing and the Need for a PETs Evaluation Framework."

More News

Selected publications

Yang, X., Stevanoski, B., Meeus, M. and de Montjoye Y. A. Alignment Under Pressure: The Case for Informed Adversaries When Evaluating LLM Defenses. ArXiv preprint (2025).
Yao, Z., Krčo, N., Ganev, G. and de Montjoye Y. A. The DCR Delusion: Measuring the Privacy Risk of Synthetic Data. ArXiv preprint (2025).
Mao, Y., Stevanoski, B. and de Montjoye Y. A. DeSIA: Attribute Inference Attacks Against Limited Fixed Aggregate Statistics. ArXiv preprint (2025).
Meeus, M., Shilov, I., Jain, S., Faysse, M., Rei, M. and de Montjoye Y. A. SoK: Membership Inference Attacks on LLMs are Rushing Nowhere (and How to Fix It). IEEE Conference on Secure and Trustworthy Machine Learning (SaTML) (2025).
Pollock, J., Shilov, I., Dodd, E. and de Montjoye Y. A. Free Record-Level Privacy Risk Evaluation Through Artifact-Based Methodsa. 34th USENIX Security Symposium (USENIX Security 2025) (2025).
Stevanoski, B., Cretu, A.-M., and de Montjoye Y. A. QueryCheetah: Fast Automated Discovery of Attribute Inference Attacks Against Query-Based Systems. ACM Conference on Computer and Communications Security (ACM CCS 2024) (2024).
Meeus, M., Jain, S., Rei, M. and de Montjoye Y. A. Did the Neurons Read your Book? Document-level Membership Inference for Large Language Models. 33rd USENIX Security Symposium (USENIX Security 2024) (2024).
Selected Press: Le Monde
Meeus, M., Shilov, I., Faysse, M. and de Montjoye Y. A. Copyright Traps for Large Language Models. 41st International Conference on Machine Learning (ICML 2024) (2024).
Selected Press: MIT Technology Review, Nature News
Gadotti, A., Rocher, L., Houssiau, F., Cretu, A.-M., and de Montjoye Y. A. Anonymization: The imperfect science of using data while preserving privacy. Science Advances, 2024 (2024).
Cretu, A.-M., Jones, Daniel, de Montjoye Y. A, and Tople, Shruti. Investigating the Effect of Misalignment on Membership Privacy in the White-box Setting. In Proceedings on Privacy Enhancing Technologies 2024(3), 407–430. (2024).
Guan, V., Guépin, F., Cretu, A.-M., and de Montjoye Y. A. A Zero Auxiliary Knowledge Membership Inference Attack on Aggregate Location Data. Proceedings on Privacy Enhancing Technologies 2024(4) (PoPETS 2024) (2024).
Cretu, A.-M., Guépin, F., and de Montjoye Y. A. Correlation inference attacks against machine learning models. Science Advances, 2024 (2024).
Cretu, A.-M.*, Rusu, Miruna*, and de Montjoye Y. A. Re-pseudonymization Strategies for Smart Meter Data Are Not Robust to Deep Learning Profiling Attacks. In Proceedings of the Fourteenth ACM Conference on Data and Application Security and Privacy (CODASPY ’24), June 19–21, 2024, Porto, Portugal. ACM, New York, NY, USA. (2024).
Meeus, M., Shilov, I., and de Montjoye Y. A. Mosaic Memory: Fuzzy Duplication in Copyright Traps for Large Language Models. ArXiv preprint (2024).
Guépin, F., Krčo, N., Meeus, M. and de Montjoye Y. A. Lost in the Averages: A New Specific Setup to Evaluate Membership Inference Attacks Against Machine Learning Models. ArXiv preprint (2024).
Guépin, F., Meeus, M., Cretu, A.-M., and de Montjoye Y. A. Synthetic is all you need: removing the auxiliary data assumption for membership inference attacks against synthetic data. 18th DPM International Workshop on Data Privacy Management, Sept 2023, The Hague (2023).
Meeus, M., Guépin, F., Cretu, A.-M., and de Montjoye Y. A. Achilles’ Heels: Vulnerable Record Identification in Synthetic Data Publishing. 28th European Symposium on Research in Computer Security (ESORICS), Sept 2023, The Hague (2023).
Meeus, M., Jain, S., and de Montjoye Y. A. Concerns about using a digital mask to safeguard patient privacy. Matters Arising in Nature Medicine, July 2023 (2023).
Jain, S., Cretu, A.-M., Cully, A. and de Montjoye Y. A. Deep perceptual hashing algorithms with hidden dual purpose: when client-side scanning does facial recognition. 2023 IEEE Symposium on Security and Privacy (SP) (2023).
Selected Press: Imperial College London News
Rocher, L., Tournier, A. J., & de Montjoye, Y. A. Adversarial competition and collusion in algorithmic markets. Nature Machine Intelligence (2023).
Selected Press: POLITICO Pro Fair Play, POLITICO Pro Morning Tech
Houssiau, F., Liénart, T., Hendrickx, J. and de Montjoye Y. A. Web privacy: a Formal Adversarial Model for Query Obfuscation. IEEE Transactions on Information Forensics and Security (2023).
Houssiau, F., Sapieżyński, P., Radaelli, L., Shmueli, E. and de Montjoye Y. A. Detrimental network effects in privacy: A graph-theoretic model for node-based intrusions. Patterns, 4(1) (2023).
Houssiau, F., Schellekens, V., Chatalic, A., Annamraju, S. K. and de Montjoye Y. A. M2M: A General Method to Perform Various Data Analysis Tasks from a Differentially Private Sketch. The 18th International Workshop on Security and Trust Management (2022).
Cretu, A.-M.*, Houssiau, F.*, Cully, A., and de Montjoye Y. A. QuerySnout: Automating the Discovery of Attribute Inference Attacks against Query-Based Systems. In Proceedings of the 2022 ACM SIGSAC Conference on Computer and Communications Security (CCS '22). Association for Computing Machinery, New York, NY, USA, 623–637. (2022).
Selected Press: Imperial College London
Tournier, A. J., & de Montjoye, Y. A. Expanding the attack surface: Robust profiling attacks threaten the privacy of sparse behavioral data. Science Advances (2022).
Jain, S.*, Cretu, A.-M.*, and de Montjoye Y. A. Adversarial Detection Avoidance Attacks: Evaluating the robustness of perceptual hashing-based client-side scanning. 31st USENIX Security Symposium (2022).
Selected Press: Imperial College London News
Gadotti A., Houssiau F., Annamalai M.S.M.S., & de Montjoye, Y. A. Pool Inference Attacks on Local Differential Privacy: Quantifying the Privacy Guarantees of Apple's Count Mean Sketch in Practice. 31st USENIX Security Symposium (2022).
Cretu, A.-M., Monti, F., Marrone, S., Dong, X., Bronstein, M. and de Montjoye Y. A. Interaction data are identifiable even across long periods of time. Nature Communications (13), 313 (2022).
Selected Press: Science News, RFI
Houssiau, F., Rocher, L. and de Montjoye Y. A. On the difficulty of achieving Differential Privacy in practice: user-level guarantees in aggregate location data. Nature Communications, 2022 (2022).
Gadotti A., Houssiau F., Rocher L., Livshits B., de Montjoye Y. A. When the signal is in the noise: Exploiting Diffix's Sticky Noise. 28th USENIX Security Symposium (2019).
Selected Press: TechCrunch, Wall Street Journal
Rocher, L., Hendrickx, J. M., & de Montjoye, Y. A. Estimating the success of re-identifications in incomplete datasets using generative models. Nature communications, 10 (1), 3069 (2019).
Selected Press: New York Times, Guardian, CNBC, The Telegraph, TechCrunch, Technology Review, New Scientist, Gizmodo, Scientific American, RT, Forbes, El Pais (ES), Sueddeutsche Zeitung (DE), Le Soir (FR), La Libre (FR), L'Echo (FR), De Morgen (NL)
de Montjoye Y. A., Radaelli L., Singh V. K., Pentland A., Unique in the shopping mall: On the reidentifiability of credit card metadata. Science 347 (6221), 536-539. DOI:10.1126/science.1256297 (2015).
Selected Press: New York Times, Wall Street Journal (1, 2), BBC, Harvard Business Review, Nature, Technology Review, Le Monde (FR), Die Zeit (DE), Die Spiegel (DE), El Pais (ES), The Hill, Les Echos (FR), Scientific American, New Scientist, Five Thirty Eight
de Montjoye Y. A., Shmueli E., Wang S., Pentland A., openPDS: Protecting the Privacy of Metadata through SafeAnswers. PLoS One, 10.1371 (2014).
Selected Press: BBC, New-York Times, Wall Street Journal, Technology Review, World Economic Forum, Real Time with Bill Maher (HBO), Le Monde (FR), MIT News, Wired (UK), New Scientist, Baratunde for Fast Company, GigaOM, Scientific American (1, 2)
de Montjoye Y.-A.*, Quoidbach J.*, Robic F.*, Pentland A., Predicting people personality using novel mobile phone-based metrics. International Conference on Social Computing, Behavioral-Cultural Modeling, & Prediction, Washington, USA (2013).
Selected Press: Le Monde (FR), RT, Boston Globe, Mediapart (FR), Radio Canada (FR)
de Montjoye Y.-A., Hidalgo C. A., Verleysen M., Blondel V. D., Unique in the Crowd: The privacy bounds of human mobility. Nature srep. 3, 1376; DOI:10.1038/srep01376 (2013).
Selected Press: BBC, CNN, Wall Street Journal, New-York Times ( 1, 2), Le Monde (FR) (1, 2), Guardian (UK), Technology Review, Nature, Wired (US, UK, JP), Bruce Schneier, Der Spiegel (DE), Die Welt (DE) (1, 2), Boing Boing, Fast Company, PopSci, GigaOM, TechDirt

The full list of our papers is available on Google Scholar.

Our Team

Yves-Alexandre de Montjoye

Group leader

Yves-Alexandre is an Associate Professor at Imperial College London. He received his PhD from MIT before joining Harvard IQSS for his postdoc. He currently is a Special Adviser on AI and Data Protection to EC Justice Commissioner Reynders and a Parliament-appointed expert to the Belgian Data Protection Agency (APD-GBA).

Euodia Dodd

PhD student

Euodia obtained her BSc in Computer Science from The University of Sheffield, and an MPhil in Advanced Computer Science from the University of Cambridge in 2022. She then spent two years at Goldman Sachs in the Applied AI team. Her interests include privacy attacks on ML systems, privacy-preserving ML, and interpretability.

Nataša Krčo

PhD student

Nataša obtained her BSc degree in Computer Science at the University of Novi Sad in Serbia. After that, she obtained her MSc degree from EPFL. During her time there, she conducted research both in academic and industry settings, on responsible AI. Her research interests include explainable AI, algorithmic fairness, and privacy-preserving ML.

Yifeng Mao

PhD student

Originally from China, Yifeng obtained his BSc from Tsinghua University in Automation. After his master degree, he spent one year working as a full-time research assistant in the US. His research interests include privacy-preserving ML, synthetic data generation protecting personal anonymity, and attack methods against census datasets and ML models.

Matthieu Meeus

PhD student

Matthieu obtained his BSc from KU Leuven in Mechanical Engineering. Next, he spent four years in the US, pursuing two years of graduate study (Energy, Computer Science) and two years of working as data scientist at McKinsey & Company. His research interests include privacy attacks against (large) language models and ML systems.

Igor Shilov

PhD student

Igor obtained his undergraduate degree in Computer Science in 2013 and had been working as a Software Engineer since, most recently at Meta AI. His research interests include differential privacy and privacy attacks against ML systems.

Bozhidar Stevanoski

PhD student

Bozhidar has obtained his MSc in Data Science at the University of Ljubljana and his BSc in Computer Science at the Ss. Cyril and Methodius University in Skopje. His research interests include machine learning and privacy attacks against query-based systems.

Xiaoxue Yang

PhD student

Xiaoxue holds a BSc in Statistics/Economics from the University of Toronto and an MSc in Artificial Intelligence from Imperial College. Before her MSc, she worked as a technology consultant and later as a data scientist in Deloitte’s management consulting practice. Her research interests include privacy attacks against ML models and data releases such as synthetic data.

Zexi Yao

PhD student

Zexi is from Singapore, and obtained his BScs in Computer Science as well as Electrical and Computer Engineering from Carnegie Mellon University. He then spent 2 years as a research engineer at A*STAR in Singapore. His research interests include secure computing, distributed machine learning and privacy preserving techniques in machine learning.

Click here for more information on our team. →

Contact

Email:X@Y where X=demontjoye, Y=imperial.ac.uk.

We are located at the Data Science Institute in the William Penney Laboratory. The best entry point is via Exhibition road, through the Business school (see map below). From there, just take the stairs towards the outdoor court. Enter the outdoor corridor after the court and the institute will be on your right (please press the Data Science intercom button for access).

Please address mails to:
Department of Computing
Imperial College London
180 Queens Gate
London SW7 2AZ

Somewhere