How private is anonymised data? Count the noise, not the promise

Latest Model, AI-assisted writing · 2026-10-29

Overview

This question came from the site's owner: how do you anonymise data so a model can still learn from it? The honest answer has two halves. Deleting names and emails is not anonymisation, because a few ordinary details are enough to pick most people out. What works is adding a measured amount of noise, and the price of that noise, in accuracy, is something you can compute exactly before you share anything.

Why removing names is not enough

In 2000 Latanya Sweeney used census figures to estimate that 87% of the US population, 216 million of 248 million people, was likely unique by five-digit ZIP code, sex and date of birth [2]. Using the town instead of the ZIP code still left about half unique [2]. A later re-analysis by Philippe Golle, with census data of the same kind, found 61% for 1990 and 63% for 2000, and could not explain the difference [3].

The exact share matters less than its size. Three columns that look harmless single out most people, so any dataset that keeps them, or anything like them, can be joined to a public list and re-identified. Coarsening helps: a year of birth instead of a date, the first three digits of a ZIP code. It also throws away exactly the detail a model might need, and there is no clean point where it becomes safe.

Noise you can measure

Differential privacy takes the other road. Instead of hiding who is in the data, it adds random noise to every answer the data gives, sized so that no single person's presence changes the answer much. The size is set by one number, ε (epsilon): smaller ε means more privacy and more noise [4].

For a count, such as how many people in the data have some condition, the standard Laplace mechanism adds noise whose 95% range is ± ln(20) / ε, about ±3 / ε [1] [5]:

ε	Noise scale	95% of answers within	On a count of 100
0.1	10	±30	±30%
0.5	2	±6	±6%
1	1	±3	±3%
2	0.5	±1.5	±1.5%

The noise is the same size whatever the count, so it matters for small groups and vanishes for large ones. At ε = 1, a count of 20 is off by up to 15%, a count of 10,000 by 0.03% [1]. Privacy is cheap for questions about many people and expensive for questions about few, which is the right way round.

The catch: every question spends privacy

Each answer leaks a little, and the leaks add up. If you allow a total ε of 1 and split it over ten questions, each question gets ε = 0.1, and its noise grows tenfold to ±30 [1]. Over a hundred questions it is ±300 [1]. This is why NIST's 2025 guidelines tell anyone evaluating a differential privacy product to ask how the total budget is set and spent, not only what ε a single query uses [4].

For machine learning the same arithmetic applies to training: a model trained with differential privacy spends its budget across all its training steps, and pays for it in accuracy, most on the rare cases.

Nerd corner

A mechanism is ε-differentially private when adding or removing any one person changes the probability of any output by at most a factor of e^ε [4] [5]. For a count, one person changes the true answer by at most 1, and Laplace noise with scale b = 1 / ε meets the definition [5].

The Laplace distribution's tails fall as e^(−t/b), so P(|noise| > t) = e^(−t/b), and the 95% range is t = b × ln(20) ≈ 3.0 b [1]. A simulation of 200,000 draws gives 3.00 at ε = 1 and 29.9 at ε = 0.1 [1]. Splitting the budget over k questions multiplies each error by k under basic composition; tighter accounting methods do better, which is one of the questions the NIST guidelines tell you to ask a vendor [4].

Check yourself

You release a dataset with names removed but ages, towns and job titles kept. Is it anonymous? Probably not: a few ordinary columns single out most people, as the ZIP, sex and birth date studies showed [2] [3].

At ε = 1, a count comes back as 47. What can you say about the true count? It is between 44 and 50 for 95% of answers [1].

Why does the same ε protect large groups better than small ones? It does not protect them better; it costs them less. The noise is the same size, so it is a smaller share of a larger count.

A notebook entry for latestlmodel.com, written with AI and checked by hand. The re-identification figures are quoted from the two studies, which disagree; the noise figures are computed exactly and confirmed by simulation.