Do AI Models Really Need to Forget Everything? Apple Researchers Say No
A new study finds that much of what we ask AI to 'unlearn' barely mattered to the model in the first place, which could cut the cost of privacy fixes dramatically.
Key points
- Apple ML Research identified subsets of training data so low in influence that removing them may be unnecessary.
- Current AI unlearning methods treat every piece of data equally, even when some had almost no effect on the model's behaviour.
- Skipping removal of these low-impact data points could reduce the computing time and cost of privacy-related model updates.
- The findings apply to both language models (the kind that power chatbots) and vision models (AI that analyses images).
When you ask a company to delete your data, you probably assume the AI they trained on it will change. Researchers at Apple ML Research now suggest that assumption is often wrong, and that gap could save a lot of money.
AI models learn by processing enormous amounts of data. After training, some of that data leaves a strong fingerprint on how the model behaves. Some barely registers at all. This new research focuses on that second category, and it's the follow-up to Apple ML Research work we've covered twice this week.
Machine unlearning means teaching a trained AI model to behave as if it never saw a specific piece of data. Privacy laws in several countries already push companies toward this capability. The problem is cost: current unlearning methods treat every item in the forget list the same way, whether that item shaped the model heavily or barely touched it.
Apple's team used influence functions, a technique that measures how much any single training example actually shifted the model's final behaviour. Think of it like checking receipts after a party: some purchases moved the needle on the total bill, others were noise. A meaningful share of training data, the researchers found, falls into that noise category.
If a data point had negligible impact on the model, skipping its formal removal produces almost identical results to doing the full deletion, at a fraction of the computing cost. GPUs, the specialised chips that handle AI's heavy number-crunching, aren't cheap to run. Fewer GPU hours means real money saved.
The finding held across language and image-recognition tasks.
Does this mean companies can ignore deletion requests?
No. The research doesn't give companies a legal pass to ignore privacy requests. What it does suggest is that engineers building unlearning tools could prioritise high-influence data points and safely fast-track low-influence ones, cutting costs without harming the outcome.
For ordinary people, the honest implication is uncomfortable: the data you contribute to an AI system doesn't affect that system equally. Your interaction might be formative, or it might be background static.
Survivorship bias is worth flagging here. The researchers identified the low-impact cases, but the costly work remains for high-impact data, precisely the sensitive records most likely to matter in a real privacy dispute.
The part being oversold, in my reading, is the ease of the triage itself. Influence functions are not a cheap calculation either, and at scale the cost of sorting data into "matters" and "doesn't" may eat into the savings this method promises. Watch whether independent teams can replicate the cost reduction on production-sized models.
What to do now: If your business handles AI models and user data, ask your technical team which data in your training set actually moves the needle. You may be paying to forget things the model never really learned.
