Inside Meta's secret experiments that improve its AI models
Gilbert Flores/Variety via Getty Images
- A legal case involving Meta revealed the company's secret experiments with training data.
- Meta used "ablation" to identify how specific data improved its Llama AI models.
- Some researchers say this could support a system to assign value to AI data and pay compensation.
A high-profile legal case has unearthed a trove of internal Meta communications, and one particular document has caught the eye of some AI researchers.
This reveals new insights into how models are built and could influence who gets to share in the spoils of this new technology.
Buried in these court filings is a description of how Meta researchers used a process called ablation to identify which data helped improve the company's Llama AI models.
Ablation is a medical technique that purposely destroys tissue to improve things like brain function. In AI, it involves removing parts of a system to study how those components contribute to performance.
: BSIP/Universal Images Group via Getty Images
In Meta's ablation experiments, the company replaced a portion of its AI training data with pirated books from a giant database called LibGen. Then, the company re-trained its Llama model to see the impact.
In one experiment, Meta added books about science and technology, along with fiction books, to the training data. In a second experiment, Meta only added fiction books.
In both experiments, Llama performance improved notably in industry benchmark evaluations, according to the internal Meta document disclosed in court filings. (Check out pages 18 and 19 here.)
This shows that Meta has the ability to assign value to specific training data, said Nick Vincent, assistant professor in the School of Computing Science at Simon Fraser University.
Ablation is common, but also a secret
Sony Pictures
Ablation has become a common practice at the company and across the AI industry. For instance, one Meta engineer on LinkedIn mentions doing more than 100 ablations during the development of Llama 4 and previous iterations of the company's big AI models.
Meta doesn't publish the results of these experiments, and other AI companies keep this stuff private, too, Vincent said.
One potential reason: If tech giants tell the world which training data specifically helped their AI models, then the creators of this information would want to be paid β and they would have a handy estimate of how much money they're owed.
"Stating these numbers publicly would potentially give some content organizations firmer ground to stand on," Vincent said.
Making the results of ablation experiments public could also impact high-stakes copyright lawsuits that rage across the tech industry β with this specific Meta case (Kadrey v. Meta) being a good example.
In these cases, tech giants and AI startups argue that it's not copyright infringement for machines to "learn" from published material online.
Internal documents assigning value to specific content may not help with this.
"It's possible that publishing these value estimations would undermine the stances that Big Tech companies will take in these copyright lawsuits and court cases," Vincent said.
A Meta spokesperson said the company disagrees with the plaintiff's arguments in this legal case and added that its Llama models are helping individuals and companies be more innovative, productive, and creative.
"We will continue to vigorously defend ourselves and to protect the development of GenAI for the benefit of all," the spokesperson said.
Training data sources are now hidden
Matthias Balk/picture alliance via Getty Images
Keeping ablation experiments secret follows a broader trend away from sharing how data contributes to the creation and performance of AI models.Β
In 2017,Β the Google research paperΒ that kicked off the generative AI boom disclosed granular information on the training data used. It included about 40,000 sentences from The Wall Street Journal, for instance. Years ago, OpenAI, in its GPT-2 paper, described scraping web pages using millions of outbound links from Reddit.Β
Fast forward to today, and companies share very little. When Meta released Llama 4 in early April, the company published a model card describing how it built the product.Β It didn't mention ablation at all, and it only discussed the training data generically as "a mix of publicly available, licensed data and information from Meta's products and services."
Again, the likely reason for thisΒ is thatΒ telling everyone what data you used might mean having to pay the creators of this information.
"It's really disappointing that they're not being open about it, and they're not giving credit to the material," said Bill Gross, CEO of ProRata, a startup that's trying to compensate creators for their contributions to AI.
Gross said content creators should be paid twice: once for having their data used to train AI models and again when AI models rely on this content to answer user questions.
Meta's secret ablation results
Don Mason/Getty Images
Meta's ablation experiments focus on this first training step, which uses mountains of data to help models understand the world. For example: To teach a machine to recognize a llama, you must show it as many photos of llamas and alpacas as possible so it can distinguish between the two animals.
Meta's first ablation experiment found that adding science, technology, and fiction books to the training data improved Llama's performance by 4.5% on an industry benchmark called BooIQ. Just adding the fiction books resulted in a 6% improvement.
The performance gains from these ablation experiments were as high as 5.5% on another benchmark known as SIQA, the Meta internal document said.
Peter Henderson, an assistant professor of computer science at Princeton, tweeted out some Meta charts from the court document showing these gains.
Lots of internal Llama 2 data mix ablations revealed as part of discovery in the ongoing copyright litigation. Link below. pic.twitter.com/7YeRyYSEWV
β Peter Henderson (@PeterHndrsn) January 15, 2025
While performance gains of about 5% seem small, in the AI race, any advantage is important.
"That's actually a lot because it's so hard to get every extra point on AI benchmarks," Gross said.
Can elves mate with humans?
New Line Cinema
Llama's improvement on the BooIQ benchmark shows the power of specific training data and how much AI models and tech companies rely on this information, Vincent said.
BoolQ is a series of 15,942 yes/no questions that AI models must answer. The more questions they get right, the higher the performance. A 5% improvement is the equivalent of answering almost 800 extra questions correctly.
One question on the BooIQ test asked, "Can elves and humans mate in 'Lord of the Rings?'"
You can only really know the answer to this for sure if you've read J.R.R. Tolkien's books β or rather if these books are in the training data, Vincent said. (Elves and humans can have babies in the LOTR universe, by the way.)
Vincent hopes revelations like this about Meta's secret ablation experiments will help create a new system that assigns credit to sources of training data and provides appropriate compensation.Β
"AI chatbot products rely on the fact that some human somewhere did something useful, wrote it down, and published it," he said. "This technology repackages this stuff into something that is hopefully more useful."
"Ultimately, it's all humans at the top of this. Without this data, AI models will not be so good," he added. "Evidence of ablation like this could end up serving the mission of setting up a healthy data flow. It's important to sustain the institutions where people are incentivized to create content and knowledge and share it."
