MIT says a high-profile AI productivity study used data that cannot be trusted
A recent AI research paper touting dramatic productivity boosts has come under fire for questionable data and unverified claims—prompting MIT to publicly distance itself from the study.
Published in November 2024 on arXiv, the paper, titled "Artificial Intelligence, Scientific Discovery, and Product Innovation", was submitted to the Quarterly Journal of Economics. Its author, Aidan Toner-Rodgers, was a PhD student in MIT's Department of Economics at the time.
The study claimed that an AI tool used at a U.S. company employing over 1,000 researchers led to measurable gains. According to the paper, AI-assisted teams produced 44% more new materials, filed 39% more patents, and generated 17% more product innovations compared to teams working without AI. The findings attracted broad media coverage, including writeups in Nature and The Decoder.
But now, MIT is calling those results into question.
MIT says data can't be trusted
On May 16, 2025, MIT released a public statement after a confidential internal review triggered by complaints about the study. The Institute's Committee on Discipline said it has "no confidence in the provenance, reliability or validity of the data" and "no confidence in the veracity of the research contained in the paper." Toner-Rodgers is no longer affiliated with MIT.
MIT said it intervened because the paper was already shaping public debates about AI's role in science, even though it had never been peer-reviewed. "Research integrity at MIT is paramount – it lies at the heart of what we do and is central to MIT’s mission," the Institute wrote.
MIT formally asked Toner-Rodgers to withdraw the paper from arXiv. "We have directed the author to submit such a request, but to date, the author has not done so," the statement reads. Because arXiv only allows authors to retract their own work, MIT has now sent its own request to arXiv, asking that the paper be marked as withdrawn "as soon as possible." The editors of the Quarterly Journal of Economics have also been notified.
The controversy points to larger challenges in scientific research, where commercial interests, sprawling corporate research teams, and mounting pressure to publish quickly—and attract attention on social media and in the press to secure better jobs—can undermine established standards for review and verification.
AI News Without the Hype – Curated by Humans
As a THE DECODER subscriber, you get ad-free reading, our weekly AI newsletter, the exclusive "AI Radar" Frontier Report 6× per year, access to comments, and our complete archive.
Subscribe nowAI news without the hype
Curated by humans.
- Over 20 percent launch discount.
- Read without distractions – no Google ads.
- Access to comments and community discussions.
- Weekly AI newsletter.
- 6 times a year: “AI Radar” – deep dives on key AI topics.
- Up to 25 % off on KI Pro online events.
- Access to our full ten-year archive.
- Get the latest AI news from The Decoder.