Content
summary Summary

A new study shows that accounts on X (formerly Twitter) that use AI-generated profile pictures make up about 0.052 percent of profiles and often share content about politics, COVID-19, and the war in Ukraine.

Ad

Researchers from Ruhr University Bochum, GESIS Leibniz Institute, and CISPA Helmholtz Center analyzed nearly 15 million X accounts and identified 7,723 profiles using AI-generated portraits.

These accounts showed distinct patterns: they had fewer followers, followed fewer accounts themselves, and more than half were created in 2023—sometimes hundreds within hours.

Lead author Jonas Ricker notes that this is "clear indication that they weren’t real users." After nine months, X had suspended more than half of these accounts.

Ad
Ad

Trump, Corona and Ukraine as main themes

The study's methodology focused on recognizing images generated with the StyleGAN 2 AI model, so images generated by newer and potentially more widespread AI systems were not detected. This means that the study does not provide a comprehensive picture of the proportion of fake AI profile pictures, let alone fake accounts on X.

Example of an AI image generated with StyleGan. | Image: via thispersondoesnotexist.com

What is interesting, however, is the content of the detected accounts with fake AI images on X. A manual content analysis of 1,000 accounts with fake AI portrait images revealed a clear thematic focus on politics, often related to Trump, COVID-19 and vaccinations, and the war in Ukraine. Lotteries and financial topics such as cryptocurrencies were also common.

Bar chart: Distribution of 1000 Twitter accounts across 5 topic categories, with politics leading the way with almost 400 accounts.
The analysis of 1000 manually checked active X accounts shows a clear dominance of political and financial topics, while business, sex and various other topics take up less space. | Image: Ricker et al.

The researchers speculate that some of these accounts were created specifically to spread disinformation and political propaganda, with the mass creation and identical metadata suggesting they are part of an organized network.

UMAP diagram: colored clusters of English tweets grouped by topic, plus bar chart of topic distribution of active accounts.
The visualization shows a distinct clustering of topics in English-language tweets, with politics and finance dominating. | Image: Ricker et al.

The researchers plan to expand the automation of AI fake profile image detection and include newer models in future analyses to better understand the scope and impact of the technology on social media platforms like X.

Although the study did not examine the reach of the fake AI image accounts, their lower average follower count and fewer followed accounts suggest they have limited reach. This aligns with an OpenAI study that found social media accounts spreading AI-generated messages for political propaganda received little response, rendering them largely ineffective.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Recommendation
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • A study by German researchers found that 0.052 percent of profiles on X use AI-generated profile pictures created with StyleGAN 2, with a likely much higher number of unreported cases.
  • An analysis of 1,000 accounts with AI-generated profile pictures revealed thematic focuses on politics (often related to Trump), Covid-19 and vaccinations, the war in Ukraine, lotteries, and financial topics such as cryptocurrencies.
  • The researchers say the mass creation of these accounts and their identical metadata point to organized networks aimed at spreading disinformation and political propaganda.
Sources
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.