An OpenAI analysis shows the efforts of some state-backed actors to conduct online disinformation campaigns using AI systems. So far, these campaigns have gained little traction.
OpenAI has released a report detailing how partially state-backed groups from Russia, China, Iran, and Israel tried to use AI models to spread false information online. According to OpenAI, the campaigns did not reach many people or drive engagement.
The study found five operations: two from Russia, one from China, one from Iran, and a commercial company in Israel called STOIC.
All groups used AI to make content or work faster, but not only AI. They mixed AI-made material with regular things like text they wrote or memes they copied.
For instance, a previously unknown Russian network called "Bad Grammar" used OpenAI models to create and post political comments on Telegram focused on Russia, Ukraine, the US, Moldova, and the Baltic States.
The Chinese operation "Spamouflage" generated content praising China and attacking its critics. The Iranian "International Union of Virtual Media" (IUVM) created pro-Iran, anti-Israel, and anti-US articles.
AI-spoofed social media engagement
The campaigns also used AI to generate fake engagement for posts, such as by generating replies to their own posts. According to the study, none of the networks achieved real interactions to any large degree. OpenAI says the operations did not go above category 2 on a six-level "breakout scale" for judging the impact of influencer operations.
Amid the technical change, human error remained: operators accidentally posted OpenAI system messages that showed their generated content as AI products. Wrong captions or mixing of topics also happened.
The report stresses the importance of "defensive design" when building AI systems, which means building in safeguards. OpenAI also uses AI models themselves to more efficiently detect suspicious activity. Because the campaigns were designed to spread content across many platforms, sharing information with industry peers is key to stopping the actors, the company says.
In mid-February, Microsoft and OpenAI reported that state-affiliated actors from China, Iran, North Korea and Russia were using the company's AI services to create content, code, or conduct cybersecurity research. The companies said all actors have been successfully shut down.
