AI trained on photos creates photorealistic images that are almost indistinguishable from the real thing. One example is the imaginary arrest of Donald Trump. This again raises the question of how to regulate AI images.
The release of Midjourney v5 in particular highlights the risks of AI images in social and editorial media. The results of Midjourney's latest AI image model are even more photorealistic than in previous versions. The often inaccurate rendering of hands and fingers, previously used to quickly expose AI generations, has also been significantly improved since Midjourney v5.
On Monday, March 20, one topic was particularly present on social media: the possible arrest of former U.S. President Donald Trump. Eliot Higgins, founder of the Dutch investigative organization Bellingcat, spent his time visualizing these scenes with Midjourney v5.
Making pictures of Trump getting arrested while waiting for Trump's arrest. pic.twitter.com/4D2QQfUpLZ
— Eliot Higgins (@EliotHiggins) March 20, 2023
Fake Trump pictures reach more than five million people
"Making pictures of Trump getting arrested while waiting for Trump's arrest," he commented on a tweet with two of a total of 50 images generated, which at that point had been viewed well over five million times. Media outlets picked up the images, giving them an even wider reach.
His comments, along with the images' obvious flaws, such as the odd depiction of legs, should be enough to show that the images are not factual, Higgins said. "I didn't intend to do any clever criticism or anything like that. But then it kind of took on a life of its own."
The images are still available on Twitter - but Higgins suspects that he no longer has access to Midjourney as a result of this incident. While Buzzfeed News has not heard back from Midjourney, one indication is that the word "arrested" has since been added to its list of banned words. Someone at Midjourney seems to have noticed and reacted to what happened.
Midjourney does not explicitly ban political AI images
Midjourney's rather vague community guidelines only broadly prohibit images that are "inherently disrespectful, aggressive, or otherwise abusive" as well as "adult content or gore". Explicitly political content is not mentioned.
So far, Midjourney has not commented on the incident, nor on whether they might consider tightening the rules to prevent political characters from being generated. In our own tests, we were able to create images with Putin, Trump, Merkel or Biden without any problems, while "Xi Jinping" is blocked.
However, banning individual prompt words is an afterthought, because with the fake Trump images already achieving enormous reach, the potential damage would have already been done. Moreover, alternative phrases such as "Trump in handcuffs" were still possible after the "arrested" ban, and the general ban has been criticized in the Midjourney community because the term can be used in other, legitimate contexts for image generation.
"Not that this will show up anywhere on anyone's metrics, but I just unsubscribed from my $50/mo plan because I think it's stupid that the word "arrested" is now banned as the result of the totally inevitable images of a fake arrest of Donald Trump being circulated on Twitter," wrote one user on Midjourney's Discord.
As AI photorealism continues to advance, criticism of Midjourney and similar image generators is likely to increase, although the dam for fake images may have broken anyway with open source solutions like Stable Diffusion.
It seems that the 2017 prediction by Ian Goodfellow, the inventor of the initial deepfake technology, Generative Adversarial Networks (GANs), is coming true: that people can no longer trust images and videos on the web. Now all that's left is for someone to tell people.
