OpenAI's new DALL-E 3 text-to-image system offers significant improvements over its predecessor and is natively integrated with ChatGPT. According to OpenAI, this integration should eliminate the need for "prompt engineering".
According to OpenAI, DALL-E 3 shows significant advances in its ability to generate images that accurately follow complicated prompts. This includes accurately representing a scene with specific objects, their relationships to each other, and details such as human hands or text in the image, typical weaknesses of previous text-to-image systems.
DALL-E 3, like DALL-E 2 and other text-to-picture systems, supports a variety of styles, from artistic abstracts to comic book drawings to photorealism. The prompt for the featured image of this article was: "An illustration of an avocado sitting in a therapist's chair, saying, 'I just feel so empty inside,' with a pit-sized hole in the center. The therapist, a spoon, scribbles notes."
DALL-E 3 will be a ChatGPT app
Another new feature is that DALL-E 3 will become a ChatGPT app for paying ChatGPT Plus users. According to OpenAI, ChatGPT can be used as a prompt partner for DALL-E 3 to help brainstorm and optimize prompts.
If the resulting image is close but not exactly as intended, the user can correct it by telling ChatGPT what to change. This conversational image generation should reduce the need for complicated prompts, according to OpenAI. The company claims that complicated "prompt engineering" is no longer necessary.
Video: OpenAI
OpenAI says that all images generated by DALL-E 3 are fully owned by their creators. The images can be reused, marketed and sold.
The launch of DALL-E 3 in ChatGPT-Plus, ChatGPT Enterprise and via API is planned for October. A standalone version of DALL-E 3 via OpenAI Labs, similar to DALL-E 2, will follow later this fall.
OpenAI offers artists an opt-out of their work - for a future model
A major change and restriction compared to existing image systems is that DALL-E 3 rejects prompts that include the names of living artists. This is a popular method in other image systems to give stylistic direction to the system.
Furthermore, artists have the option to exclude their images from the training of future (!) OpenAI image models. This in turn could mean that DALL-E 3 is still being trained with these artists' work, which could lead to further copyright discussions. OpenAI may be able to provide updated versions of DALL-E 3 after an opt-out period.
Prompts with the names of famous people should also be rejected for safety reasons, to prevent, for example, AI-generated images of politicians from going viral on social media, as happened with Midjourney.
OpenAI plans to address these and other risks, such as visual over- or underrepresentation, or misuse for propaganda, with red-teaming techniques. OpenAI is also internally testing a tool that can detect whether an image has been generated with DALL-E 3.
