Content
summary Summary

OpenAI's new DALL-E 3 text-to-image system offers significant improvements over its predecessor and is natively integrated with ChatGPT. According to OpenAI, this integration should eliminate the need for "prompt engineering".

Ad

According to OpenAI, DALL-E 3 shows significant advances in its ability to generate images that accurately follow complicated prompts. This includes accurately representing a scene with specific objects, their relationships to each other, and details such as human hands or text in the image, typical weaknesses of previous text-to-image systems.

"An illustration of a human heart made of translucent glass, standing on a pedestal amidst a stormy sea. Rays of sunlight pierce the clouds, illuminating the heart, revealing a tiny universe within. The quote 'Find the universe within you' is etched in bold letters across the horizon." | Image: DALL-E 3 prompted by OpenAI

DALL-E 3, like DALL-E 2 and other text-to-picture systems, supports a variety of styles, from artistic abstracts to comic book drawings to photorealism. The prompt for the featured image of this article was: "An illustration of an avocado sitting in a therapist's chair, saying, 'I just feel so empty inside,' with a pit-sized hole in the center. The therapist, a spoon, scribbles notes."

"Close-up photograph of a hermit crab nestled in wet sand, with sea foam nearby and the details of its shell and texture of the sand accentuated." | Image: DALL-E 3 prompted by OpenAI

DALL-E 3 will be a ChatGPT app

Another new feature is that DALL-E 3 will become a ChatGPT app for paying ChatGPT Plus users. According to OpenAI, ChatGPT can be used as a prompt partner for DALL-E 3 to help brainstorm and optimize prompts.

Ad
Ad

If the resulting image is close but not exactly as intended, the user can correct it by telling ChatGPT what to change. This conversational image generation should reduce the need for complicated prompts, according to OpenAI. The company claims that complicated "prompt engineering" is no longer necessary.

Video: OpenAI

OpenAI says that all images generated by DALL-E 3 are fully owned by their creators. The images can be reused, marketed and sold.

The launch of DALL-E 3 in ChatGPT-Plus, ChatGPT Enterprise and via API is planned for October. A standalone version of DALL-E 3 via OpenAI Labs, similar to DALL-E 2, will follow later this fall.

OpenAI offers artists an opt-out of their work - for a future model

A major change and restriction compared to existing image systems is that DALL-E 3 rejects prompts that include the names of living artists. This is a popular method in other image systems to give stylistic direction to the system.

Recommendation

Furthermore, artists have the option to exclude their images from the training of future (!) OpenAI image models. This in turn could mean that DALL-E 3 is still being trained with these artists' work, which could lead to further copyright discussions. OpenAI may be able to provide updated versions of DALL-E 3 after an opt-out period.

Prompts with the names of famous people should also be rejected for safety reasons, to prevent, for example, AI-generated images of politicians from going viral on social media, as happened with Midjourney.

OpenAI plans to address these and other risks, such as visual over- or underrepresentation, or misuse for propaganda, with red-teaming techniques. OpenAI is also internally testing a tool that can detect whether an image has been generated with DALL-E 3.

Ad
Ad
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.
Support our independent, free-access reporting. Any contribution helps and secures our future. Support now:
Bank transfer
Summary
  • DALL-E 3, OpenAI's latest text-to-image system, offers significant improvements over its predecessor DALL-E 2 in generating images from text.
  • DALL-E 3 is newly integrated with ChatGPT, which acts as a brainstorming partner, refining prompts and enabling precise adjustments in the generated image without requiring extensive prompt knowledge.
  • In addition to the visual enhancements, OpenAI emphasizes security features such as prompt blocking for known people's names and better considers artists' rights by blocking the generation of living artist-style images and allowing opt-out from training data of future models.
Sources
Online journalist Matthias is the co-founder and publisher of THE DECODER. He believes that artificial intelligence will fundamentally change the relationship between humans and computers.
Join our community
Join the DECODER community on Discord, Reddit or Twitter - we can't wait to meet you.