Generative Fill is the most powerful AI tool in Photoshop. But how and what is it best used for? Here is an overview of Generative Fill and how to enable it.
Adobe recently integrated its generative image AI, Firefly, directly into its popular Photoshop software. Called "Generative Fill," it goes beyond text-to-image like Stable Diffusion, Midjourney, or DALL-E 2.
Instead, Adobe is combining it with existing AI features such as automatic object selection. This allows Generative Fill to adapt directly to the object. Firefly also monitors its environment: For example, if it detects a reflective surface such as water, it will insert the appropriate reflections on the water based on the object it has placed in the image.
With Generative Fill, Adobe is making a powerful tool available to professional graphic designers. It also lowers the barrier to entry for newcomers to complex image editing. In this article, I've selected a few use cases for this new Photoshop feature.
Activate Photoshop beta channel to use Generative Fill
Generative Fill is currently in beta testing. It will be available to all users in the second half of the year. You can activate the beta channel from the Creative Cloud application on Windows or MacOS. To do so, launch Creative Cloud and go to the Applications tab. In the left sidebar, the last entry under "Categories" is "Beta Apps."
Locate "Photoshop (Beta)" and click Install. Note that this will install a standalone version of Photoshop that requires additional disk space. Once installed, you can launch the application as usual, directly from Creative Cloud or from the system tray.
Once you have an image open in Photoshop, the Generative Fill interface will launch as soon as you select an area with the lasso or other selection tool. You can leave the box blank if you want to delete an object, or just for a fun surprise object.
Generative Fill is a versatile tool in Photoshop. To generate or replace objects, simply select an area in your image and enter a text description of what you want. The software automatically creates a new layer with its edit, so it's easy to get rid of it if you don't like it.
Similarly, you can create a new background by selecting the background and entering a text prompt. The tool also allows you to extend an image. Simply select the empty area of your expanded canvas and either generate without a prompt for a seamless scene continuation or use a prompt to add unique content. If you want to remove an object, the AI can do it without prompting.
Generative Fill encourages experimentation and the rapid generation of creative variations. It looks like a game-changer in image editing.
Removing elements with Generative Fill
Content-Aware Fill offered a glimpse of what AI could do in image editing. It allows you to select areas and fill them with whatever Photoshop deems appropriate. In addition to the Eraser tool, this is another way to remove unwanted elements from an image.
Generative Fill takes this feature to the next level. It works by leaving the input field blank before clicking Generate, or by typing in a prompt such as "remove. Compared to the older Content-Aware feature, Adobe's AI image generator performs much better, as the following comparison shows.
Content-Aware Fill vs Generative Fill. 🤯
Content-Aware certainly has its place, but when you need to remove objects against complex backgrounds, try out Generative Fill! pic.twitter.com/qakZ2utEoM
— Howard Pinsky (@Pinsky) May 25, 2023
Enhancing the Sky with Text Prompts
Manipulating the sky has a long tradition in landscape photography. Some programs even specialize in it. These programs usually detect the sky reliably, but only replace it with a fixed number of alternatives.
Generative Fill, on the other hand, offers countless ways to change the sky in a scene. But you have to mark the sky carefully, otherwise, objects above the horizon may be painted over.
Change the aspect ratio of an image and fill the image frame with Generative Fill
Content-Aware Fill can already convert a portrait image to landscape. However, Generative Fill takes this capability to the next level by inserting new, thematically appropriate content into the enlarged frame, rather than simply enlarging existing content.
The following video shows how a portrait image becomes a landscape image with additional content that wasn't in the original image, all with the click of a button. In case you don't speak German, the guy in the video is pretty excited. "Oh my God, this is going to change everything, it looks perfect. I only needed one click to go from portrait to landscape."
Hab mal ein Bild durch die neue KI von Photoshop gejagt. Es ist übel. In zwei Klicks aus einem Hochformat ein Querformat gemacht, ohne auch nur ein wenig editieren zu müssen. pic.twitter.com/Jdq5svFz8l
— Martin Neuhof (@martinneuhof) May 25, 2023
But especially for non-artistic images and photos, and perhaps even for these, the feature raises ethical questions. The AI adds imaginary elements to a real scene, and the transition is fluid. What is real and what is generated remains unclear at first glance. With Content Credentials, Adobe offers a way to make AI's manipulation of an image more transparent.
In addition to the features shown here, there are many more, and of course, Photoshop's Generative Fill can also be used to create entirely new images from scratch. You can start with an empty canvas and fill it with the Generative Fill function via a text prompt, similar to what you know from Midjourney, DALL-E, or Bing Image Creator.
Although Firefly does not yet reach the quality level of Midjourney v5, Adobe should have the copyright-compliant (!) data and the computing power to catch up in the foreseeable future. And its editing capabilities are already far more advanced than those of other AI image generators.
