Using Generative Fill in Photoshop

With the meteoric rise of AI-generated images, you might be concerned that traditional Photoshop artists are being left behind. But that’s all set to change with the release of the first public beta of Photoshop to include its Firefly engine, which generates imagery to order.

You don’t even need Photoshop to try it out: Adobe Firefly is publicly available at firefly.adobe.com, and you can try out its magic for yourself.

First, though, we’ll look the version of Generative Fill that has been built into Photoshop. As stated above this is a public beta, which means it’s not yet ready for prime time – indeed, the license specifically prohibits is use for commercial work. Here’s what it can do for you.

Outfilling

We’re used to the Content-Aware Fill, which does a reasonable job of extending or reshaping images. Generative Fill takes that to a whole new level. In this example, a portrait shot of an ornate room has been extended to the right by widening the canvas with the Crop tool.

Selecting any region in Photoshop – in this case, the white area – brings up the Generative Fill button in a new floating palette. Click the button and you can enter a text prompt, as we’ll see later. Leave it blank and hit the Enter key, and in a few seconds you’ll be presented with three alternative images. Here’s the best: note how the perspective of the radiator correctly follows the angle of view, and it even has a reflection on the polished floor. Note also the lighting on the curtain from the window – even though the curtain material doesn’t quite match the original and the tie back is missing.

Adding a text prompt

You can choose to fill a selected area with literally anything you can imagine. This view of a storm drain is fully grassed over.

Selecting it with the Lasso tool, and typing “water” in the Generative Fill prompt, produces this near-perfect result:

What’s of interest here is not just that water has been added, but that grass now grows in front of it, matching the grass at the sides. An image like this would take a very long time to put together using traditional Photoshop tools.

Never work with children or animals

Typing “child” into the Generative Fill dialog produces a content warning error: Adobe are so concerned with the tool being used for illicit purposes that their moral filter has gone into overdrive. You can ask it to produce images of animals, although the results are often questionable.

This image of an abandoned mine entrance makes the perfect backdrop. Start by making a rectangular selection where you want the animal to appear.

Adding the prompt “horse and cart” produces three results to choose from. This one is just about acceptable, once you ignore the odd cart construction. It’s worth noting here that the filter doesn’t just add the horse and cart, it also modifies the background around it: see how the building behind has been remodeled in the process. It seems this is, at present, unavoidable.

Sometimes, though, the Generative Fill engine gets confused, producing images that are downright surreal.

It’s not just nature

Although Generative Fill works best with natural surroundings, it makes a fair stab at manmade backgrounds. In this street scene, I’ve selected the left-hand end of the street and filled it with no text prompt. In these cases, the filter just tries to extend the background as best it can.

The results are variable, but curiously compelling. Granted, one of the windows doesn’t make a lot of sense, but the perspective is spot on and at first glance it’s a perfectly acceptable street.

Building on success

You can keep working on an image, adding and replacing elements as required. As each new iteration is created on its own layer, you always have the option to go back and modify the results.

Here’s a vintage car parked outside a church. Photoshop’s Select Subject feature did a remarkably good job of selecting just the car; by inverting the selection, we’re able to change its location.

In this case I asked for a winter scene. The result is a road that matches the perspective of the car perfectly.

But there’s more that can be done here. How about adding a puddle? It’s easy to draw the approximate shape with the Lasso tool, and then just type “puddle” in the Generative Fill dialog. The extra snow produced here – complete with reflections – creates a much more appealing scene.

How about adding another car behind? Simple. Just select the space behind the original car, and type “1930s car” in the Generative Fill dialog to be presented with three alternatives. Once again, the perspective of the new car is just right for the scene.

Let’s add a bear – again, by selecting the area where we want it to appear, and typing the prompt “bear”. But here’s something truly extraordinary: the bear’s front paws are raised, and resting on the fender of the original car. This shows an awareness of context that is truly mind-boggling.

In the process, though, the radiator of the car has been recreated along with the bear. But this isn’t a problem: as each Generative Fill process appears as a new layer, it’s easy to mask the layer to reveal the original radiator behind.

The web alternative

If you don’t have Photoshop, or don’t want to install a Beta app, you can try it for yourself using the web version of Firefly, found at firefly.adobe.com. This offers a range of AI-driven image creation tools, including Generative Fill. I started with this image of myself on a sunny day.

Use the built-in brushes to select the area you want to replace. I selected the shirt. Then, in the text prompt area, I typed “business suit” and hit the Generate button.

And here’s one of the four returned results: a perfect-fitting, rather sharp suit. But not just that: it’s at exactly the right angle for the pose, even replicating the lighting direction with the shadow of my head on the collar.

Change of background? Easy: just use the Select Background button, and type in the sort of background you want to replace it with.

Text to image

Among Firefly’s tools are, of course, an image generator that creates imaginative new images from nothing more than a text prompt. Here, I asked for “futuristic house with robot mowing the lawn”, and these are the four results it returned.

Firefly and Generative Fill are in the early stages of development. Here’s the result of a Firefly prompt “highly detailed woman wearing lots of jewelry in grand hotel setting.” The results are similar to where Midjourney was a year ago; the faces are far from realistic, and some of the jewelry is, frankly, bizarre.

The technology has some way to go to catch up with the AI leaders. But these are early days, and AI is making weekly rather than annual leaps forward. It won’t be long before Photoshop can produce anything we choose, with photographic realism.

Unlike the other AI image generation tools, working directly in Photoshop means we can integrate generated results with our own images, and choose exactly where the new elements appear – and then use standard Photoshop tools to modify them. It’s like having an infinite, free photo library at your fingertips.

Where this leaves traditional Photoshop artists, though, is another question. Are the skills we’ve painstakingly learned over our careers are now becoming obsolete? Or is this just another tool that we can learn to master, and turn to our advantage? One thing is certain: the gap between inspiration and execution is getting shorter and shorter.