Fun with Depth Map Photos

If you’re on Facebook, you’ve probably run across photos lately that move in an odd way. They’re not videos, but still photos that appear to have three-dimensional depth. Facebook calls them 3D Videos, and they’re built on technology offered in a few recent smartphones senses the depth of objects in the scene.

Phone manufacturers generate this effect in a few ways. Some models sport dual cameras that create a stereo image to determine depth, while others rely solely on software to identify faces and objects and make a depth map based on those areas, just as you might create a mask in Photoshop to edit a specific area.

With those areas identified, they can be manipulated. The most common effect is to simulate shallow depth-of-field to create a blurry background that looks like the image was shot with a wide-aperture lens (which is nearly impossible to do using a smartphone camera’s tiny hardware). In this article, I want to point to some ways to take advantage of depth maps, and wrap up with how to use Facebook’s 3D Photos feature. Since I don’t have a recent Android device that can record depth map information, I’m focusing on the iPhone.

Editing in the Apple Photos App

Starting with iOS 12, the Photos app on the iPhone includes a control for adjusting the faux background blur. It works only with Portrait Mode photos captured using the iPhone XS and iPhone XS Max, but some other iPhone models can use this control to edit those photos.

Open a Portrait Mode photo in the Photos app and tap Edit. Below the image is an aperture control: slide it to the right to simulate a wider aperture (up to f/1.4) or left to reduce the blurry effect (up to f/16).

Editing in Other Apps

Surprisingly, Apple wasn’t the first to enable editing of its own depth map images. Other developers jumped on the chance to play with the tools Apple provided. Focos, for example, includes controls for adjusting the blur, of course, but also lets you change the shape of the simulated aperture blades, giving blurry points of light in the background circular, oval, or multiple other shapes—even objects like hearts or butterflies in the paid version of the app.

You have control over highlights and color, and there’s a mode where you can define which areas are the foreground and which are the background. If that’s not enough, it has a Patch tool for filling in or erasing areas that the camera incorrectly defined as foreground or background elements.

Other apps worth checking out that edit Portrait Mode attributes include Camera+, Darkroom, and Slør.

Lightroom Classic range masking

One of the new features in Lightroom Classic CC 8.0 is the ability to create a selection mask, such as a linear or radial gradient, and then refine the mask based on the photo’s depth map. (See “What’s New with Lightroom Classic CC 8.0.”)

This feature isn’t focused on changing background blur, but instead using the depth map to define objects within the scene. Then, you can selectively adjust those areas using Lightroom’s tone and color controls. Here’s how to do it:

Open an image with depth information.
In the Develop module, select the Linear Gradient, Radial Gradient, or Adjustment Brush tool, and draw a gradient that covers most of the image. (I’m using Linear Gradient for this example.)
At the bottom of the tool’s options, click the Range Mask pop-up menu and choose Depth. It helps to also enable the Show Depth Mask option.
Use the Range slider to define the masked areas; increasing the left slider, for instance, exposes more foreground images. The Smoothness slider affects the falloff at the mask’s edges.
Use the other controls in the panel to adjust the tone, color, and effects such as Clarity and Dehaze for the selected area.

3D Photos in Facebook

Facebook’s 3D Photos feature is currently rolling out to members’ accounts. It reads the depth information in Portrait Mode images captured by supported iPhone models (iPhone 7 Plus, iPhone 8 Plus, iPhone X, iPhone XS, and iPhone XS Max), and creates its own animated 3D effect. Most noticeably, the image’s perspective shifts as you scroll through a Facebook timeline, but you can also move the phone or, on a desktop, click within the image to change the appearance.

It’s a cool effect, but it also needs refinement, in my opinion. If the depth map is fairly shallow, such as just the outline of a person in the foreground, the effect appears more pronounced than it should, with obvious blurred areas being filled in for the space behind the subject. Photos that contain objects on several layers of depth work much better. Also, I hope Facebook offers an option for viewers to turn off the 3D effect; I know several people for whom these kinds of motion effects make them queasy.

That said, it’s a fun and easy way to impress Facebook friends. When you create a new post on a mobile device, you’ll see a new 3D Photo media option. If you don’t see that yet, here’s how to make it appear without waiting for the rollout to include your account on an iPhone (credit goes to the Mac Observer for sleuthing this sequence):

In the Facebook app, “Like” the Facebook 360 Page.
Force-quit the Facebook app: on an iPhone X, XS, or XS Max, swipe partway up from the bottom until you see the app switcher, and then swipe the Facebook screen up until it disappears at the top of the screen; on an iPhone with a Home button, double-press the button and then swipe the Facebook app up.
Relaunch the Facebook app. The 3D Photo option appears when you create a new post.

Totally Deep

By far the most interesting thing about photos with depth information is that what we’re seeing today is really pretty crude compared to what’s coming in the next few years. (I’m reminding myself to revisit this article in 2022.) As processing power and machine learning improves, perhaps depth maps will simply become the norm.