Adobe adds AI-powered masking tools to Lightroom
Adobe Camera Raw will also get tools that let you select with a single click.
Today, Adobe announced a completely redesigned and reimagined way to make selective adjustments in Adobe Camera Raw (also known as ACR, the raw photo processing tool found within Photoshop), Lightroom, and Lightroom Classic that we now collectively refer to as masking. These new masking tools will become available on October 26th, and we wanted to share a bit of the process that went into creating these powerful new features ahead of the release.
Adobe has revealed some new masking upgrades that are coming to Lightroom, Lightroom Classic and Adobe Camera Raw (or ACR, Photoshop’s raw photo processing tool). The company calls it the “biggest change to providing control over selectively enhancing photos” since it released Lightroom 2 in 2008.
The Adobe Research team wanted to bring AI-powered selection tools such as Select Subject and Sky Replacement from Photoshop into Lightroom and ACR, but the image processing engine used in the latter two was incompatible. The team had to make some big changes under the hood, which gave it a chance to change how selections are handled in Lightroom.
Until now, ACR, Lightroom and Lightroom Classic have only supported vector-based selections (which are recorded as mathematical expressions), but the AI-powered masks need bitmap (or image-based) support. So, to bring the AI-based tools to those apps, Adobe had to make both approaches work together. It’s still able to use vector-based selections for brush, gradients, and range masks to minimize the storage space needed, while the select subject and select sky tools (which can create a mask for a subject or sky with a single click) use bitmaps.
As it figured out how to make those two kinds of selections work together, Adobe developed new features for ACR, Lightroom and Lightroom Classic across desktops, mobile devices, tablets and the web. One such upgrade is mask groups, which will let you combine any mask tools. For instance, you’ll be able to use a gradient vector-based tool in concert with an AI-powered feature such as select sky. It’ll be possible to separate a mask from another masking tool as well. You’ll be able to invert selections and there’ll be more options for range masks, such as targeting the entire image.
Selectively adjusting certain areas of a photo differently than other areas is a technique that’s nearly as old as photography itself. With black and white negative-based photography, the most common techniques of dodging and burning enable photographers to lighten (dodge) or darken (burn) specific areas of a photo, with the goal of adjusting the balance of tones to support the intent of the image maker. Ansel Adams may have summarized it best in his quote, “Dodging and burning are steps to take care of mistakes God made in establishing tonal relationships”, and his finished prints were realized only through extensive dodging and burning.
Early versions of Photoshop introduced tools like the dodge and burn tools, as well as selections, masking, layers, and layer masks which enabled photographers to make selective adjustments in the digital darkroom. Lightroom 2, released in 2008, introduced the brush, linear gradient, and radial gradient tools enabling direct selective adjustments in a non-destructive, photography-specific environment. The new masking functionality represents the biggest change to providing control over selectively enhancing photos since the release of Lightroom 2.
The Adobe Research team (our internal skunkworks group that is tasked with dreaming up the impossible and then making it a reality) has been working on creating new approaches to making selections for some time and had recently partnered with the Photoshop team to release some of their AI-powered selection tools, including Select Subject and the Sky Replacement tool. These tools were incredibly well-received, and made us think — how could we bring them into ACR and Lightroom? The existing image processing engine used by ACR and Lightroom wasn’t compatible with the new AI-powered tools — we were going to need to make some really big changes to that image processing engine. With this big foundational work came the opportunity to reassess the fundamentals of how selections are made within Lightroom, which meant we were going to want to talk to our customers.
One of the things I love most about working on the ACR and Lightroom team is that we build our products with our customers, not for our customers. We’ve always prided ourselves on our connection to the photography community, and to build the new masking tools, we pulled out all of the stops. In fact, we started working on these features over a year ago, partnering with our amazing user research, design, and marketing teams to figure out exactly what was missing and how we could solve for those needs. We ended up surveying tens of thousands of customers, across ACR, Lightroom, and Lightroom Classic, ranging from hobbyists to professionals to get an understanding of the biggest pain points with making selections, as well as what they loved about the existing tools, and what they wish they could achieve with their photos. Our Design Research team then talked directly to over a hundred customers over the course of a year and a half to watch how selective adjustments were being done in the current tools and eventually to start testing prototypes and refine the experience until we had a balance of power, precision, and familiarity, something that new customers could become successful with quickly, but that didn’t hamper creativity and control, all while ensuring that existing customers would feel at home with the tools.
New improved Masking Engine
The first opportunity we identified was how we could take advantage of all the new and powerful AI-based, machine learning-powered masking tools coming out of Adobe Research. These tools first made an appearance in Photoshop in the Select Subject as well as Sky Replacement tools, making it possible to identify the subject in a photo (regardless if it’s a person, animal, or inanimate object) or the sky, resulting in a precise selection. The trouble was, there was no way to integrate these new models into the existing selection tools in ACR, Lightroom, and Lightroom Classic.
This required the team to go back to the drawing board and re-imagine how the new masking engine could incorporate these AI-based tools across both mobile and desktop devices. We knew it would be essential that photographers have access to our best new tools anywhere, and on any device.
Previously, the only types of selections that were supported were vector based, meaning that any adjustment made via brush or gradient were recorded as mathematical expressions. This meant that small amounts of data could be used to represent even complex masks on very high-resolution images. Vector-based masks are useful for avoiding creating large files and catalogs, which would eat up valuable hard disk space and make synchronization slower. The new AI-based masks, however, require bitmap, or image-based, support. In essence, the new AI-based masks create a grayscale image, where lighter values represent varying levels of selection and pure black represents no selection being made to that area. A rather common example of these grayscale bitmap-based masks can be found in Photoshop’s layer masks. With the new masking functionality, we needed to make it possible for both vector-based as well as bitmap-based masks to live together in harmony. The brush, gradients, and range masks continue to be vector-based (to limit space used when making masks) while the AI-powered tools such as select subject and select sky use these bitmap-based masks. We’ve also got some fun new tech brewing that we’re really excited about that will take further advantage of bitmap-based masks which we’ll be releasing sometime next year.
A reimagined Masking experience
The second major opportunity was to improve the overall interactive experience. With the major architectural work being done to the masking engine, we decided that there was an excellent opportunity to rethink and greatly improve the experience and capabilities in every flavor of ACR, Lightroom, and Lightroom Classic, across desktops, mobile devices, tablets, and the web.
Over the course of a year and a half, the Design Research team brought together user experience designers, engineers, and a wide range of customers to build and test prototypes that drove the process of creating the new masking experience Through working with our customers, we identified four major areas of opportunity for the new masking features:
- More control & flexibility
- Improved workflow & organization of selections
- Consistency across all devices
- Better in-app support to maximize the potential of the tools
More control & flexibility
The most important job of any selective tool, obviously, is making it possible to select specific objects or areas, and only those objects or areas. We needed to find a new paradigm that would not only enable these fancy new AI-powered tools (as well as future tools that are currently in the works) but also make it possible to refine the selections, and make selections that current (and possibly even future) AI-powered tools could not.
To do this, we created mask groups, which make it possible to combine any mask tool with any other mask tool. This means any selection tool can be mixed and matched with any other tool, including the brush, gradient, and luminance and color range tools and AI-powered tools like select subject and select sky.
We also realized that just adding masking elements together wouldn’t be enough, so we also created the ability to subtract any mask from any other. The ability to subtract an AI-powered mask from another masking tool results in some amazingly powerful selections, and is one of my favorite ways of using these tools.
Next, we made sure any selection could be inverted. For example, everything but the subject can be selected by inverting select subject. Or, select just the ground by inverting select sky. And for an even more fun technique, if there’s a subject in the scene, invert select sky and then subtract select subject from the group to result in the ground being selected without the subject or the sky.
Finally, we heard from a number of our customers that they wanted more powerful range masks, including the ability to target the entire photograph. So we made the range masks work globally (they can still be applied within a gradient, just as before, using the add and subtract controls) and brought them to Lightroom desktop and mobile. Additionally, we added more control over the luminance range’s falloff.
Improved workflow & organization of selections
With all this new power, we realized it could pretty quickly get overwhelming and hard to keep track of all of the masks. We created a new masking panel to help organize everything. On desktop, the masking panel can be placed anywhere, docked into the edit panel, or minimized showing just a preview of the mask.
Another major request was the ability to name each mask, making it easier to keep track of each mask group and what it’s doing.
Lastly, we brought in a number of the overlay visualizations from Photoshop, so that the mask could be previewed a number of different ways, including the default color overlay, color overlay on black and white, image on black, image on white, and more.
Consistency across all devices
A major topic we heard from our customers was that they wanted a consistent experience no matter where they used the tools. Many customers use a combination of devices and applications and want to make sure that they can start and refine edits with the same tools and capabilities, regardless of where they are. To support this goal, we made sure that every single tool and capability works on any device and in all apps. The AI-powered tools work on mobile devices equally as well as on desktop. The powerful range masks, previously only found in ACR and Lightroom Classic, are also now available in Lightroom as well as in Lightroom mobile. And all of the powerful capabilities to modify and refine masks are equally available throughout the ACR, Lightroom, and Lightroom Classic ecosystem.
Better in-app support to maximize the potential of the tools
Something that came up time and time again during our research sessions was the need for a different type of help. Something that wasn’t constantly in your face but also wasn’t full of arcane symbols and shortcuts, expecting you to ‘just get it.’ Found in every client is help built directly into the masking experience. In Lightroom, there’s also a new interactive tutorial that walks you through the various tools, step-by-step.
What’s next?
Both the Adobe Research and Design Research teams have a bunch of more tricks up their sleeves. From new and more powerful AI-powered tools to further refinements and improvements to the interactive experience, keep your eyes open for even more announcements in the near future — we’re just getting warmed up.