Adobe adds "Super Resolution" Enlargement Tool to Photoshop
One question I get asked frequenlty by photographers of all varieties is:
"How can I increase image resolution without losing too much image quality?"
Enlargements are something we deal with every day in the world of digital imaging and output. In order to make digital photos larger, images must be "upsampled" using a software algorithm to add more pixels.
Because most digital photos are made up of a grid or "bitmap" of pixels, enlarging these images frequently results in either blurry or jagged results as the software is forced to either increase the size of the pixel (which produces jagged results) or "guess" what additional pixels should look like (which produces blurry results).
Software makers have created a variety of solutions over the years in an effort to solve the inherent problems of upsampling.
Photoshop alone offers quite a number of different enlarging algorithms you can use when resizing an image. They have names like "Bicubic", "Bilinear", "Nearest Neighbor", and "Preserve Details" and each one is designed to work with different types of images and for different enlargement (or reduction) tasks.
"Bicubic" works well for gradients. "Nearest Neighbor" works well for hard edges. "Preserve Details" works best for enlargements. But they all have their limitations and the larger you try to go with an image, the more you will begin to see flaws like jagged edges and blurred details.
We need a better way to make bigger images!
The first great leap forward in enlarging software came with the release of a plugin called Genuine Fractals Print Pro from Alatamira Software all the way back in 1999. This program used a branch of mathematics called fractal geometry to create an entirely new algorithm for enlargement that was capable of scaling images up or down with minimal loss of quality. The results it produced were far superior to the native ones in Photoshop like Bicubic and Bilinear and it really was a game-changer at the time it was released to the public.
Genuine Fractals was later purchased by OnOne software and this excellent upsampling algorithm still lives on today as part of On1 Resize and On1 Raw.
Alien Skin (now rebranded as Exposure Software) has their own upsampling plugin called Blow Up and Topaz has one called Gigapixel AI. Both do a fairly decent job of making enlargements with minimal quality loss.
The upsampling tools available to us reached a plateau over the last few years. There hasn't been much in the way of advancement until last week when Adobe released an update for Photoshop and Camera Raw that included a new feature they are calling Super Resolution.
You can read all the details in this blog post from the ACR team explaining what it is and how they developed it.
The short version is that they have used machine learning to develop an entirely new algorithm for image enlargement and it has pushed Photoshop right up to the front of the pack when it comes to the quality of the enlarged images.
As you can see from the example below, it does a noticeably better job than the existing Bicubic/Bilinear/Nearest Neighbor routines when it comes to enlargement. The results are smoother with far fewer artifacts and interference patterns.
Super Resolution Upsampling
How the feature was developed:
"The idea is to train a computer using a large set of example photos. Specifically, we used millions of pairs of low-resolution and high-resolution image patches so that the computer can figure out how to upsize low-resolution images.
With enough examples covering all kinds of subject matter, the model eventually learns to up sample real photos in a naturally detailed manner.
Teaching a computer to perform a task may sound complicated, but in some ways it’s similar to teaching a child — provide some structure and enough examples, and before long they’re doing it on their own. In the case of Super Resolution, the basic structure is called a “deep convolutional neural network,” a fancy way of saying that what happens to a pixel depends on the pixels immediately around it. In other words, to understand how to up sample a given pixel, the computer needs some context, which it gets by analyzing the surrounding pixels. It’s much like how, as humans, seeing how a word is used in a sentence helps us to understand the meaning of that word."
The programmers at Adobe concentrated on training the system using "challenging" images with lots of textures and small details. These types of images typically do very poorly when enlarged resulting in heavy artifacts so they were excellent choices for training the algorithm.
The results of Adobe's research and development are nothing short of amazing.
The new "Super Resolution" enhancement represents the next big leap in upsampling quality and it comes free with your Creative Cloud subscription. The feature is currently available as part of the recently released Camera Raw 13.2 and Photoshop 22.3 updates and it will be added to Lightroom Classic and Lightroom Cloudy in an upcoming release expected within the next few weeks.
I've had a chance to try it out on a variety of images and the results are truly impressive. I was able to upsample the images with no apparent loss of quality allowing me to print them at considerably larger sizes. Combine this with the sharpening and de-noising tools available from Topaz Labs and you've got a recipe for making high quality enlargements that don't suffer from any of the typical detail loss or artifacts seen with older upsampling routines.
Sounds great! How do I use it?
Super Resolution is capable of working with both Raw files (CR2, NEF, ARW, DNG, etc) and linear files (JPEG, TIFF, PSD) but typically does best when you give it a high quality Raw file to begin with. The cleaner your image is to begin with, the better it will look when enlarged. If you are starting with a JPEG that has strong artifacts, those are likely to be exaggerated as part of the enlargement process. If your source image has a fair amount of noise, I suggest running it through something like Topaz DeNoise AI first so you don't amplify the noise when you enlarge the image.
Because the Super Resolution feature is part of the Camera Raw architecture, you'll need to open your file using Camera Raw in Photoshop regardless if it is a Raw or Linear file to begin with.
Raw files are easy to open in Camera Raw and usually do so automatically. TIFF, JPEG, PSD and other linear files will require an extra step to get them open in Camera Raw.
If you are coming from Adobe Bridge with a linear files, you can Right-Click on it and choose "Open in Camera Raw" from the pop-up menu. This will bring the file directly into the Camera Raw interface in Photoshop.
If you aren't using Bridge and want to open the file up directly into Camera Raw within Photoshop, it takes one extra step. From the Open File dialogue in Photoshop, choose any linear file you wish to open and change the Format to "Camera Raw". This will tell the program that you wish to open the in Camera Raw even though it isn't actually Raw to begin with.
Note that you cannot open a linear file normally within Photoshop and then apply the Camera Raw filter. It must be opened up directly into Camera Raw or the Enhance feature will be unavailable.
Once you have your Raw or Linear file opened in Camera Raw, right-click on the image and choose Enhance... from the pop-up menu that appears. From there you can choose Super Resolution. (The Enhance Details option will be automatically selected when you choose Super Resolution)
Click the Enhance button and wait for the program to process your image. The task is very processor intensive. Depending on the speed of your GPU and storage, this can take a few seconds or even a minute or two to complete.
When it finishes, the program will have created an entirely separate (much larger) DNG file which will be visible on the filmstrip at the bottom of the Camera Raw window. You don't need to do anything to "save" this file as it is automatically saved to disk in the same location as the original as part of the enhancement process.
Because the file is being doubled in size both horizontally and vertically, the result is actually four times larger in overall area and megapixels.
In the case of my camera, applying the Super Resolution enhancement to a 20-megapixel original file (5272x3648) results in an 80 megapixel enlargement (10944 x 7296). And because the output file is a DNG, you can continue to edit it using the tools available in Camera Raw. The output produced by this new algorithm is remarkably free of artifacts and retains all the details of the original without getting blurry or jagged in the process. It really does end up looking like a larger version of the exact same picture. Honestly, the results of my first few tests were so clean that I wasn't sure it had even done the enlargement until I went in and looked at the pixel dimensions to confirm it really was four times larger overall.
One last tip:
Because this is an enlargement algorithm and not a sharpening one, you may also want to apply some extra sharpening after the upsampling to enhance some of the details in the enlarged image. The sharpening routines currently available in Photoshop are decent enough but a dedicated program like Topaz Sharpen AI will do even better on a job like this.
Adobe has done a superb job of developing their new Super Resolution tool and integrating it into Camera Raw and Photoshop. I look forward to it being part of Lightroom Classic as soon as possible. This is something that has broad application across the entire field of photography.
I can't wait to see what the programming wizards at Adobe come up with next using this kind of machine learning to train their algorithms. If I had to guess, based on the blog post linked above, I suspect they will use this same training technique to develop better sharpening and noise removal algorithms of their own. I'd love to see them come up with something built-in that can give Topaz DeNoise AI and Sharpen AI a run for the money.