Photo Roadmap Update (June 2023)

Product Updates

Since May’s roadmap update we’ve improved the Remove Noise model for non-raw files and fixed many smaller issues.

Improved Remove Noise for non-raw images

The new Remove Noise v2 offers substantially improved noise reduction for non-raw image formats like jpeg, tiff, and png. The new models generally produce less artifacts, preserve more detail, and offer more controllable sliders than before. Based on our quality testing, Topaz Photo AI now performs 13% better on average with default Autopilot settings on noisy images.

There are now four options in the (non-raw) Remove Noise filter:

  • Normal v2: Preserves maximum detail on images with a low-to-moderate amount of standard ISO noise.
  • Strong v2: Removes moderate-to-high amounts of standard ISO noise while preserving detail, but does not deal well with already-processed images.
  • Normal: Older model version that will be replaced in a few releases.
  • Strong: Removes non-standard noise, like when a noisy image has been upscaled, pre-processed, or otherwise damaged. Blasts away noise but also removes more detail.

You’ll see an improved ability to retain natural detail in the v2 models for most images:

v2 will also better preserve many textures:

Autopilot will attempt to select the most appropriate model and then determine the parameters to use. However, Normal and Strong v2 will now respond much better to parameter changes, so you can adjust your results with much more control than before:

You won’t see these options if you directly import a raw file, as we use a distinct denoising engine and AI models specifically designed for raw images. We will update Raw Remove Noise within the next few months, but in the meantime you can access the new models by first converting your image into tiff or jpg. Read more in the Remove Noise docs.

Other improvements

We’ve also fixed several other important app issues over the past month:

  • Fixed Face Recovery sometimes only applying to half a face, not showing in the preview, and working improperly when switching scales
  • Autopilot now sometimes selects the Lens Blur model and recommends higher Sharpen Strong values
  • Login prompt has updated messaging and no longer activates every launch in certain circumstances
  • Fixed downscaling and lowered limit from 0.5x → 0.2x
  • Improved raw handling by adding new lens correction profiles, fixing various color issues, and fixing several issues with XMP sidecar files
  • Keywords are no longer truncated when saving to formats that use IPTC metadata
  • Added / hotkeys for switching between images and Esc to close Preferences panel
  • Fixed app freeze when manually checking for updates fails
  • CLI now respects Autopilot preferences and can preserve folder structure with --recursive flag

For a full list of changes each week, read the release threads for v1.3.7, v1.3.6, v1.3.5, and v1.3.4.

Next

Within the next few months we plan on improving model output quality, reorganizing the right panel, and improving the export workflow.

  • Add a Refine Text filter that mildly restores detected text and removes artifacts caused by other models (in beta)
  • Improve RAW Remove Noise by reducing artifacts, enhancing strong noise handling, and adding ability to recover original detail.
  • Improve blur reduction (Sharpen) model quality
  • Fix occasional blurry patches caused by tile depth issues
  • Reorganize the right panel controls, allow editing detected noise/blur levels, and add Autopilot customization options
  • Improve the export flow
  • Improve batch processing stability and performance

Later: new tools with generative AI

Topaz Photo AI exists to give superpowers to professional image creators, so we’ve been exploring the potential of newer generative AI methods to give high resolution and realistic output for professional use cases. There’s several promising directions that we wanted to get your feedback on. Which new features would you find most useful in your work?

  • A targeted enhance function that generates highly detailed and realistic results of a single object within a small selection
  • Select and remove an object with a single click, and fill in the area with believable results.
  • Add natural bokeh to your image based on the detected depth of field of your image
  • Improve the visual quality of phone photos that often contain splotches caused by suboptimal on-device processing
  • Improve blurred or severely degraded text by selecting it and typing in what it should read

We believe in focusing on the most useful tools for professional use cases, so we’d love to get a chance to see your current workflow and have the opportunity to ask you questions about where we could be most helpful. If you’re interested in setting up a Zoom call with us to discuss your workflow, please contact us and we’ll schedule a time.


We’re looking forward to hearing your feedback! Thanks for using Topaz Photo AI.

11 Likes

Thank you for the update.

2 Likes

I just learned about tiling images at low-rez, using generative ai to describe the image and write a stable diffusion seamless tiling prompt to blow up each low-rez tile stitching together the 512px AI tiles into larger generated images. Initially I thought it was stable diffusion doing the scaling but turns out it is the traditional AI scaling like ERSGAN and gigapixel.

But researching some way to do this directly with img2img stable diffusion methods and building it into seamless enhance photo AI flow would be interesting.

As would using generative AI to identify images along with text to help guide enhancing, denoising/deblur to know what texture it is working with.

In the context of “Generative AI”, it would be interesting if you could completely regenerate and upscale a poor Source-Image with a new level of photorealistic detail :wink::+1:

7 Likes

This is the most important. :heart:
At the moment, the upscaled image is lack of realistic detail.
It will be great if the generative AI can regenerate skin texture, clothes texture, detail hair, realistic fingers, etc. :tada: :100:

10 Likes

My preferences for further development would be:

  • A targeted enhance function that generates highly detailed and realistic results of a single object within a small selection
  • Select and remove an object with a single click, and fill in the area with believable results.
  • Improve blurred or severely degraded text by selecting it and typing in what it should read
1 Like

Generative AI tools ordered by what I’d find most useful, starting with:

  1. Select and remove an object with a single click, and fill in the area with believable results.
    –– If you can deliver Photoshop-quality fill-in without being subscription software, that’s a total game-changer.
  2. Improve blurred or severely degraded text by selecting it and typing in what it should read
    –– I’ve had this problem a lot with improving Topaz photos. The picture looks great, but the text is a dead giveaway. This would be really amazing to fix.
  3. Add natural bokeh to your image based on the detected depth of field of your image
    –– Cool, but not as important to me.
  4. A targeted enhance function that generates highly detailed and realistic results of a single object within a small selection
    –– Seems great, but since it’s such a small area, I probably wouldn’t find this that useful.
  5. Improve the visual quality of phone photos that often contain splotches caused by suboptimal on-device processing
    –– Sometimes it seems like the iPhone already over-processes zoomed-in photos to the point that Topaz software doesn’t know how to work with it, but I’m not sure what more can be done.

p.s. I’m very excited that Topaz is looking into generative AI!

2 Likes

Agree, this would be killer! It wouldn’t have to add new objects in the sense of “add a dog”. If it removed selected distractions to create a clean background plate, that would be cool!

Your other wishlist items I’m on board with too. Though how it would replicate typed in text to match the original, vs all Times Roman, I don’t know. I’m more picky. I’d want same fonts/styles/sizes as original image.

And, b/c Android is the most sold phones…your phone cam wish would have to work for Androids.

Nice update! Here’s my list of new features going from most important for me to least important:

  1. A targeted enhance function that generates highly detailed and realistic results of a single object within a small selection
  2. Select and remove an object with a single click, and fill in the area with believable results.
  3. Improve the visual quality of phone photos that often contain splotches caused by suboptimal on-device processing
  4. Improve blurred or severely degraded text by selecting it and typing in what it should read
  5. Add natural bokeh to your image based on the detected depth of field of your image

These are the improvements I’d like to see, with iPhone especially

  • Select and remove an object with a single click, and fill in the area with believable results.
  • Add natural bokeh to your image based on the detected depth of field of your image
  • Improve the visual quality of phone photos that often contain splotches caused by suboptimal on-device processing

Hi Eric!

Because you’ve stated above that the target mkt is professional users (which is helpful to know!), that may make my opinion moot. But that won’t stop me from sharing it! If only for others to think about. :wink:

My priorities - based on your proposed options - are:

1- Improve the Visual Quality of (Android) Phone Photos - I have a ton of them. From different generations of phones (not all great cameras in them). Given they were phones & not cameras, I often want to crop out distractions then enlarge, sharpen, denoise what remains. And, the earlier phones didn’t support shooting Raw images…With all those factors, a Topaz product’s ability to salvage older (mostly) photos would be great. If enhancements to newer phone cam pics too - better.

2- Add Natural Bokeh (soft blur - not circles of light - like either smooth background blur or gradient blur relative to foreground, midground, background) to Photographic Images - My critical gotcha on this is I don’t want the A.P. to just detect where it thinks the blur should be and where it should emanate from. (A) That would take away my creative prerogative, (B) I suspect it won’t place the gradient of blur accurately for each scene (including b/c I may want to emphasize a subject that’s unintuitive to A.P.). What I would be welcoming of is something like Topaz offered in the classic 'Lens Blur 'product… including the Depth Mapping with customizing features & control over the position of the key subject, sharpness plane & the blur areas (no matter where we want them to fall). Ideally the custom Depth Map created could be carried back into a host app like Photoshop and be used as an alpha channel and/or mask on adjustment layers. If there was both an Auto function (for pros who may care more about speed/profits than very high quality indiv. image outputs) + the Custom functions I’ve noted, that would be desirable.

3- Improve blurred or severely degraded text - Including both text that starts out bad (from motion, noise, poor lighting, etc.) & text that gets mutilated in the process of denoising and/or sharpening in PAI. I wouldn’t need this frequently. But when needed, it would be useful. And, I think it falls w/in the functionality personality of PAI.

4- Select & Remove an Object - Obviously you’d be going head-to-head with Firefly’s inpainting-style generative fill (& others of its ilk - DALL-E, Midjourney, etc.) so the quality output would have to be competitive & less costly. This is more outside the usual Topaz sandbox. But would be very useful (if on par with Firefly & its development trajectory - how much staff do they have to work on it …) for creating clean background plates - either to use as is or be able to composite other content into. Your note implies strictly inpainting vs outpainting (expanding & filling beyond the boundaries of an image…).

5- A Targeted Enhance Function - Would definitely be least important to me. I rarely do macro photography & my entire workflow is in Photoshop. So, I can very easily currently control where effects are added to my images on layers and using masked layers.

** Any of the functions offered in PAI (or, other offering…) I’ve noted above, I’d want supported in the Photoshop Plugin (just like all the current PAI components).

Thx for soliciting input! As a former tech mktg exec I know it’s challenging to ask customers to come up with options out of the blue. Having possible features to react to is great (& also means the company believes the options are within the realm of the developers’ skills & doable in our lifetimes!).

2 Likes

Which new features would I find most useful in my work?

  1. Add natural bokeh to your image based on the detected depth of field of your image.
  2. Select and remove an object with a single click, and fill in the area with believable results. Similar to this, (a) I’d like to be able to remove a wire (electric lines) and (b) I’d like to be able to remove the background and replace it with white/black/transparent pixels.
  3. Improve the export flow
  4. Reorganize the right panel controls, allow editing detected noise/blur levels, and add Autopilot customization options
  5. Improve batch processing stability and performance

Thanks for the information. I would find select and remove an object with single click helpful, plus improve the visual quality of phone photos.

1 Like

Of the 4 features you mentioned I would say remove an object with believable results is best followed by targeted enhance, the other two I have no particular preference. However, above these two the feature I would most like to see would be an ability to train to a specific face for face replacement. I use Topaz AI mainly to upscale old photos from old phones and scanned in photographs while the results look great, they are not the actual subject and therefore worthless to me for this purpose. In the spirit of AI with all that is coming out now this seems like an achievable milestone no?

1 Like

It seems like PhotoAI has all the tools needed to select photos that are properly focused and have low noise content. PhotoAI could scan all the photos in a directory/folder, find the ones with great focus, low noise and even good exposure (as does the FastRawViewer application), then Rate them via 5 Star or Colour system. Giving users the ability to set thresholds for Fine Detail, less than X percent noise, etc would be loved by most photographers.

I am sure Topaz has already considered using PhotoAI as a photo culling application. Is there already an ETA on when it might be available?

Thank you!

1 Like

Interesting idea!

Now if it could cull from hundreds of images based on composition too, then I’d get really excited! (But I would end up with a lot fewer images to process in Ps too).

That looks excellent! Maybe I’ll go back and re-enhance some photos that I used DeNoise on in the past.

I’ll follow @hkeely and order this list from most wanted to least wanted with reasons.

  1. Targeted Enhance. Sometimes Photo AI gives a good upscale overall, but some small object or feature doesn’t look good and can disproportionately affect the whole image. A while ago I did some testing with Gigapixel (I think version 4) and concluded that a 4x upscale with Gigapixel was on average “equivalent” to a 1.6x “perfect upscaling” (I assume Photo AI is better), in part because Gigapixel had trouble with realistically upscaling small regions of heavy detail. That is understandable, so a targeted enhancement feature would be welcome.
  2. Improve blurred or severely degraded text. The same reasons as Targeted Enhance apply here. However, since Topaz has already added a Refine Text filter, this feature is not as high on my list of priorities at this time.
  3. Visual quality of phone photos. I take tons of photos on my iPhone, and while many of them are “good enough” for me, it would be helpful to upscale and improve some images, especially those from the ultra-wide lens which are of much poorer quality than the main lens. (To my understanding, those with telephoto lenses are in a similar boat.) Not to mention, the ultra-wide lens captures 4x the content at the same 4032 × 3024 pixel count as the main lens (on my iPhone). So a photo from the ultra-wide lens will need to be upscaled 2x to have the same pixels per degree as a photo from the main lens anyway.
  4. Natural bokeh. I’m neutral on this one.
  5. Remove an object. I have this feature at the bottom because I feel that it’s currently a bit far from what Topaz currently does. The current feature set and the other features in the above list work to make a photo look closer to reality, while removing objects changes the reality. While I don’t have a problem with object removal and fill-in per se, I’d prefer that Topaz works on other features first.

I have some further comments and questions on a few of these proposed features for @eric (or anyone else).

  • Is there some technical reason why Targeted Enhance is (apparently) only for a small portion of a photo? Don’t get me wrong, I think this feature as stated in the roadmap is great, but in a vacuum I would presumably want the whole photo to be “highly detailed and realistic.”
  • Is the object removal feature related to outpainting? I admit that I don’t know how the proposed feature would work in Topaz, but a way to “fill in” the space beyond the image would be neat. I consider outpainting a photo to be a feature that “make[s] a photo look closer to reality,” since the human eye’s field of view is much wider than that of most cameras.

One suggestion I’d like to make is to activate Face Recovery only for faces that are smaller than a user-selected threshold (with some quality considerations). Often the “Auto” setting runs Face Recovery on faces that are medium sized and they end up looking worse than not running Face Recovery on them. Hence I end up spending time toggling Face Recovery and checking if a face looks better with or without it.

Recently, Photo AI has gotten to the point where I’m usually comfortable with setting everything on Autopilot…except for the Face Recovery.

2 Likes

I read that feature as providing an even more improved upscale/enhancement of a small portion of an image, so beyond what can be achieved with the existing models. Basically like Face Recovery and Refine Text but with more general objects/shapes/features.

But maybe I misunderstood.

If you’re right, then I move it to the bottom of my list.

1 Like

Sounds like the proposed option could use some more description from Topaz to clarify intent & ensure whether of interest to customers in our priority rankings!

I’d personally (depending on what’s intended) prefer a near-term development related to the Masking that’s already in PAI (and it’s a feature I’ve requested a few times since last Fall…) - that is, I’d like an Invert Mask option.

That way I could use Mask and/or Invert Mask inside PAI to determine which aspects of an image I can “target” to be enhanced via sharpening, denoise, face recovery, text enhancement, etc.

1 Like