Video Roadmap Update (Feb 2023)

Product Updates

We’ve made several improvements to Topaz Video AI since the January roadmap update, primarily focused on frame interpolation (FPS conversion / slow-mo) and Stabilization.

Frame interpolation can replace duplicate frames

Topaz Video AI can now detect and replace duplicate frames before running any frame interpolation model.

image

Duplicate frames confuse the AI model and often lead to choppy output, so you can now instruct Topaz Video AI to replace them with model output. You can see the difference between running Chronos Fast on footage in v3.1.1 vs v3.1.5 with Replace Duplicate Frames enabled:

Previously, you would have to remove these duplicate frames from outside the app. This change makes it more convenient to get smooth results from videos with duplicate frames.

Sharper Apollo model

The Apollo model also now creates sharper interpolated frames, particularly on footage with lots of motion. While the difference can be subtle, here’s a screenshot from a clip comparing the updated Apollo in v3.1.5 with v3.1.1:

apollo-change

We’ll keep improving the sharpness of interpolated frames, and have several early candidate models that show promising results.

Stabilization handles longer videos with scene changes

Previously, the Stabilization model occasionally created completely blurred frames for longer videos with scene changes or heavy forward/backwards motion.

We now handle many of these edge cases, so you should see more consistent results with Stabilization on longer videos. Here is an example from the same frame from a tricky video in v3.1.1 vs v3.1.5:

Also, you should see improved performance on Stabilization’s second pass and the Reduce Jittery Motion option on Nvidia GPUs.

Up to 400% speed increase for Nvidia GTX 16 series GPUs

As a follow-up to the major performance update in v3.1, we’ve significantly improved export and preview speeds for Nvidia 16 series GPUs:

We recognize the impact faster processing times have on your workflow, so we’ll continue to improve performance for various backends in future releases.

Other improvements

We’ve also fixed various other issues like slow playback on Mac, parameter issues on various models, and exporting to network drives. You can see a full list of improvements by reading the release notes for Topaz Video AI v3.1.5, v3.1.4, v3.1.2, and v3.1.1.

Feature Roadmap

Our two core focus areas remain unchanged this month:

  1. Video quality models for upscaling and denoising. While true face restoration for video is currently infeasible, we’re exploring options to improve how natural faces appear when upscaled.
  2. Smoother in-app preview and player experience, particularly with scrub performance and before/after sync.

We’re also continuing to work on several other areas:

  • Frame interpolation (FPS conversion and slow-mo) models and experience
  • Processing speeds for Nvidia and AMD GPUs
  • GUI stability
  • Explore feasibility of crash recovery and pause/resume functionality
  • Explore reducing file sizes on some encoders and containers
  • Other quality of life improvements like trim experience and batch-downloading models

You may have noticed that some features take longer than others, particularly model development and major app structural changes. There might not be release activity every week on every feature, but please do rest assured that we are working on our priorities as stated in the roadmap!

As always, we’re looking forward to hearing your feedback. Thank you for your support!

19 Likes

Great update!

Since face restoration for me is a priority, I would like to share a study made about face generation that makes a 3D recognition of the face. Maybe it can help for this process, since in video, the face is rotating and changing all the time. Here is the link:
GitHub - MrTornado24/IDE-3D: [SIGGRAPH Asia 2022] IDE-3D: Interactive Disentangled Editing For High-Resolution 3D-aware Portrait Synthesis

9 Likes

I know my GTX 1070 is an older model but still very much snappy in most areas of my daily work. Is there a chance to speed up processing for GTX 10xx series as well?

3 Likes

Any plans to let us select Models again like we did in the earlier 3.x versions? I am asking because Artemis Low Quality is what I usually use for upscaling Anime. Since you released ALQ v13 performance and results have but sub-part with artifact injection in dust/smoke/blurred areas vs ALQ v12. (For some reason ALQ v13 seems like it tries to inject a transparent layer or something that appears as grid blocks). With the latest versions, we can no longer select specific models that we want to use and are stuck with the defaults. Being able to actually select a model in the Preferences would be a nice feature to have again for more advanced users.

8 Likes

Excited to see it moving forward. One thing I would like to see improve is the natural look of faces and fingers in enhanced video. Sometimes the natura look is lost, and enhanced faces look unnatural. But I am sure you guys will get to that.

Agreed. Don’t let us anime and cartoon editors fall to the wayside :frowning:

That is very good to see the regular updates you are bringing !
Yet, I am raising again the question of adding the option to save files as MPX beside MOV, the latest are perfect for Mac users but MPX would be a real asset for PC users.

So happy to see more time being spent on the stabilizer which is significantly better than the first version of 3.0.

I do hope Topaz will improve speeds with all video cards including older versions of Nvidia like my RTX 2070. I upgraded my computer and video card specifically to work with Topaz and RTX 2070 was the recommended card at the time.

Happy to hear that you will be working in improvements to face upgrades. These days I only use Topaz for stabilizing because the unnatural plastic look on faces make most of my SD video unwatchable.

On a technical note I would appreciate it if Topaz could increase the width of the scroller on the right side. I find it difficult to get my mouse on it to scroll up or down.

Also as many have discussed, it would be nice to be able to use a mouse to choose the spot to trim a video as we could on previous 2.0 versions.

Talking about the trimmer, I’m finding that it’s off by a few seconds.

Hi, I like the way things are developing. I know that faces are problematical, however, would it be possible to have/create a ‘travelling mask’ to omit faces/hand from the processing in parts of the video where the distortions are to severe?

Either way, thanks for creating such a useful program. :smiley:

1 Like

OK, I’ve got to admit it, I’m getting older. LOL I have so many versions of Topaz, I don’t have any idea of which release is the latest. Is there any reason that when you release an upgrade, it doesn’t overwrite the previous version? I really don’t want to go through every version to find the latest. Any suggestions?

If you uprade via the program, that update clears the C:\Program Files\Topaz Labs LLC\Topaz Video AI directory completely before installing the new files.

You don’t have to download every version posted here to update.

Hi, Here’s my problem. Sorry, but I’m used to the new version overwriting the old one. But I seem to have all sorts of Topaz programs, and I can never find what I want when I need it.
And your reply doesn’t allow me to send you the screen shot of all the Topaz programs I have. Not good. Just remember, you may have a million customers that are aren’t happy with their failure rate, but when I’m trying to use the program and it doesn’t work, that’s a 100% failure for me.

Any suggestions?

Tnx

What exactly does the Duplicate Frames Threshold do? What is is the result of a setting of 0 versus 100? And what is the recommendation if I have choppy results with previous output? 0 or 10?

3 Likes

Indeed, it should have a specific explanation, according to my tests, it affects the range of replaced frames.

1 Like

I’m surprised that no one has mentioned processing progress feedback in CLI mode, which is very important for long videos.

1 Like

I agree a full explanation of replace duplicate frame threshold settings please.

2 Likes

I’m welcoming all the efforts being made and put into this version and future one’s so far!
Put I find it odd, that my 1 day old Mac Mini M2 base renders 3 sec. Video 4 times slower than a PC with RX 570 4GB
( I am referring to this channel for my verying analysis :grin: GPU speed Topaz Video Enhance AI - und GPU PCIe /VRAM Speed Test - Grafikkarten - Hackintosh-Forum - Deine Anlaufstelle für Hackintosh & mehr... )

My vids ran Yesterday all got around 1.2 frame per sec… regardless of the setting CPU or Apple M2 bound!!!

I think M2 GPU utilization falls behind quit obviously?!?:pleading_face:

x2
I own 1080 TI

What about AMD AI cores?..

2 Likes

While not perfect, I’d still like some integration with things that can improve faces like GFPGAN or codeformer. Again, these aren’t perfect, but they’re better than nothing, until a better solution is available.

4 Likes