Two exciting new Enhancement models & interface updates (September 2023)

Topaz Video AI Roadmap
1

Nyx: The all-new denoise model

Reduce noise and fix compression artifacts with our first AI model designed specifically for cinema-grade footage.

Model strengths: Nyx is optimized for denoising high resolution out-of-camera videos captured in less than ideal lighting conditions with high ISO settings. You can now shoot videos at higher ISO levels with confidence, knowing that Nyx can clean up the results.

Nyx currently runs in 1x upscaling mode only, so users get the best possible denoising without additional AI processing or resizing.

With Nyx, videos retain the highest amount of image detail while significantly reducing noise and compression artifacts, like stair-stepping and reverting chroma subsampling.

Nyx is also a great tool for reversing the effects of video compression. Many of our users are working with media that has previously been encoded to H264, H265, or another compressed delivery codec. Even in Auto mode, Nyx can intelligently detect and heal some of the visual artifacts created by video compression:

nyx-compression-closeup
nyx-compression-closeup3840Ă—2160 4.63 MB

Iris v2: improved details for low resolution faces

The new version of Iris includes a major upgrade to the results from v1. You can use Iris v2 to get more natural and realistic details in both human and non-human subjects, but particularly in human faces. Refined training allows Iris to more accurately generate facial features and details, including features like beards and skin. While Iris v2 has a focus on human facial feature details, it also works great for non-human subjects like trees and buildings.

iris-still
iris-still1920Ă—1080 93.3 KB

Iris v2 also reduces the “ghosting” effect that could be seen in footage with changing scenery and subjects, as well as the “marching ant” artifacts (diagonal movement of textures).

Lastly, Iris v2 includes a massive performance increase for NVIDIA GPUs using the power of TensorRT. The new native models may result in speed improvements of up to 50%.

comp3.00_00_04_21.Still002
comp3.00_00_04_21.Still0021920Ă—1080 126 KB

Other improvements

  • Video AI 3.5.0 adds support for rotating videos 90, 180, and 270 degrees. Use this to correct videos with missing metadata for rotation.

  • A new advanced feature in the app’s preferences enables the use of two Enhancement filters, which pro users have previously incorporated into their workflows by exporting and re-importing a large intermediate file. This new system will allow editors to stack Enhancement filters and process multi-step AI tasks in fewer clicks.

  • User guides in the app have also been improved, with animated tooltips that instruct new users on the use of the Preview system the first time the app is run.

  • Improved dynamic bitrate controls: The High setting for H264 and H265 exports has been adjusted to target a VMAF score of about 95, and is now the default setting for new installs of Video AI.

  • The lossless FFV1 codec is now available by default in the export options panel, and can be used at 4:2:0, 4:2:2, and 4:4:4 chroma subsampling levels.

  • Adds AV1 encoder for AMD 7000 series GPUs and 700 series integrated graphics.

  • Adds change to copy all subtitle tracks for MKV exports. Support for more containers will be added in the future.

  • New preference to default to searching for image sequences during import.

Please read the release threads for 3.5, 3.4.4, 3.4.3, 3.4.2, and 3.4.1 for the full list of changes in each release.

What’s Next?

We have several foundational improvements in the pipeline that should significantly improve how useful Topaz Video AI is to you.

Workflow improvements

  • Direct integration with popular NLEs
  • Improved preview experience that allows easier comparison between settings - this update is currently available as a very early alpha version which our Beta Testers group can access here.
  • Allow offloading processing to a cloud backend
  • Allow pausing and resuming processing

We plan on shipping most of these improvements before the end of 2023. Please apply for the video beta program if you’d like to shape the development of these features. Thanks, and we look forward to hearing what you think!

8 Likes
2

@tony.topazlabs
Iris V2 is a Big Step backwards!
It introduce extremely “Motion Jitter ”, which IrisV1 has not! :rage:

1 Like
3

Test image with FFMPEG’s built-in testsrc filter. ver.3.5.0

Nyx has less color alteration.
For example, the red color changes from R:G:B=255:0:0 to R:G:B=254:0:0, which is within the margin of error.

Iris v2 has some strange color transformations.
For example, red goes from R:G:B=255:0:0 to R:G:B=238:14:11, which is fundamentally wrong.

If the color balance is made absurd, it will affect the grading process, so please consider an option that does not change the colors.
Other models can have less color alteration, so why can’t Iris do it as well?

FFMPEG testsrc
FFMPEG testsrc & nyx v1
testsrc-none_010
testsrc-nyx1_010

FFMPEG testsrc & FFMPEG scale x4
FFMPEG testsrc & Iris v2 x4
FFMPEG testsrc & GaiaCG(GCG) x4

testsrc-scale_010
testsrc-scale_0101280Ă—960 33.9 KB

testsrc-iris2-4x_010
testsrc-iris2-4x_0101280Ă—960 41.2 KB

testsrc-gcg5-4x_010
testsrc-gcg5-4x_0101280Ă—960 34.1 KB

6 Likes
4

Hi. With the latest version of VAI I’m getting a jerky motion in the first second of my video when I slow it down 2x times with the Chronos model. I’m using an image sequence as a source. Please check the video and the image sequence used.

balcony_chr2.zip (736.0 KB)
camera_balcony_image_sequence.zip (18.2 MB)

5

Hello,

Thanks for uploading these files for testing. I was able to export a smooth result using the Apollo model, which is optimized for slow motion conversions. I’d recommend trying Apollo for 2x slow motion and comparing the smoothness of the results.

2 Likes
6

Nyx model look great, but without sharping for me is useless.

7

Thanks for checking out the files. I tried Apollo too, which doesn’t have the jerky motion at the start, but it has 2 issues of its own: 1. It gives me a lower quality image than Chronos and 2. it seems to repeat a couple of frames at the start of the video.

8

I hope the developers are aware of this now “KNOWN” problem and will try to fix it soon! :melting_face:

1 Like
9

Nyx seems to work a bit like Artemis where the noise is nicely removed but the resulting faces are more plastic looking and less sharp, especially with older vhs footage.

Honestly I’ve been disappointed that after more than 2 years paying for Topaz, they still cannot seem to get rid of the unnatural plastic look. I thought Iris was the answer but it was not.

Maybe next year or perhaps the plastic look seems natural to the developers.

For now it’s back to After Effects and Vegas Pro to improve image quality for me.

2 Likes
10

Why is there still no way to reorder videos?

11

My opinion after testing is Iris V2 is generally a good improvement, and the controls are more effective than in V1.

You sound like a broken record, catching up on the forums tonight if I had started taking shots every time you posted about motion jitter I would surely not be able to write to you now. None of these models perfect, but you may have noticed they are already discussing Iris-V3 plans, so maybe chill with the angry emoji, bold text, and aggressive @'ing in every single thread. I am certain they have noted your many complaints. V1 is still there, and you can use it without having to roll back program versions, so that’s nice.

5 Likes
12

yes, you are right, IrisV2 is not optimized for LowQuality Video.

But the Problem is:
They released a Beta-Version 3.5.0.0.b, and there was no Motion-Jitter visible with IrisV2.
After that they released another Beta 3.5.0.1.b, and there the Motion-Jitter has been introduced! I have said thousands of times in the Beta-Forum that this is a big Step backward for IrisV2 (regardless of the fact that IrisV2 is not suitable for LowQuality videos).

But they didn’t listen and were desperate to release this unfinished version overly fast. And that was the Reason why I was annoyed again… They should rather take some more Time instead of releasing an unfinished, buggy Version :frowning:

5 Likes
13

Is this not available for Artemis?
Is that why I get the same wait times after upgrading from a 3080 Ti to a 4090??
Also, the times got extended back in April or May update, with like 1 hour for the shortest conversions (series episodes). So what happened there???

1 Like
14

Artemis already using Tensor RT model since v3.0
You can check the TVAI model folder, you can see that they are using TensorRT model.

15

Agree. Compared to non AI neat video, Nyx seems very disappointing. Temperal noise reduction in Resolve Studio also does a better job. The plastic look from Nyx needs to be addressed. The whole point should not no to lose fine details. Blurring an image is not what one expects.

1 Like
16

So why is there no difference in times between a 3080 Ti and a 4090?
Is somehow my 10900K processor a bottleneck?

17

You may want to check the results in Benchmark pages.
And compare with others using the same GPU.

1 Like
18

:crossed_fingers: hoping this means DaVinci Resolve support!

2 Likes
19

And Premiere Pro! :sweat_smile:

20

Christian, as a beta tester, do you have knowledge that Iris V2 was purposely not optimized for LQ? Or this is just feedback from your experience? I’m curious because I’ve seen other beta-testers say that V3 will be optimized for MQ-HQ in the release thread. Why would LQ optimization be nerfed? I think LQ is what a good many of us are using TVAI for in the first place and V1 was such a game changer for that. I’d be completely disappointed if LQ wasn’t part of the game plan in the future.

I also agree that V1 handles LQ much better as well. It does feel there was a rush to release V2. With a much anticipated release like this, I would have been happier had it stayed in beta longer and the bugs and optimization for LQ be worked out if that was intended.

3 Likes