Topaz Video AI Alpha v4.0.10.0.a (Proteus v4)

nipun.nath · December 21, 2023, 8:16pm

Hello everyone,

We have an Alpha build to test the new Proteus-v4 model.

4.0.10.0.a.model

Windows

note: Mac builds are currently not available due to some known issues

Changelog from 4.0.8.0.a

Please test and verify the following:

Handles scene changes
More details for MQ videos (play with Recover Detail slider)
Fixes background face enhancement for some cases

Proteus v4

In this new version, we aimed to expand the capability of the current Proteus by handling more severe cases (e.g. more noise, compression, etc) and more input video types (progressive, interlaced, and interlaced progressive). This version can also enhance human faces better than the Proteus v3. While the Iris model is dedicated to face enhancement, Proteus v4 would be the model to handle all face and non-face inputs equally well. Please test this new version and let us know your comments.

Changelog from Proteus v3

Handles more noise and compression
Improves faces
Works on interlaced and interlaced progressive videos as well
Better results on Auto
More details and temporal consistency
Better handles scene changes

Send us your feedback

Please let us know what areas you suggest to improve
Submit Files :
– Send us a short few seconds of input video and the corresponding output
– Share the ffmpeg command or settings that you used
– Leave a short note on what was the issue (e.g. artifact) or what you want to improve
– Any comparison video/screenshot with any of our existing model
Please also share what you like about this model so that we do not compromise that aspect of the model in the next checkpoint

Thank you so much!

Akila · December 21, 2023, 11:17pm

i tested few clips (the same I tested with v4.0.8.a), I noticed that the older prob4 (4.0.8.a) better cleans noise then the newer Prob4 ALPHA (4.0.10.a), even if I set the ROD to “0” it is still noisier then the previous Prob4 ALPHA, mainly in shaded areas. Level of details are about the same.

I liked better the previous Prob4 ALPHA, it provided me cleaner videos with the same level of details.

ssbroly · December 22, 2023, 1:35am

I think this one is a little better than the other, it removes a little less detail

david.123 · December 22, 2023, 2:11am

Wow! I will be testing this for sure!

david.123 · December 22, 2023, 3:47am

Well, regarding scene changes, my first test shows there is still an issue with interpolation and frames bleeding into each other at scene change points. It looks better than before but still not perfect:

Frame 1:

Frame 2 (frame bleeding is apparent):

Frame 3:

I think the onus needs to be on the interpolation model to deal with scene changes - it should be creating a duplicate frame at scene change points rather than generating a ‘hybrid’ frame.

gene-8240 · December 22, 2023, 4:18am

Have just run a couple of clips, but it appears that the new Proteus does a better cleaning up very low res (320x240) clips, but not as good on higher res. Maybe more than one variant for different quality input may be called for?

s_g_k · December 22, 2023, 8:19am

Why I am getting Topaz watermark.

TPX · December 22, 2023, 9:22am

I’ve also problems to solve it manualy, the estimating did a better job than me. ← i know they want to read that.

I try to find a sweet spot still for the video i’ve.

once the video looks clean i try to get a better result by raise the Recover detail slider.

We need a rating system, stars or colours, to rate the best result.

Seems like i’m happy with this settings.

Does not look that artifical.

comparison does not work, the frames are off by far too much.

ralfstula · December 22, 2023, 10:22am

I think Proteus V4 does even a great job with upscaling 4k to 8k videos. If you could optimize it for UHD videos it would be nearly perfect! Perhaps you could find a way to reduce the blue chroma noise that often appears when I try to brighten dark scenes with HDR.

TPX · December 22, 2023, 10:38am

Bug:

If I generate a preview with the same settings at two different positions, the comparison becomes unusable because it jumps around uncontrollably.

joachim.ehrhardt · December 22, 2023, 10:50am

I get permanently preview errors

all of a sudden (after some minutes processing continued) it delivers

I’m happy with the result

TPX · December 22, 2023, 12:13pm

Preface.

In the meantime I have compared the output of TVAI Proteus 4 with the procedure presented below and realized that everything is not so simple.

I like the area content of TVAI better, i.e. it has been able to collect enough information to improve something.

Where it has problems are areas where too much detail is lost through compression and it has to generate something from “experience” or use areas from other frames oder does simply use parts of the compression to fill something.

read through it, do your own tests and give feedback.

Because TVAI outputs a different frame number for individual images, depending on the position, a comparison is very difficult.

So frame 100 is not always frame 100.

I am now referring to the new Proteus model, I have problems with its decompression.

I have made attempts because I think there are fundamental problems with the decompression.

In this example we have 8x8 pixel compression.

The software has divided the “image” into 8x8 pixels and has reduced the pixel differences in the 8x8 “tiles” to make the file smaller.

The whole thing is based on the data bandwidth, the more details in the image, the fewer pixels are available to display details.

So basically the file is so fuxxxd that you can’t recognize anything anymore.

So, the biggest problem is the compression, TVAI works with it and takes the edges of the compression partly as details.
So that means that if we want to do anything, we have to get rid of the compression first.
It disturbs the whole video.

Because I am not a programmer and I cannot write a decompression algo, I have to do what is available to me.

So i added simple noise.

Basically, I would need an algo that fills the 8x8 pixels with the correct information so that the pattern is no longer recognizable, but I don’t have one, so I add the noise.

I also can’t rely on the other images (as imformation source) because, as described above, the bandwidth creates different patterns that don’t necessarily have anything to do with each other but are simply there.

I also don’t believe that this can be trained because it is totally random.

The compression pattern is now almost unrecognizable.

Unfortunately, the sharpness will ultimately suffer from this method.

Now that we have added the noise, we can remove it again.

Now the algos of the AI software take effect.

Here I have worked with TPAI (2x denoise) because it makes a pixel reorder.

I don’t know what the whole thing looks like in motion.

If you want to test this, you can add noise with another software until the compression pattern disappears and then use TVAI to remove noise or whatever.

Input

Nosie & TPAI

TVAI - Prob 4

Difficult.

The video that was made available to me is perhaps just really bad.

Imo · December 22, 2023, 1:25pm

I applied a cropping and then all my settings including the enhancement settings were turned off unexpected. Edit: the preview was set to Original for couriosity. Did I miss click accidently? Anyway my Enhancement settings are lost.

TPX · December 22, 2023, 1:28pm

I was able to test it but its realy difficult.

The Add noise slider in TVAI does add color noise, monochromatic would be better i think.

If i add noise via resolve the image becomes soft.

But resolve does add noise as overlay and does not change the pixel itself.

Imo · December 22, 2023, 1:33pm

Please do NOT turn down the brightness for unprocessed views. I can barely see anything!.

Faces blurry in the background still look strange after processing:

jcp29 · December 22, 2023, 5:24pm

This Proteus V4 is very promising but currently, I cannot stabilized in Full Frame, only crop with a clip which is very stirred or very damaged (poor S8 which requires a very large improvement).
When I am in full frame, there are whole blocks which stand out (for example 100x100 px) when there is a sudden movement. In crop it’s not too bad.

For the moment my best results are in 2x (1480x1080 to 2960x2160).
Hope this help.
Edit : one comparaison with S8 working

Imo · December 22, 2023, 8:39pm

the new proteus produces ugly hairs compaired to iris MQ. also overall iris MQ shows more details than protues v4 at least in this example with the same settings. Iris LQ is not worth mentioning here. I dunno why two examples show deinterlaced. I noticed that it is too easy to activate unwanted options my miss hitting or miss clicking.

left original, right iris MQ:

left original, right proteus v4:

left original, right Iris LQ:

ssbroly · December 23, 2023, 12:26am

I don’t understand why you left proteus on interlaced and the others on progressive. I also noticed that when you select Proteus v4 it automatically goes to interlaced, you have to manually set it to progressive

here are some original/final examples (with vai stabilization at 15)