Topaz Video AI Alpha v4.0.10.0.a (Proteus v4)

[nipun.nath (Nipun)]

Hello everyone,

We have an Alpha build to test the new Proteus-v4 model.

4.0.10.0.a.model

• Windows

note: Mac builds are currently not available due to some known issues

---

Changelog from 4.0.8.0.a

Please test and verify the following:

• Handles scene changes
• More details for MQ videos (play with Recover Detail slider)
• Fixes background face enhancement for some cases

---

Proteus v4

In this new version, we aimed to expand the capability of the current Proteus by handling more severe cases (e.g. more noise, compression, etc) and more input video types (progressive, interlaced, and interlaced progressive). This version can also enhance human faces better than the Proteus v3. While the Iris model is dedicated to face enhancement, Proteus v4 would be the model to handle all face and non-face inputs equally well. Please test this new version and let us know your comments.

Changelog from Proteus v3

• Handles more noise and compression
• Improves faces
• Works on interlaced and interlaced progressive videos as well
• Better results on Auto
• More details and temporal consistency
• Better handles scene changes

---

Send us your feedback

• Please let us know what areas you suggest to improve
• Submit Files :
  – Send us a short few seconds of input video and the corresponding output
  – Share the ffmpeg command or settings that you used
  – Leave a short note on what was the issue (e.g. artifact) or what you want to improve
  – Any comparison video/screenshot with any of our existing model
• Please also share what you like about this model so that we do not compromise that aspect of the model in the next checkpoint

Thank you so much!

[Akila]

i tested few clips (the same I tested with v4.0.8.a), I noticed that the older prob4 (4.0.8.a) better cleans noise then the newer Prob4 ALPHA (4.0.10.a), even if I set the ROD to “0” it is still noisier then the previous Prob4 ALPHA, mainly in shaded areas. Level of details are about the same.

I liked better the previous Prob4 ALPHA, it provided me cleaner videos with the same level of details.

[ssbroly]

I think this one is a little better than the other, it removes a little less detail

[david.123]

@nipun.nath

Handles scene changes

Wow! I will be testing this for sure!

[david.123]

Well, regarding scene changes, my first test shows there is still an issue with interpolation and frames bleeding into each other at scene change points. It looks better than before but still not perfect:

Frame 1:

Frame 2 (frame bleeding is apparent):

Frame 3:

I think the onus needs to be on the interpolation model to deal with scene changes - it should be creating a duplicate frame at scene change points rather than generating a ‘hybrid’ frame.

[gene-8240 (GeneK)]

Have just run a couple of clips, but it appears that the new Proteus does a better cleaning up very low res (320x240) clips, but not as good on higher res. Maybe more than one variant for different quality input may be called for?

[s_g_k]

Why I am getting Topaz watermark.

[TPX (Thomas K.)]

I’ve also problems to solve it manualy, the estimating did a better job than me. ← i know they want to read that.

I try to find a sweet spot still for the video i’ve.

once the video looks clean i try to get a better result by raise the Recover detail slider.

We need a rating system, stars or colours, to rate the best result.

Seems like i’m happy with this settings.

Does not look that artifical.

---

comparison does not work, the frames are off by far too much.

[ralfstula]

I think Proteus V4 does even a great job with upscaling 4k to 8k videos. If you could optimize it for UHD videos it would be nearly perfect! Perhaps you could find a way to reduce the blue chroma noise that often appears when I try to brighten dark scenes with HDR.

[TPX (Thomas K.)]

Bug:

If I generate a preview with the same settings at two different positions, the comparison becomes unusable because it jumps around uncontrollably.

[joachim.ehrhardt]

I get permanently preview errors

all of a sudden (after some minutes processing continued) it delivers

I’m happy with the result

[TPX (Thomas K.)]

Preface.

In the meantime I have compared the output of TVAI Proteus 4 with the procedure presented below and realized that everything is not so simple.

I like the area content of TVAI better, i.e. it has been able to collect enough information to improve something.

Where it has problems are areas where too much detail is lost through compression and it has to generate something from “experience” or use areas from other frames oder does simply use parts of the compression to fill something.

read through it, do your own tests and give feedback.

Because TVAI outputs a different frame number for individual images, depending on the position, a comparison is very difficult.

So frame 100 is not always frame 100.

---

I am now referring to the new Proteus model, I have problems with its decompression.

I have made attempts because I think there are fundamental problems with the decompression.

In this example we have 8x8 pixel compression.

The software has divided the “image” into 8x8 pixels and has reduced the pixel differences in the 8x8 “tiles” to make the file smaller.

The whole thing is based on the data bandwidth, the more details in the image, the fewer pixels are available to display details.

So basically the file is so fuxxxd that you can’t recognize anything anymore.

---

So, the biggest problem is the compression, TVAI works with it and takes the edges of the compression partly as details.
So that means that if we want to do anything, we have to get rid of the compression first.
It disturbs the whole video.

---

Because I am not a programmer and I cannot write a decompression algo, I have to do what is available to me.

So i added simple noise.

---

Basically, I would need an algo that fills the 8x8 pixels with the correct information so that the pattern is no longer recognizable, but I don’t have one, so I add the noise.

I also can’t rely on the other images (as imformation source) because, as described above, the bandwidth creates different patterns that don’t necessarily have anything to do with each other but are simply there.

I also don’t believe that this can be trained because it is totally random.

The compression pattern is now almost unrecognizable.

Unfortunately, the sharpness will ultimately suffer from this method.

---

Now that we have added the noise, we can remove it again.

Now the algos of the AI software take effect.

Here I have worked with TPAI (2x denoise) because it makes a pixel reorder.

I don’t know what the whole thing looks like in motion.

If you want to test this, you can add noise with another software until the compression pattern disappears and then use TVAI to remove noise or whatever.

---

---

Input

---

Nosie & TPAI

---

TVAI - Prob 4

---

---

Difficult.

The video that was made available to me is perhaps just really bad.

[TPX (Thomas K.)]

I was able to test it but its realy difficult.

The Add noise slider in TVAI does add color noise, monochromatic would be better i think.

If i add noise via resolve the image becomes soft.

But resolve does add noise as overlay and does not change the pixel itself.

[jcp29]

This Proteus V4 is very promising but currently, I cannot stabilized in Full Frame, only crop with a clip which is very stirred or very damaged (poor S8 which requires a very large improvement).
When I am in full frame, there are whole blocks which stand out (for example 100x100 px) when there is a sudden movement. In crop it’s not too bad.

For the moment my best results are in 2x (1480x1080 to 2960x2160).
Hope this help.
​Edit : one comparaison with S8 working

[ssbroly]

I don’t understand why you left proteus on interlaced and the others on progressive. I also noticed that when you select Proteus v4 it automatically goes to interlaced, you have to manually set it to progressive

here are some original/final examples (with vai stabilization at 15)

on the last one with the red boat, we can see the rope on the boat and the two antennas much better in HD

[ssbroly]

yes, when you have a lot of parameters activated, such as stabilization, deblur motion, two passes, grain etc. you have to be careful that everything is activated as you want

[annemartensa]

Interesesting setting Especially the recover details 1 X 80 and 1 X 20 I normally use a lot of deblur to deblur old videos and leave the rest default and yes recovery detail to 60 to minimize artifacts I preseume this protheus v4 is for bad videos old vhs tapes that at a very late stage got digitized 30 years later indeed no Iris or protheys or Nyx artifacts but still a bit noisy in my first run

[annemartensa]

Got It
Protheus 4 is the best for my 1995 videos
I tested iris 1 iris 2 with theia fidelity and Artemis HQ then the SD Dinone and Iris 1.0 and iris 2.0 with and without Recovering detailed percentage.

For UK X factor 2006/7/9/8/9 Artemis HQ 2 X and Dione clean up SD is and Recover Detail 95 percent is better than the original, but…

My marriage from 1995 in Hungary from VHS to MP4 then anything with Topaz was a bit or more frustrating

Till I used protheus 4.0 X 2 increase and did an manual estimation on several extreme points in the video then set
the slider of the most extreme a bit lower 20 percent as the estimated are not giving good results still to many i ncorrect faces

Now it is very important to set the recovery details to 0 not 20 and let protheus 4.0 do its work use the slider after the estimation a bit to left when having set it to relative to automatic

Protheus will take much longer than iris LQ or MQ but gives so much better results than Iris frget iris protheus 4.0 with some work is the LLM to woork with your old VHS noisy videos

I have a few more videos Team keep up the Good work

Found some very very bad videos from 1994 copied to a VHS tape of 3 hours then forgotten and 2 years ago digitized hardware based to mp4. I used protheus 3 since version 2.7 but was never ever happy with the result the tapes MP4 s were to so incredible noisy vague but at least we had a videocamera in Hungary just 5 years after the fall of the wall. But with protheus 4 everything has changed the estimation is better much better and so is the handling of the source input Wow

[ralfstula]

After the first tests, I noticed that the rendering speed of the Iris (first) and Nyx 2 (second) models in combination is significantly higher than in the previous versions (approx. 10x faster). I tested the same video with version 4.0.8 and 4.0.10.a in an i9-9900KF with an RTX 2080TI and in an AMD Ryzen 9-30900x with an RTX 3080. So far, I have not been able to see any relevant difference in quality. The settings for sharpness and denoising are also more sensitive and cannot be transferred 1:1 from version 4.0.8 to 4.0.10.a. I am thrilled! Keep up the good work! If version 4.0.10 had a live preview during the preview and also during the final export, it would be perfect!