Experiments with New Iris Model

After a few experiments with some my worst footage, I’m very impressed.

On my test footage previously, Proteus’s noise removal often created a strange ‘grainy’ texture that would appear intermittently in different areas which was quite noticeable on skin. Proteus did a great job overall with noise removal, but the intermittent weird grainy texture was often distracting and made the video feel somewhat “artificially altered”–a situation also reinforced by the plastic effect often seen on faces. Proteus also tended to produce strange artifacts on faces sometimes.

Every time I use Proteus, it requires a lot experimentation in Preview before finding the right compromise. And even then I often export several output videos using different settings to layer in Premiere so I can adjust opacity between segments to hide different Proteus artifacts. Very tedious work to say the least. Iris seems to have solved all of this!

The only issue I’ve observed about Iris so far is reduced detail and softening of the image. But I discovered that cranking up both the Sharpen and Recover Original Detail settings (+85 or so) seems to offset this problem.

Again, good work! And to anyone reading this who is using an older TVAI version because of an expired license, the new Iris model in 3.3 is definitely worth the price of upgrade in my opinion.

11 Likes

Going to have to try your 85+ settings. This is fun to experiment with!

Yay, Iris! It certainly is a promising start for our latest model. I am glad to hear that increasing Sharpen and Recover Original Detail is giving you the results you are after :slight_smile: Would you to see some examples :slight_smile:

2 Likes

me too am amazed by iris! a fantastic model that not only improves faces and on top of that is better than proteus :slight_smile:

4 Likes

I deliberately chose some low-quality input sources, and the effect of Iris looks much better than Artemis low.

So, iris is a good “overall” model that could replace Artemis Low? Or is it mostly focused on facial details?

I didn’t compare the facial details output by iris and alq because it wouldn’t be fair. On several test samples, iris has a better impression.

Initially I used IRIS with AUTO (which sets Recover Original Details to 20) and the resulting clip looked like petroleum jelly had been smeared onto the camera lens. I then tried Craig’s advice of setting higher values for SHARPEN and RECOVER ORIGINAL DETAILS but I had to set these to 100 (85 just wasn’t good enough). Actually I’m now experimenting with running the IRIS output clip through Proteus to sharpen it another 20 to 50 … otherwise the clip looks too dull and the colours still look washed out. It’s probably my very low quality source (a many generation VHS tape to tape copy) that’s making IRIS give me worse results than what others are getting. Anyway, thanks to Craig for his extremely helpful advice.

1 Like

I have one also where the original source footage is severely noisy and soft in focus. I jacked up sharpen and recover original details in Iris to 100 and still wished the slider option went higher. But in fairness, there’s only so much magic anything can do with very crappy source footage. When compared with the original, its very comparative but without the original’s severe noise and weird artifacts previously produced by Proteus. My intention is to sharpen the Iris-cleaned version gradually through up-sampling using Gaia in stages (150%), but I’ll also try your idea of doing an additional pass on Proteus.

One thing I really like about Iris is how it handles noise. If there’s residual noise in areas, it seems much more uniform and natural. Proteus cleans out some areas very well and then there is often an area where the the noise is converted into a patch of large-grained gritty looking texture. It’s very distracting when it happens and especially on skin. When Iris leaves residual noise, it appears more natural (uniform and less “gritty”). That’s a game-changing improvement in my opinion.

Regarding color shift, I haven’t noticed any changes. All footage I’ve tested looks exactly the same to me. If the color is too washed, that may be because it seems more noticeable when improved and can be color corrected in your NLE.

If Topaz is looking for input, my only wish is that the next model or revision of Iris increases the ability to sharpen more.

2 Likes

After a number of experiments, I haven’t noticed “improvement of facial details” per se. What I do notice is that Iris doesn’t turn people occasionally in ghouls (ahem, Proteus!).

2 Likes

Could you share a screenshot and the input file for testing?

FWIW, I played with sharpening up the softness of my Iris experiments during upsampling with Proteus. Still too much Proteus weirdness even with the settings dialed all the way down. I then experimented with Gaia, which is usually my go-to for gently sharpening clean videos during upsampling. In this case, I found Artemis Low is better since the Iris output is so soft. Plus Artemis was successful in getting rid of much of the residual noise left behind by Iris. A perfect combo!

Here’s the workflow I’m liking with Iris and old HD videos:

PASS 01) Clean up the source video with Iris at original resolution. Jack the max out of Recover Original Detail and Sharpness. Everything else to taste. If there’s some light noise left, no problem (Artemis will take care of it in Pass 2). No other treatments in Pass 1.

PASS 02) Stabilize if needed and upsample 150% using Artemis Low, playing with Recover Original Detail settings according to taste.

PASS 03) Before Pass 03, experiment in Preview with Artemis Low, Artemis Medium, Gaia, and Recover Original Detail settings. Select the best, and upsample the video 150%. It’s now at 1080 and time for Premiere.

Damn! Now I want to go back and re-edit every video I cleaned up last year! My inner perfectionist won’t allow me to sleep properly until its done.

3 Likes

I had my HDD misbehave and delete over half of my videos. So now I have to redo them anyway. It’s like the most cursed blessing ever.

Hi Craig. Your 3 pass plan sounds like a winner. I’m afraid Proteus after IRIS just made the faces (especially the eyes) look weird again.

Hi Ida. Here are snapshots before and after I processed it with IRIS with Sharpen and Recover Original Detail set to Max (100).
Still of original before IRIS
Still after IRIS with Max Sharpen and Recover Original Detail

Sorry, I tried to upload a sample of the clip but your upload option doesn’t allow me to upload my few seconds of mp4. I guess I could set up a link if I upload it to Youtube if you really wanted to see it.

Gary

2 Likes

I have found that there is no single Iris workflow that always works well, it all depends on the video quality and content. Cranking up recover detail too high can itself cause weirdness, for example I noticed weird ears in some scenes when the head is turning, and when certain things pass in front of a face, that too can be problematic with recover detail set high. The wavyness too on some video scenes is sometimes also noticeable, as mentioned by others in other threads.

But there are some definite improvements over all other models, for some videos/scenes especially faces, so as an initial release, things are looking good for the future with this model. You just have to be aware that there is no single workflow or set of settings that will work well for all material.

1 Like

I do agree on the workflow comment. The one I noted earlier seems to be working well with my noisest videos, but others maybe not. I haven’t noticed much weirdness with Iris yet–especially nothing comparing to Proteus. Proteus tends to solve one problem while creating a new one.

I do have a video now where the noise isn’t so terrible that I’m trying a different workflow to avoid losing a lot of detail and sharpness. Basically reversing the order by upsampling through Artmis and Gaia and then running a final pass with Iris. I’m curious if more detail is retained and if Iris can get rid of the upsampled noise.

My dream model…A model with Iris’s noise reduction and face & skin texture retention qualities combined with Proteus’ detail retention & sharpness capabilities.

3 Likes

Thank you, Gary! I think in this case a screenshot is enough :slight_smile: I have seen the same thing occur on my end.

@craig-Tampa looking forward reading on your new findings / workflow for the less noisy videos you trying out now.

The result wasn’t perfect, but overall worked pretty well for that video. I upsampled once using Artemis Low and Recover Original Details at around 80 and then upsampled the last step with Iris. I also did a second Step 2 video with Artemis Low for comparison and maybe compositing if I wanted to try to get the best from both by layering, masking, and adjusting opacity in Premiere.

I could tell from comparisons between the two Step 2 videos that some detail was lost in the Iris video. But it was still sharper than starting first with Iris and then upsampling with Artemis. Even though I lost some detail, the wonderful way Iris handles faces made it the winner. I later applied Unsharp Mask in Premiere to sharpen the fine details further.

I have another one like that I’m going to experiment with this weekend.


BTW, I also tried that experiment with Gaia. But Gaia sometimes does something similar to Proteus with noise where it “bakes it” into a texture (patch of yucky, dotty stuff or streakey lines) in a way that the models don’t recognize it as noise in later stages of upsampling. Iris couldn’t clean that stuff out. But Artemis removed a lot of the noise during the upsample from Step 1 to Step 2 and what noise was left was successfully reduced by Iris.

1 Like

FWIW, it also appears that turning up Anti-Alias/Deblur settings helps a little with Iris’s lack of sharpness.

1 Like