I need deinterlace video in Proteus model. Better faces (like in Iris), hands and text in Proteus model. Better tree/plants trained model in Proteus 
4 Likes
Not sure what the feasibility would be, but being able to upload HD reference footage or photos might be another way to enhance the output in certain cases.
Example: Deep Space Nine had a bunch of footage remastered for the documentary about the show “What We Left Behind”. It’s far from a full remaster, of course, but it does give a crystal clear look at what the show would look like in HD or 4K, so I wonder if there’s a way to train the model with that footage & allow it to inform the final output when trying to upscale the rest of the show. Sort of combining upscaling with more advanced generative AI.
It’d be useful for fans doing their own upscaling attempts, but I also imagine it could be way for studios to cut costs when it comes to remastering expensive shows like DS9 or Babylon 5.
6 Likes
- I use iMovie and DaVinci Resolve (free version), but integration is not a feature that I have a strong opinion on.
- Most if not all models besides Gaia have the problem of not providing enough detail in upscaled videos. The Recover Original Detail slider helps, but it doesn’t quite seem there yet. This month’s Photo AI roadmap mentioned the possibility of adding generative AI features. I wonder if it’s possible (as in the Photo AI roadmap) to use generative AI to create new details and textures to make upscaled videos look more realistic.
- I have some noisy camcorder footage that I’ll test with Proteus and Iris when I have time.
2 Likes
Yes I have also suggested this, it should be possible by implementing a training program that runs before processing the video, similar to how deepfake tools work. Optimally it should detect all the faces in the video and separate them by person like person 1, person 2, person 3 and so on and build a model for each face.
I imagine something like this would be able to greatly improve faces far beyond what a one-model fits all solution would. For instance there could be shots where the face is too far away to contain eye color information.
Upscaling that could lead to incorrect eye color being applied.
However if the video contain a different scene where the same face is zoomed in then that information would be available there and could be used in the scenes where the person is further away. And that idea could be applied to more than just eyes.
And then maybe you can run a second pass with Iris or something to upscale the resolution even further, because by that time there’s already enough detail in the face that applying a different model won’t make the face look like a completely different person.
3 Likes
I had a project that was already edited with the second closeup camera badly out of focus. I was having marginal results when I received the Iris upgrade. It was amazing. Iris focused not only the subjects eyes but the glasses as well. Perfectly natural. It also lightly tuned the other shots with out overdoing. It saved a reshoot. A very hard test and it worked perfectly.
2 Likes
+1 to a noise filter to enhance low light noisy footage.
Also, I would love for some of these enhancements to work at 1x. Sometimes the footage is ok but the faces are really blurry, so it would be nice to be able to enhance the face at 1x rather than upscale the entire video.
2 Likes
It’s great to have stabilization in the product, and for a lot of cases it works great. For very shaky videos, especially those where the camera has rolling shutter, I still find that it doesn’t remove some of the warping artifacts. It’s still not up to par with stabilization offered by tools like Premiere, Final Cut Pro or even iMovie. (I have in the past uploaded examples when trying out the betas)
I’d love some more future development work on stabilization and overall want to echo other posters in this thread who have asked for more focus on low-quality videos.
1 Like
I’m not using Topaz Video for low-rez videos. – I want it to work with Hi-Rez material: My #1 request is to sharpen up items that were created in 4k, but for whatever reason are not tack-sharp. Especially enhancing faces that were shot out of focus. On any production, there’s bound to be a shot or two that just misses, and being able to salvage these shots for an edit is really needed.
I’ve been able to do a bt of saving with current models. I was hoping Iris would be an improvement but it’s barely working on one test, and on another it did nothing (face looking sideways). - Any shot with people far away get’s some improvement with the old models, but any face up medium-to-close is not seeing improvements from Topaz at this time.
REALLY NEEDED! HUGE MARKET FOR THIS! : )
I had some success with one 1080p scene with missed focus on the subject’s face by downscaling it first to 540p in ffmpeg then upscaling it 2x in TVAI with Iris. I tried it on another similar scene and the loss of identity in the face was too bad so I gave up. YMMV.
1 Like
I use a great deal of found footage on projects I direct or co-direct. Much of that is from encoded VHS tapes. The approach mention could be beneficial. Also, I have creative directors wanting to update their reels by re-formatting some of their older spots depending on how they were encoded the results are mixed with current Topaz although Iris definitely game changer and hopefully improving upon it is also a priority. Thanks! Thoughts? Questions?
2 Likes
Agreed. What I’m seeing with Iris is both a vibrance and luma enhancement when exported. That really doesn’t bother me as much but better color management is desirable.
1 Like
I suppose there are times when auto color correction that AI makes during enhancing process, can be good. For example I’ve notice the human flesh tones are averaged according to probably average skin tone and sometimes slight color cast is corrected. Which is great. Except not so great if its always automatic and no way to turn it off or on. That is why we need color management, so that when we have a color correction or grading in the original we retain it. If the color appearance is not important, having that auto enhance feature would be useful, but it must be optional.
2 Likes
I think the future has to include facial recognition. Where a close up of someone would be able to be used to enhance the same face from a distance.
7 Likes
Well put.
2 Likes
Well put!
2 Likes
Iris is indeed an improvement for low res interlaced / progressive content however there are allot of vertical lines distortion appearing in most scenes which is a major disappointment. this does not happen with Dione TV, it would be great to fix this issue, most if not all Iris converted videos have these lines.
Samples
The other problem is faces far away appear very blurry and unrecognizable
Sample
Iris is also lacking detail even with restore original detail at 100%
Sample
Iris will be awesome if these issues are addressed?
9 Likes
I agree vhs to 1080p quality needed
7 Likes
The far away face example is a tough one. When you look at the source file and the export on the right, the results are extremely impressive. Am I looking at this image correctly?
For the patterning you are seeing, if you reduce recover details do you still see this?
1 Like
Yes, you are looking at the image correctly, left source, right remastered.
These are interlaced videos from DVD using video type “interlaced” option but same with issue with Progressive videos.
Most far away faces lack detail and appear really weird and distorted.
Sample
The lines or Patterning are still visible even with recover details set to 0.
Sample
3 Likes
Good tests and info! Thank you! Definitely Iris needs updating but I think you would agree that Topaz has found the proper route to conquer some of these issues.
1 Like
