Honestly speaking the low res face on the left picture is also creepy.
Iris actually made it more like a face 
As you can see, there is hardly any face data that VIA could work with.
7 Likes
It would be nice if Iris could be applied only to faces and another model could handle the rest of the scene. A face detection algorithm could draw an outline around the face and mask in Iris output on top of a full scene output from another model like Proteus.
This would be a huge UI feat I suppose, but could allow users to disable face enhancement on specific faces (like the “creepy” one above), and also let you train Iris to do even better at faces while not having to train to resolve other issues in non-face areas of a scene (like the above ghosting).
As users we could do this manually in another editor, rendering out scenes twice and doing a manual overlay of Iris faces on a Proteus (or other) scene, but this is a tremendous amount of manual effort setting up mask keyframes, etc. Also the color/levels would need to be 100% consistent between the TVAI outputs or it would require manual color/grade matching, Very tedious.
I expect this would make performance worse also, having to use GPU/VRAM to do at least 2 models in the enhancement step.
Anyway, just an idea.
1 Like
I thought I’d tell you that I was dead wrong about my method to fix the dupe bug, so instead of RIFE, I figured I’d run the video through TVAI, and clean up the frames after, as it’s still exponentially choppy as I usually do some jobs with 240 frames as I want accuracy for some of my own 3D renders as I do 3D art as a hobbyist.
Yet, the only thing TVAI’s interpolation accomplished was to actually multiply the duped frames, so instead of: 1-2-3-44-5-6-7-88-9, etc… I receive this: 1-2-3-44444-5-6-7-88888-9, etc… ⇐ (It scales exponentially the higher/lower the frames.) I’ll run a test and load my videos/logs once I’m done testing, of course…
EDIT: I finished testing and uploaded the results.
So yeah, RIFE is the only choice for interpolating videos that are not the standard, my video is 25 fps BTW, also of note, Virtuadub2’s remove dupes plugin is still pretty fast, relative to the frame count that is, so it’s a great resource if you’re pressed for time.
1 Like
what is rife?
3 Likes
when you trim videos, it just blacks the screen but leaves the audio and file stays original length. this program is always one step forward and 5 steps back.
1 Like
For RIFE check out FlowFrames. That will get you started faster than looking at a paper and repo, and it’s free. Flowframes - Fast Video Interpolation for any GPU by N00MKRAD
2 Likes
I am sorry for my silence. After running this beta for hours I think it works quite good. There are some preview issues still btw. but the results and the processing speed is good.
Iris has a lot of dehalo built in that can’t be adjusted with the sliders. The ghosting you mention is a byproduct of that, as it is like a glow over the black parts. It is visible on some sources more than others. If the developer can reduce the dehalo effect, it would help a lot.
2 Likes
I remember from the early days of Artemis that the a. i. seems to hide image areas behind the shine when it is unable to improve the image there. if you remove the halo shine you might notice this. :eyes
Apart from the slow performance I’m actually quite happy with the Iris model. It really helps in upscaling LQ/MQ SD content where Proteus earlier did a decent enough job except for faces which sometimes were really terrible. Iris keeps a more natural appearance here and also has less “oil-painting” (with the right settings).
Compared to what I’ve got before with Proteus that IS an advance.
4 Likes
The only thing about flowframes is that in order to use the current version of RIFE 4.6, you will need the Patreon version, but that’s only around $6… (Donation at $6 per month) Also of note, the developer as well as the program takes a glacially long time to update the program/complete interpolating, but it’s still useful.
The current update was in February, and the one before that was in Sept 2022, and in all that time, mostly radio silence from the developer…
But to be fair, while it’s still only one developer, as a content creator/developer using that platform you still need to provide better support/progress in order for people to have a solid reason to pay money to support you financially every month.
EDIT: Another disadvantage with flowframes is that it cannot accept image sequences, unfortunately…
It works very well with 240p videos, particularly with videos where the faces are of a few feet and mid-range distances but still has issues with faces any farther than say, 10 feet (3.05 metres), with 20 feet (6.1 metres) you’ve got mutant-generator on your hands! 
This is from Cyrak’s “Beast Enders”:
It still has issues with facial approximation predictions, but that is mostly due to the model having to predict facial components from pixels affected by low rez artefacts, so it is constantly predicting over an ever-changing transition between pixels, as the artefacts disrupts accuracy.
The fact that it can accomplish this at all, which makes this tech pretty amazing in of itself, think of it as trying to approximate the correct face behind a pane of constantly changing patterned glass, so it’s quite remarkable, which should get better at predicting over the time of its development/learning.
The drawback is that it also ends up over-simplifying non-facial objects, so they end up looking like illustrations rather than realistic representations, but that too should get better over time, thanks to TPAI’s continued image interpolation improvements with it’s model!
NOTE: Cyrak, uses real-world images for his animations.
My guess is that the same tech used for TPAI will eventually be used with TVAI, which is different from a single image, as you have to account for an ever-changing progression of both the individual frames, and the artefacts across the video…
The good news is that TPAI has a new text interpolating model, so it will most likely make it over to TVAI, so hopefully, it will take care of the over-simplification of interpolating non-facial objects!
4 Likes
in a lot of low quality videos like this video, there are a few frames the face shown in good quality, this frames can be selected manually by user in VIA, and VIA enhance face in them and use them as reference to enhance the face throw the whole video, this is something like roop in stable diffusion, but instead of using it to make deep fake we use it to enhance the same person face, hope Topaz Team add this feature 
1 Like
That’s why I don’t overdo it with the “enhancement” and add quite some grain, that reduces that “painted” look quite a bit.
Even with larger crowds in the background (this is where earlier models often struggled and created “Zombie-faces”) Iris did fare quite well here:
But, yeah, there is always room for further improvement. Iris currently softens the output too much.
2 Likes
You know, I avoided grain, due to it writing over the original frames as I’m a stickler for original preservation, which is why I use/encode files with Huffy-Lossless, in between programs, but I think I’ll give it a go at the end of my pipeline as most of my finished projects are not going to be encoded again!
Some questions for developers:
Will Iris v1 continue to be trained so it can pick up some of the fine details that it currently cannot?
Is it possible to tone down the built in dehalo function in Iris, so that the user has more control in manual mode? Currently, in some videos, the dehalo ‘glow’ or ‘shine’ makes blacks look gray or pale (even when the dehalo slider is at 0).
Can face recovery be a separate function from the model (so it can be used with any AI model), similar to the functionality in Topaz Photo AI?
Will Topaz Video AI ever be able to remember user settings (without having to make a preset), so opening a new video or a batch of videos can have the same settings applied immediately?
Will starting a new preview ever be as quick as in version 2.6.4 (which was pretty much instant)?
5 Likes
Probably not. According to my experiences with early Artemis modes this dehalo shine hides ares where the a. i. was unable to improve the image properly…
Sure, but Artemis doesn’t have adjustable sliders - big difference. 
1 Like
After endless hours of video processing my Windows 11 Pro ended with a bluescreen and a sad smiley. I upload the log files to the dropbox shown above.
1 Like
