2023 MacBook Pro M2 Max
64GB Memory
MacOS Sonoma 14.5 (23F79)
Topaz Video AI 5.1.4
AI models are trying way too hard to turn noise into teeth. This is particularly an issue with Iris, but I have encountered this with Proteus as well to a lesser extent.
The overall enhancement results are impressive, but the undesired teeth are problematic.
Maybe make “recover teeth” optional?
Here are my settings for this example for reference.
Setting Input condition to “Medium Quality” makes the problem even more pronounced.
Well I thought it was teeth as well in the original image.
This is the problem with posting stills of temporal signals (video). Perhaps the video motion would have indicated the “teeth” are just random noise flickers, or perhaps it wouldn’t. Impossible to say from a single image. But since you only posted a single frame, that’s all we have to go on to assess your comment, and as I wrote, it does look like teeth to me on that single frame 
I get what you’re saying, and the “teeth” only appear for a second before TVAI decides in later frames that it is, in fact, just noise.
The subject of the video is a two month-old baby, so there would obviously be no teeth at all, and certainly not the set of chompers TVAI created. I also recognize that TVAI has no way to know the age of the subject and therefore the appropriateness of teeth. This is why I suggested that something like “teeth recovery” could be a setting that is either optional (switch) or variable (slider).
For those who prefer it, here is the portion of the original video clip the stills were taken from. The frame represented in the stills is at frame 20 in the clip.
121-2135.mov.zip (737.7 KB)
Yes, it makes sense. Especially since you mention the “teeth going away” shortly after this frame, and might also not have been present prior to this one.
That sort of “temporal” information is what TVAI is already drawing on to make a lot of its enhancement. In fact, it’s the main thing that differs between the TVAI and the photo (e.g. gigapixel) models. So theoretically, this is a solvable problem for the team as they have all the types of information at their disposal to teach the models the difference between teeth and noise. It’s in the temporal signal.
Thanks for clarifying what the single still didn’t convey.
I replied to my original post with the video clip attached for reference.
Yup, definitely more non-teeth than teeth in that clip. Should be “easy” enough for them to fix.
Thank you for posting the video sample as well. The team will take a look at it to see how to work with the research end of things to further train the models on this type of setting.
This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.
