Hi all,
I’m experimenting with converting some 4k 60fps to 120fps and having very underwhelming results. I usually just use Speedwarp + Optical Flow in Resolve. I figured that Topaz could do a better job, since it’s built for this.
I’m currently trying export settings set to 2x, Apollo, Duplicate Frames set to Replace, and leaving Sensitivity to 10. I also trying giving Topaz a Rec709 clip, instead of Log. Same result.
What are your go to settings? Thanks!
For 4k you’ll probably need the new Aion model. Uncheck Duplicate Frames Replace. It does not do what it’s supposed to.
That should do it.
I’ll give it a shot, thanks!
Well this is odd…It keeps telling me i’m out of memory when I’m monitoring it and I’m clearly not. Topaz uses about 4GB of my 64GB unified (Mac Studio M1 Ultra), and then it gives me that error…
I’ve heard something about that on Macs. Is there a way you can dedicate more RAM to the GPU?
Maybe turn the Max Memory down to around 10% in TVAI.
Yeah, no luck. Technically, Topaz has access to around 48GB of VRAM, that’s how Resolve allocates my 64GB of Unified RAM and it works great. I can’t get Aion to work at all.
I’m hoping someone with a similar Mac will speak up.
Saturday I did slowmo tests with Aion and a 4K clip @25FPS in input to 2x slowmo 16x output + the double defined images on replace and leave sensitivity on 10 (deflault).
Very slow but beautiful results.
Windows machine with 64 GB RAM:
In the Windows control panel: FFMPEG RAM use: +52GB max (Oops …). 1st time I see as much use RAM by a single program.
Hope this help.
Nice! I get great results in Davinci using Speedwarp + Optical Flow. But I wanted to see how it compared. I couldn’t get Aion to work at all. Instead memory issue even though I clearly have plenty of RAM / VRAM to offer the program. Sounds like it’s an Apple Silicon compatibility issue.
Well I could try here on my Mac Studio M2 Ultra with 64GB RAM - but I don’t have a 4K 60 fps video at hand.
Maybe you could provide a short snip of the vid you’re trying?
Yeah, I’ll definitely send one! I have an M1 Ultra with 64GB unified ram as well.
Here you go!
Obviously, this a really tough shot for AI to handle, but Resolve’s Speedwarp did a decent job. Better than the Apollo model.
I gave you an H265 Vlog and rec709 converted prores version of the clip
https://drive.google.com/drive/folders/1R_2VatWACCGkkpnAu94UwBm6CY4zJAe4?usp=sharing
Links below to download.
Hi, are you able to elaborate on the duplicate frames issues (“It does not do what it is supposed to”)? I have seen a few strange results in particular circumstances where I suspect Aion wrongly decides to drop frames such as when distant objects not particularly clear in old analogue SD material seem to result in a dropped frame and also when an object moving quickly suddenly appears in the frame heading bottom to top or top to bottom (but not left to right or vice versa). I am wondering if that has something to do with your observations.
And infact I just repeated my testing again just now following your suggestion (to uncheck that check box) and it fixed the problem so thank you!! I would never even have considered unchecking that checkbox myself since intuitively I would want Aion to do what it advertises there! I’d been mucking with the sensitivity slider instead and all that did is either nothing or made things 100 times worse.
I wonder though if unchecking it now introduces some undesired side effects? Hopefully Topaz can comment on this issue and I am happy to provide two sippets of tests input footage that precisely shows the problem perfectly.
I know what it was suppose to do, but it never has done it. It was supposed to fix the motion of duplicate frames. Standard NTSC DVDs from films have an extra duplicate frame every 6th frame. The interpolation models treat that duplicate frame as intended motion. Apollo being the worst. It will regress the motions a tiny bit.
The best fix for it right now is to transcode the DVD and drop duplicate frames before sending it to TVAI.
Thanks, well it is 100% causing unwanted side effects in every example I have tried of PAL encoded 576p25 motorsport footage processing to 1080p50. It basically appears to misinterpret frames where motion is actually occurring as frames where it isn’t occurring. And as a result it is deleting frames that it should have kept all along. Unchecking the checkbox resolves this issue 100% however and in my testing over the last couple of hours I have not seen any undesirable side effects from unchecking the check box.
I will be doing a full hour of test footage in the next day or so once I am certain I am not seeing any issues in the 5 minute snippets I have been testing today.
In any case, now that I understand what it is meant to do, I shouldn’t have ever had that box checked to begin with when using broadcast PAL footage as input. I am not really sure why Topaz would have it checked by default to be honest however it is still 100% my mistake for not understanding the point behind it and researching beforehand. But you have saved me the trouble. Thanks again.
Hi,
Anyway, I redid the test with the clip “PS525563_TRimmed_Prores_1.mov” of the Slowmo X16 with the Aion model (without enhancement):
Whether the "Duplicate Framme replace is checked or not: in the end you get exactly the same number of output image.
At the start: 85 frames, at the exit of the slowmo: 1345.
Good luck.
I just did more testing this afternoon with Aion and the duplicate frames checkbox checked / unchecked and if checked, varying thresholds (sensitivity). I have infact found an unwanted side effect of unchecking that checkbox altogether even when the source footage is straight 25p with no source duplicate frames at all (so everything on screen is always moving all the time). It occurs when a fast moving object moves either directly away from the viewer or directly toward the viewer and the object’s “speed” in terms of how fast pixels on the screen actually change as a result of its movement is vastly lower than would be the case if that object was simply moving from left to right or vice versa (since obviously something going across your view at 100 miles per hour whizzes by if it is close to you but if it is coming directly towards you you obviously do not get the sense of speed until it is close. What happens is that when the object gets close to the camera, Aion appears to duplicate a frame in the actual output which then comes across as a very smooth stutter (a contradiction in terms but that is the best way I can describe it).
I am currently seeing if there is some setting that actually works at both extremes - a very slow moving object a very long way from the camera and the very fast object moving towards or away from the viewer.
A quick test seemed to suggest checking the duplicate checkbox but lowering the sensitivity threshold to something like only 1 or 2 may possibly work (with the very limited tests I have just done) but I need to do way more testing. Either way, I am still wondering whether Aion is behaving the way its engineers would expect it to be behaving in these particular scenarios.
I just hope I can find a “set and forget” setting for the soucre material I am working on, otherwise I could easily spend the rest of my life on it…
You might have hit the limit of what the model can accurately do. Most likely it’s from lack of similar training data. Because side to side motion is coming out great, they must have a lot of that in the training data. I remember someone sharing a video that rolls the whole picture while zooming in. Apollo could not handle it well at all.
Actually I think the Aion model is fine and it is some sort of limitation on my TV when it comes to dealing with 1080p50 content. I have noticed it’s ability to suppress very mild judder is not 100% at times and I think this is a case in point. I believe there is actually some bug as it relates to judder and the TV changing both refresh rates (as requested by the input device) and whether the content is 16:9 or 4:3 and it can sometimes produce a very mild stutter - resolved by a cumbersome procedure I have memorised but is a bit of a pain.
In any case, I checked the Topaz output a few times very carefully on the far more powerful computer and large monitor that I am running Topaz on and I could not see any issues with the interpolation no matter how hard I looked - but only if I either unchecked that duplicate checkbox or - if checked - the value was kept extremely low - either 1 or 2 as opposed to the default of 10. All three options seem to produce the same result anyway.
But I suspect that sensitivity slider might compare frame to frame on the input and the higher the sensitivity, the more it will allow one frame to differ from the next before it declares it a duplicate. So for example, a sensitivity of 2 might work perfectly in deleting pulldown frames from 24 fps native content but if you keep increasing it, it will start to allow small differences between two frames to count as a duplicate. Even by the default of 10 it seems to start doing this. That is my guess anyway based on my experiments.
