Current AI and models give acceptable results in most cases (in others the are horrible), but it is exaggeratedly slow. I have a GTX 1080 Ti and interpolating a 5 minute 30fps video usually takes a long time.
I use other video interpolation programs, which use other AIs and models, for example I use one that uses RIFE, in which I use NCNN’s RIFE4.0 model (a RIFE implementation for Vulkan) and in less than 5 minutes it does the interpolation.
I’m not sure if VEAI and the other Topaz programs run on Pytorch, if not it should. Pytorch is very effective and fast in video post processing, so if it were implemented in VEAI the processing times would be much faster and more effective.
What do you think?
PS: Oh, and the audio of the video should not be lost. I don’t understand why when interpolating a video the audio disappears, it’s quite annoying and that’s why I have to use other programs to interpolate videos with sound.
Sounds like you’re talking about the Chronos model. The main goal of that model was to take normal footage and make it slow motion. Keeping the sound does not make sense in that use case.
Personally, I would only ever use it to increase frame rate, because I value that over resolutions higher than 1920x1080. I have the Smooth Video Project (SVP) using Nvidia Optical Flow to compare against. SPV is made to do real time interpolation, but it looks really bad in some scenes. Those same scenes come out perfect in VEAI. The cost of accuracy seems to be processing time. VEAI does randomly blur patches of trees where SVP doesn’t, but that is something that a newer model of Chronos could fix.
In general, 80% of what SVP interpolates, yields very noticeable artifacts. VEAI yields about 5% on the same movie.
Maybe I need to try RIFE, but either way the bottom line is: If it works better, use it instead.
I agree that it is a shame that in order to completely enhance a video, more programs are needed. Right now, I need to have ffmpeg to correct DVD framerates and chop up the source into clips. Then I can use VEAI on it. Then I need ffmpeg to make the video a reasonable size again. Finally, I need MKVToolNixGUI to combine all the sound, subtitles, chapters and video clips back into a final complete video.
I thought ChronosFast was to increase the framerate (because of the name ChronosFast, with “Fast” I thought it meant to make the FPS faster, to interpolate xd) however, when I do slowmotion I use the default one, “SloMo”. And about the sound, I think that the option should be given to preserve the sound, making it in slow motion as well so that it doesn’t sound distorted and bad, something like serious. I don’t know how to explain it, like when you edit an audio with Audacity, you put it in slow motion and it sounds deep, something like that.
SVP never gave me good results. I recommend you to use RIFE (you can try it in Flowframes wich it has other AIs like DAIN or FLAVR), and if you want to interpolate 2D animations I recommend you TVP (it’s private tho, you’ll need to join a discord server to test it on a bot).
SVP was originally designed to process videos in real-time. The ability to convert videos was introduced later. I agree that SVP is a quick but not the best choice for high quality frame interpolation. It often generates strong halos and has servere issues with fast motion.
I missed that important little bit. After installing the simplest looking version off of github, it proceeded to tell me it was going to take 8 hours to do a 7 minute video.
