Feature request for Video Enhance AI (increase fps)

Hello
I have some super 8 footage of family events from when I was a child and I would like to increase the frame rate to 60 frames per second. I have seen several very old restoration films on youtube where the fps range from 16 to 18 fps and they are able to boost it up to 60 fps. Here is an example


Apparently they used this software
https://sites.google.com/view/wenbobao/dain
I was wondering if you could incorporate this feature into Video Enhance AI
Thanks
3 Likes

I can do that for you if you want. Using SVP (Smooth Video Project). :smiley:

Smooth Video Project only double every frame if you go frome 30fps to 60fps, so that doesn’t really make the video 60fps. You would need something like NVidiao AI Slow-Mo to create new frames to stick inbetween and make true 60fps videos.

sigtor that is not true. Smooth Video Project does calculate interpolated frames between the existing ones to smoothen the result and get the target franerate. Just give it a try. :slight_smile:

I just tried it on a panning shot where I got 60fps video, and every two frames are almost identical, I guess the method Smooth Video uses is as acurate as Dain or AI Slow-Mo.
Maybe it works on a static camera with moving objetcs, will give it a try.

Ok, tested it on a few more shots and it does blend frames so that you get a ghost frame as a filler, but it’s not recreating new acurate frames. And that is on static camera shots. I would say go with Dain or Nvidia methods if you want real frames

If you send me your video clip I might find out what was going wrong and let SVP make it smooth. SVP also offers the Nvidia frame interpolation. :slight_smile:

I found this software RIFE which seems to go faster than DAIN when interpolating the frames. Works on AMD graphics as well as NVIDIA.

1 Like

SPV can either convert a video file or just interpolate on the fly in real time during play!

It’s useful to recommend other softwares to solve an immediate problem, but I agree with kiprode that the best place to retime frame rates is within Video Enhance AI itself. The reason being that it already rewrites each frame based on a sequence from the neighboring frames. That’s the reason it does a very good job removing noise and finding details that isn’t visible in each frame. The details and pixels of each frame live within a virtual space before it’s written into the actual frame itself. Therefore the most logical place to change frame rate is within the very software that rewrites the video image data based upon a whole sequence. Sure it’s possible to get a good result with mixed framerates in Adobe Premiere or other video editing tools, but the point of Video Enhance AI is to achieve the best quality results in your video clips BEFORE further editing in other software.

There can be many reasons why you want to do this. You have some archive footage shot at one frame rate, you use another frame rate on your current camera, and you have to mix in a few stock video sequences shot in a different framerate. When you already have the ability to rewrite video frames based on information from its place within a sequence, it should also be possible to make smooth super slow motion based on that information with results matching other softwares like Twixtor. Twixtor - RE:Vision Effects If you already have to enhance some of your videos anyway, you should also be able to set your desired output frame rate directly within Video Enhance AI.

As a sidenote; One of the other reasons I use Video Enhance AI, besides the obvious noise and details, is its ability to remove compression artefacts you usually find in H.264 and other formats. You don’t see the compression initially, but if you need to modify colors or contrast significantly those compression artefacts become very visible. Video Enhance AI solve this with its ProRes 422 formats. This feature of VEAI is very helpful if you need to later use your footage in a tracking software like Lockdown or other to obtain optimal tracking results that isn’t possible with your initial archived footage. This is very well demonstrated in this video. Using the Optional Tracking Filter - YouTube
VEAI restores many otherwise lost details and should therefore also be the best place to retime footage.

Retiming frame rates by setting the desired target frame rate should be a feature in the processing queue of Video Enhance AI.