Hello!
I’m using Topaz AI for the first time. I have a video I recorded (me talking in front of a whiteboard) and the recording has a lot of dropped frames / stuttering. I’m trying various settings for frame interpolations to clean it up, but I’m not having any luck. The original video is 28 FPS, if I set it to output at 60 FPS it’s incredibly smooth when the original was smooth, but the stuttering is kept (just now at 60 FPS). Any tips??
Thank you!
Dan
Is the stuttering from duplicate frames?
Yes! I’m wondering if there a way to detect duplicate frames and engage the interpolation?
Happy to share a short clip if that would be helpful.
I had success dropping the duplicate frames by running Chronos Fast and having it lower the frame rate. My movie was formatted for DVD, so it had the DVD standard 29.97 FPS, and I told TVAI to lower it to 23.976 with Chronos.
Interesting, this almost works for me! But the stuttering was pretty bad so I think I’d need to lower even more than 23.976. I guess I could do this with ffmpeg and see what happens. I wish I could detect duplicated frames and only do it for those spots.
I’ve seen ways to do that with ffmpeg, but it didn’t work on the video I tried it with. You might have a variable frame rate on your video.
Hello!
the solution i easy: before importing the video into VE AI convert it to the lowest framerate you find in your video. Use a tool of your choice for the conversion. Maybe reduce the frames per second from 28 to 14 if in some parts you have doubled frames. VE AI will then be able to smoothly make it 60 fps!
Cheers, Imo 
@daniel-9174 Sounds like you may have dropped (missing) frames - seen in many old video recordings when the recorder or disk couldn’t keep up. The ‘missing’ frame is automatically replaced by the recorder by a duplicate of the previous frame in order to maintain the frame rate/correct number of frames a second. Sometimes, the drops and dups are caused by frame rate changes.
So you end up with two identical frames then a little jump to the frame that follows it i.e. a jumpy jerky video whenever the movement is significant. Slow movement doesn’t usually reveal it at first sight, even if it’s there.
If these ‘drops’ and dups are regularly/evenly spaced you can use various freeware to remove all the duplicates then TVAI to fill in the missing frames if you want slomo or a higher frame rate.
The problem arises when the drops and dups and not regularly spaced and this often happened with old video recordings. It is VERY hard and time consuming to deal with that as with all the usual video software, it has to be done manually, frame by frame then the video reassembled.
Video guys far more knowledgeable than me have struggled with this problem for over a decade by writing avs scripts that run in Avisynth or Vapoursynth - both of which are included with Staxrip (or use Virtualdub2). These guys have been active within the last couple of years and have posted what appear to be very useful scripts to deal with this problem - but I haven’t yet had time to learn the method and test them.
But for you or anyone who wants to head down that path, here’s a link to the discussion I followed, spend some time working through it and by the end, you should see the script that hopefully is needed.
Good luck!
Daniel are you able to share the footage. I might have some time this weekend to work more with TVAI and I would happy to look into this with you. I’m new to this too but learning quickly through lots of experimentation. If you were able to share the video I can set up a dropbox link for my business that you could drop it in so I could give it a shot for you.
Chris
Hi Chris! Sure I’d be glad to share some footage. How about just a one minute sample you can play with and we can be in touch if you are able to discover a good solution!
Thank you for all of this info, it’s excellent!
i often suffer from video files that in general run almost smooth. almost. because all few seconds there is a frame jump in the video. I have no idea how to fix this.
Well, I put some amateurs skills to work and came up with nothing. I hope you get it figured out though.
Getting rid of duplicate frames is actually REALLY easy in most cases. It is caused when a video is converted to a higher framerate than the one in which it was filmed. It happens a LOT with DVD issues of classic series; for example, the Rawhide DVDs play at 29.97fps and have a duplicate frame every 5 frames, indicating that the correct framerate should be 23.98fps.
You don’t need any fancy software. Just use freeware Handbrake (I’m sure others would work as well, but Handbrake is a great free tool, IMO), and convert the video using the correct target framerate. I’ve found in every case that the duplicate frames are dropped and the converted file plays as smooth as butter.
This works ONLY when there are consistent and regular duplicate (not dropped) frames, i.e. when you are converting to a lower (not higher) framerate.
To check the target framerate, first play your jittery video in a player than can skip frame by frame, and observe the frequency of duplicate frames. This will tell you what the framerate SHOULD be.
Most commonly, a 30fps jittery video is fixed by converting to 24fps, and a 29.97fps one is fixed by converting to 23.98fps.
After this, if you wish to increase (e.g. double) the framerate in Topaz the results will be great because you’re not processing those duplicate frames. As you have probably discovered, simply increasing the fps without deleting the duplicate frames first will not fix the jitter.
Same experience here. Just convert it back the classic way. On the other hand the option to drop doubled frames within Video AI never worked great for me.
You’re spot-on that the problem can be very gnarly. As long as the dupes or skips are evenly spaced, it’s easy to fix as others have mentioned. But when they’re not, then it’s trickier.
I often run across clips that have been through multiple invalid encodes. E.g. from VHS/super8 as you indicated, where the capture wasn’t aligned to the original frame rate. Then through some standardization encode where some ingest pipeline converted all clips to a “standard” frame rate, thus adding even more “randomness” to the skips and dupes, creating a monster of a result.
Those hard cases come in two forms:
For problem A, I’ve been using this to pre-process the clip, which creates smooth motion. Then I run it through TVAI as normal. This approach requires one to eye-ball the distance between the dupes and skips, and then fill in those distances in the script (refer to the docs in the link for step-by-step instruction). Results are the best I’ve found from any solution to this particular problem, so have been successfully using it for years.
For problem B, I haven’t found a good heuristic since the challenge is to produce a clip with the same number of frames as the source clip, and must have the same or very similar perceived amount of motion between each frame and its surrounding ones. Not only that, any frame correction must be performed in a relatively short cycle of frames, else the video becomes perceptively out of sync with the audio on a constant frame rate clip (which most clips are), yielding an even worse viewing experience.
Solution to problem A can solve the issue because there’s a well defined repeating cycle in which corrections take place, but B is unbounded requiring a completely different approach.
I’ve been pondering that problem to and from for about 12 years now, but have yet to find an optimal solution. Right now I go down the very laborious route you indicate, to come up with a bespoke fix for a given clip instead of trying to solve the problem generically.
I often stumble over wrong encoded movie files. They are all 24 fps where every 24 th frame there is a frame jump. I have no idea how to fix this. 
That’d fall into category A.
If there is a skip once per 24 frames, then the source clip was 25 fps. So the corrective measure is to interpolate that missing frame.
If the skip is always at the same location in each 24 frame cycle, then you can perfectly solve the problem by interpolating a new frame between the skip frame and the preceding frame in each 24 frame cycle [*]. If the skip location varies in the 24-frame cycle, Try the filter I linked as it’s designed for that exact use case.
The problem with the linked filter is that it operates on motion differences within each cycle, so as long as the motion is sort of uniform within a cycle, it will accurately detect and fix the skip. But if the cycle contains other sharp motion (e.g. someone flicking a sword across the screen, for 1-2 frames only) then it may incorrectly interpolate the frame in that other motion if the frame difference for that other motion is larger than the skip itself. Overall though, I find that it does a good job most of the time, so tend to just use it, then scrub the video to see of the overall motion is pleasing in the fixed clip. If it is, I go with it since the “average person” is even less discerning than I am :slight_smile
[*] Here’s an example avisynth script for fixing the first variant (fixed position of the skip in each cycle)
# Produces a clip where even frames are original and odd are new interpolated ones "in between" the original frames.
function Interpolate2xFramerate(last) {
super = MSuper(pel=2, hpad=0, vpad=0, rfilter=4)
bv1 = MAnalyse(super, overlap=8, chroma=true, isb=true, blksize=32, searchparam=3, plevel=0, search=3, badrange=(-24))
fv1 = MAnalyse(super, overlap=8, chroma=true, isb=false, blksize=32, searchparam=3, plevel=0, search=3, badrange=(-24))
bv2 = MRecalculate(super, overlap=4, chroma=true, bv1, blksize=16, searchparam=3, search=3)
fv2 = MRecalculate(super, overlap=4, chroma=true, fv1, blksize=16, searchparam=3, search=3)
bv3 = MRecalculate(super, overlap=4, chroma=true, bv2, blksize=8, searchparam=1, search=3)
fv3 = MRecalculate(super, overlap=4, chroma=true, fv2, blksize=8, searchparam=1, search=3)
bv4 = MRecalculate(super, chroma=true, bv3, blksize=4, searchparam=0, search=3)
fv4 = MRecalculate(super, chroma=true, fv3, blksize=4, searchparam=0, search=3)
MBlockFps(super, bv4, fv4, num=2*FrameRateNumerator(), den=FrameRateDenominator(), mode=0, blend=false)
}
# The source clip is 24 fps with one skip 8 frames into the clip that repeats cyclically every 24 frames.
# Note that indexing is zero based, so the frame index in each cycle is one less; 7 (first frame has index 0 not 1)
FFVideoSource("sample.mp4")
Interpolate2xFramerate() # remember: the produced clip has 2x original framerate
# Select 25 frames; 24 of which are original and 1 interpolated.
# Pick the interpolated frame **after** the observed skippy frame, to transform the skip into smooth motion.
#
# <2x original fps> <first set of original frames> <remaining set of original frames in the cycle>
# | | <the interpolated frame> |
# | | | |
SelectEvery(24*2, 0,2,4,6,8,10,12, 13 ,14,16,18,20,22,24,26,28,30,32,34,36,38,40,42,44,46)
The logic for how this works:
A cycle in the source clip has motion (o = frame with smooth motion, S = skip noticed):
o o o o o o o S o o o o o o o o o o o o o o o o
This is because there’s a frame missing at index position S-1, so we want the final clip cycles to look as follows:
o o o o o o o i S o o o o o o o o o o o o o o o o
note: i denotes a new interpolated frame between two frames (S-1 and S in this case)
We accomplish this by interpolating the entire clip to produce:
o i o i o i o i o i o i o i S i o i o i o i o i o i o i o i o i o i o i o i o i o i o i o i o i
Then we simply drop all odd (interpolated) frames, except for the interpolated frame just before the skip, and that smoothes out the skip and leaves us with a new clip that has one extra frame per cycle. As our original clip had 24 fps, that gives us the desired target frame rate of original_fps * (cycle_length + 1) / cycle_length = new_fps (from 24 → 25 fps in this case).
You can then just mux the audio onto the produced video stream and video and audio will stay in sync.
