Slowmotion /upscale/ duplicate frames issue

Hi, just something I noticed in interpolation of extra frames, some videos have duplicate frames which is bad. I just tested an old youtube rip which had that issue. ffmeg has an option to remove it

-vf mpdecimate,setpts=N/FRAME_RATE/TB

It looks like video enhance processes each frame as a unique frame which is useless processor time for both upscaling and slowmotion. In the case of upscaling I even notice a difference in both upscaled frames while they are both similar.

Just only something to mention for the developers.

1 Like

I never had a duplicated frame.

Yeah, I found the same issue, I’m hoping it was there beforehand, I’m doing a comparison between flow frames and Chronos, I’ll post back here soon with the results.

Yep, I can definitely confirm that it does indeed add duplicate frames to the interpolation calculations…

I encoded a file at 25 frames to 75 with Chronos, and 100 in Flowframes (The closest I could get to 75) and while scrubbing the file frame-by-frame there was indeed dups with Chronos, yet absolutely none with flowframes.

I hope that this will get fixed and it’s not a feature, as it’s in the same vein as encoding a 30fps clip to 60fps by just duping and doubling the frames for the desired framerate…

This is too old-school of a solution in this age of AI, but otherwise, Chronos is still a great model as you can adjust to any framerate that you want and are not bound by only 2x/4x/8/x like with flowframes also there are no wavy artifacts like with optical flow!

We got a good thing going here, so please get rid of the frame duping.

1 Like

Yup, I noticed this too. I was surprised to see it, since this is certainly not something I would expect of an A.I. solution. Doubling frames is your average, old fashioned way of doing things.

1 Like

I will say this for Chronos, it has much fewer shimmering artifacts as flowframes does, though unlike flowframes Chronos cannot encode frames with audio!

In spite of that, I still prefer Chronos to flowframes!

The problem is not content related. No matter what frames you process you get dups.

And fps increase multiplier is not the issue:
25 -> 60 = 65 frames (bad)
25 -> 50 = 50 frames
25 -> 100 = 100 frames
25 -> 120 = 120 frames
25 -> 59 = 59 frames
25 -> 61 = 61 frames

And specifying 60 fps by hand does not resolve the issue.
I still get 65 frames.
Processing 30 frames (from 25fps source) to 60 fps results in 78 frames.
And as you can see Chronos model is not separated by any multipliers (one model for all).

Changed source fps to 30:
30 -> 60 fps = 60 frames!
30 -> 100 fps = 110 frames (problem here)
30 -> 99 = 99 frames
30 -> 50 = 50

I don’t see logic here.
Only 25 -> 60 and 30 -> 100 are broken.

50fps source 50 -> 60 results in 61 frames.
Same result as if you remove dups from where dups problem exists (1 extra non dup frame will be there).
50 -> 120 = 129 frames
50 -> 100 = 100 frames

Looks like problem is not in the model but in code that works with this model…

25fps -> 25 fps
240% slowdown (same as 25->60) results in same 65 frames from 25.
239% slowdown = no dups (60 frames)
30fps -> 72fps (same as 25 -> 60) results in 78 frames…
Somehow 2.4 multiplier is broken everywhere.

I have an an SD TV series I ripped myself from DVD’s I bought. H.264 (I used same as source) and the FPS is 29.97. Now, if I drag one of those files to VEAI, at the bottom VEAI says it’s 60 FPS. It’s like I have no choice but to add frames. I don’t want to increase FPS - I just want to upscale 100% and I’m not using Chronos…The episode looked great when done, but it was now at 60 FPS…Am I doing something wrong?

Try to remux source to mkv before using. And be sure you don’t use deinterlace model.

Ahead of you. Remuxed using MakeMKV from the DVD…mediainfo says that file’s FPS is 29.97…loaded that file into VEAI and at the bottom it shows 59.94 FPS (same as it did with the already encoded H.264 file) I just watch to simply upscale with without VEAI wanting to double the FPS for some reason!

Just specify 30000/1001 fps when remuxing in mkv

That actually worked! Thanks

Small update:
Processing 25->50 fps.
When stopping processing (to jpeg) the last frame saved is always duped.
Like if 50 jpeg saved. 50th jpeg is same as 49th. Else are not duped.

Afaik the last frame is purposely duped/repeated so that the output is exactly as long as the input (duration-wise).

Some update:
I’v upgraded from 970 GTX to 3060 RTX and issue is exaclty same. So its not hardware/drivers related.
I’v tried to increase FPS of 25->60 and saved result in jpegs.
Then i’v removed duped frames (they are binary exact) and frames count is what must be for 60 fps.
But in movie (ofcourse if looking slower) motion is stuttering. Like where there were no dups - the motion is fine, but where dups were and removed motion is jumping to next frame.
Where A B C are original frames.
I are interpolated frames.
D are dups.
So from A to B everything is fine. But from B to C is jump (because were dups).
I mostly think this problem in VEAI is because of its internal frameserver.

  • About frameserver. I HIGHLY recommend to use some external frameserver like AVISynth. Because internal in most cases reports incorrect frames count and fps and…

Next time remove the dupes before you use Chronos/increase the fps as Chronos will compensate for the missing frames with AI interpolation…

And how it will understand how many dupes was removed from original?
And how to remove them?
Using nvenc feature? But then frameserver will interpolate them back before VEAI will get them.

If you have AE you can use a plugin that deletes those duped frames, or you can use FLowFrames to do both the removal of your duped frames and converting to 60fps, or you can go the manual method and convert to frames and run those frames through DupeGuru…

A lot of times videos will get recorded at say 25fps, and then somebody has used basic software to convert it to 30fps. But most of the time, this is done by duplicating every 5th frame to produce a 6th frame, so it’s not true 30fps. It’s 25 real frames and 5 duplicated frames playing every second.

It would be nice if VEAI included the ability to detect and remove those duplicate frames.

I’v got flowframes and it removes duplicate frames fine. But use “not accurate mode” in it. Its better than accurate.
If you want to use VEAI fps increase instead of flowframes you can do the following:

  1. Enable step by step mode in flowframes.
  2. Extract frames (dups will be removed).
  3. Interpolate frames using flowframes to x2.
  4. Load interpolated frames using avisynth. In script write “SelectEven()” to select only half of frames to get the same fps as original.

Then use VEAI.

Its still not related to original problem in this thread (VEAI makes dups in output in some cases, increasing movie length).

I asked a similar question on the thread “Jerky Motion on One Video”. There, user ssbroly described hitting the same problem with pans and said they’d found that uncompressing the video (making use of Vegas Video) before feeding it into VEAI solved the problem. I tried doing more or less the same thing (taking the original clip and then using Apple Compressor to render it as 8-bit uncompressed, and finally feeding the uncompressed clip to VEAI) and found it did indeed eliminate the duplicate frames issue for me.

Try it if you haven’t already.