I am studying the ffmpeg options of VAI4 at Command Line Interface | Topaz Video AI
Would it be possible to use VAI4 to interpolate exactly one position between 2 frames? Say I have 2 frames 000.png and 001.png and frame interpolation ranges from 0.0 to 1.0, would it be possible to write a ffmpeg command line to receive only the frame at interpolation 0.1234 for example?
the following command taken from VAI4 would interpolate ALL frames in between, which I do not want:
ffmpeg “-hide_banner” “-framerate” “25” “-start_number” “0” “-i” “some/in/path/in_%04d.png” “-sws_flags” “spline+accurate_rnd+full_chroma_int” “-color_trc” “2” “-colorspace” “0” “-color_primaries” “2” “-filter_complex” “tvai_fi=model=chf-3:slowmo=16:rdt=-0.000001:fps=25:device=0:vram=0.9:instances=0,tvai_up=model=nyx-2:scale=2:w=5452:h=4090:preblur=0.5:noise=0.5:details=0.5:halo=-1:blur=0.5:compression=0.5:estimate=8:device=0:vram=0.9:instances=0,scale=w=5452:h=4090:flags=lanczos:threads=0” “-pix_fmt” “rgb24” “-q:v” “2” “-start_number” “0” “some/out/path/out_%06d.jpg”
THX
Quick answer: No.
You need at least (I think it’s now) 3 input frames to do interpolation at all. It might be as high as 8. It used to be around 30.
Also, it very much depends on the content of the frames. It will do a good job on some things and not so amazing on more complex fast things. Apollo is the best model all around, but try them all for sure.
I’m not sure I understand what exact frame you are looking to get.
Let me try and use symbols for frames and interpolated frames.
[F] = source frame.
[I] = interpolated frame.
[0] = no frame but time passes.
Are you looking for the exact middle frame: [F][I][F]?
Or something more near the first frame like: [F][I][0][0][0][0][0][0][0][F]?
1 Like
Thank you for the valuable hints. You are probably talking about the 3 frames minimum required by minterpolate for motion detection, which makes sense.
I am looking to interpolate any position between 2 frames, something I am doing also using RIFE AI slow motion interpolation (github). I think, that VAI4 is better than RIFE, so I try to re-implement and update the algorithm.
S = source frame
D = destination frame
t = tween [0…1] where 0 results in S and 1 results in D
For instance I would look for an in-between frame at t=0.12345.
The frame exactly in the middle as you mention would be at t=0.5.
I would look like something exactly near the first frame.
Why would I want this at all? I animate complex speed changes and ramps in video sequences and am using AI to interpolate between frames for this purpose.
Do you think this is possible?
In the end, lets say I have a clip with 1000 frames, I would for instance request frame 857.4532, which would be an interpolation between frames 857 and 858 with t=0.4532.
You can choose a slow motion factor of any value you want. I use 2.5x often.
I’m not sure how it works internally, so it might not be generating the frame at the exact time you want.
You will have to do the math to figure out what slow motion factor to set and what frame will end up being the one you want.
For Apollo 8, someone from Topaz has said that it always generates the frames needed for a slow motion factor of 8x then drops frames if you have it set to less than 8x. You can do 16x, but I think it just runs the 8x twice. My guess is that it generates frames at the set intervals.
1 Like
that is enlightening, thank you.
