VEAI Video Stabilization using AI

I would love to request adding Video Stabilization Feature in VEAI using AI. :heart_eyes:

Currently I stabilize footages with Premiere Pro Warp Stabilizer, it can fix minor shake, however it is not very intelligent at all. Especially, the Synthesis Edges is pretty bad, even Deshaker 3.1 filter in VirtualDub can do much better job than Premiere Pro.

I am sure VEAI with AI, it can do much better job.

  • Video Stabilization (stabilize footage with AI)
  • Sharper footage, by remove blur caused by shaking using AI
  • Reconstruct Edges from frame before & after (reduce crop area)
  • Dynamic crop (auto adjust crop area base on shakiness )
9 Likes

Some older films has a small jitter. Would be nice to get this fixed with VEAI

1 Like

This is my default go-to stabilizing solution, I have tried so many others yet always come back to this one as too many programs dumb-down the options and mechanics of this highly customizable plugin…

I seriously doubt that VEAI would dedicate more than half its resources to accommodate such a robust plugin, as too many mainstream users want a simple destabilize button, rather than doing the work necessary to achieve their goal.

Also, all of these programs have a one-size-fits-all solution, but fail to consider that there is a high level of variable stabilization issues that require specific settings and not a universal one which the majority of programs try and force upon the user!

I would love to be proven wrong and see a viable stabilization option for VEAI, but I seriously doubt that it would be as robust/accurate a solution that is Deshaker 3.1!

EDIT: Also, an auto-crop feature assumes that it is self-shot, where the user has control over the framing, but this would cut out a lot of found footage’s subject matter out of the frame, which is why Deshaker is a much better solution than a dumbed-down version would provide!

2 Likes

Best video stabilizer I found is Prodad Mercalli (standalone - there is also a Premiere plugin) - been using it for years. I use V3. Never bought V4/5 - too expensive.

Anyway it is an amazingly fast and powerful video stabilizer, especially on modern high end hardware - it absolutely flies while analyzing on 5950X

The thing is they do a lot more than just basic stabilization - also rolling shutter and CMOS correction to fix the horrible jello wobble in moving footage.

Point is it’s a lot to add to this VEAI program to match it in functionality.

1 Like

The problem with prodad’s stabilizer is that you cannot isolate/fine tune the moving subject apart from the camera, so you end up with the stabilization effect following the subjects motion instead of stabilizing the camera; but with Deshaker, you can achieve this by isolating the subject by cropping it out and/or masking it with color…

Also, prodad ruined their zoom stabilization as well, plus Deshaker is free!

Prodad does do a good job for self shot videos assuming that the footage is shot with prodad’s tool in mind, found footage is much more versatile with deshaker!

The problem I have with VirtualDub is it’s by now legacy/obsolete software - it will not open any video format file I want to actually use, old codecs and new (eg M2TS, Quicktime, AVC formats) without having to trawl the internet and ancient forum posts from 2011 and sending to dodgy codec sites containing trojan executable “plugin packs” just to try and open the video file. I mean the latest version of VirtualDub is 1.10.4 from 2012. So its more trouble than its worth. So I gave up using VirtualDub years ago.

VirtualDub2… VirtualDub2 download | SourceForge.net

3 Likes

Excellent - didn’t know about this “branch” of VDub… will try it out and see if that stabiliser works on my camera footage.

1 Like

I just use an Fmpeg plugin via sourceforge that allows for mp4, avi, and even mkv files so I have never had to deal with trojan-laden sites as I use web-protection plugins and antivirus progs…

Also, here is a guide for both the 32, and 64 bit deshaker plugin as an FYI for new users!

1 Like

Downloading as we speak! Thanks for the updated Vdub info!

1 Like

+1 for DeShaker, but if VEAI did have a stabilization feature, it seems that one advantage to a VEAI solution is that the video could be upscaled as a final step rather than as a first step, so that the stabilizer wouldn’t potentially be zooming already upscaled footage, thereby achieving an effective final resolution that is lower than other parts of the video. Instead any stabilized frame could end up at the final resolution without more than one resizing (or rather upscaling) step in the process. If I run my footage through DeShaker with autozooming, some of it may be effectively resized. Then if I upscale it it is “resized” again. Or if I upscale first and then stabilize it with DeShaker it is also “resized” twice. Not sure how big of a deal that is or if the quality would be in any way distinguishable from a single resize/upscale.

4 Likes

I stabilize and remove artifacts on the very first pass. No sense on interpolating and upscaling garbage.

Did you intend to reply to my comment in another thread?

I’ve used warp stabilizer as well as tracking points in After Effects and Premiere Pro. I’ve also tried Mercalli which does work better, but still not perfect, and doesn’t allow you to anchor or frame the subject you’re interested in.

I’d like to see VEAI have a mode for reducing camera movement, and have it figure out which objects are stationary vs those that are moving, to use as fixed references to keep the image stable. You should also be able to set the target of the video so it tries to keep that as close as possible to center frame.

1 Like

I use mercalli prodad on all of my cellphone 4k videos, and also because you can stabilize in bulk (I’m doing a stabilization of about 550 video clips at the moment).

But it’s not the best stabilization out there.

The Google photos app has the best stabilization by far, but you can’t process clips in bulk. I’ve tried running it on desktop (through and android emulator), and it only exports in the resolution of my desktop (1080p instead of 4k).

An AI-based stabilization should be an absolute must in VEAI

1 Like

For those replying – in case you haven’t seen it you can vote for this as a feature here: Video Stabilization

I would love this too – I’ve tried mercalli and warp stabilization and found them somewhat lacking. I haven’t tried virtual dub (mac user, so would be a huge pain).

1 Like

I’m surprised no one has mentioned Blender yet. It’s free and the tutorials were pretty not too hard to follow. Sure it’s a complicated program that’s not made for video stabilization first, but that also enables it to have more control over the results.
Of course, an AI solution would be easier.

Thank you everyone for voting the feature. :smiling_face_with_three_hearts:
Developers has listened to us. :tada: :confetti_ball:
They were testing a Stabilization Prototype App in last few months and the feature will include in coming VEAI 3.0. :100:

Amazing news! Have they given any indication on when it might release? I think there’s a version 2.7 in the works prior to that.

The V3.0 Prototype is currently under testing, but it is still very early stage, so I don’t think it will release any time soon.