Video Enhance AI v1.4.2 - Command Line Interface and Improved Models

Here’s some interesting info regarding Gaia-HQ in the latest 1.4.2 version of Video Enhance AI that might be useful to both users and developers.

I have found that I get the best results when only scaling the source by either 2x or 4x. From there, I do the final stretch (if required for anamorphic DVD) with AviSynth via Hybrid. I use Spline144 to do the final resizing. I have found I get an improvement when running the “Limited Sharpen Faster Mod” filter at the default settings for “slow” with the preblur on. As mentioned, Video Enhance AI seems to be slightly lowering the gamma from the source DVD (darkening it), so I increase the gamma by .02.

Here’s a sample frame gallery:

The first image is from the source DVD. It has had small borders cropped and resized to 1920x1080 using Lanczos in Irfanview.

The second image is from direct output from Video Enhance AI, outputting to PNG, Gaia-HQ, 2x scale, cropped and resized to the final dimensions of 1920x1080 using Lanczos in Irfanview.

The third image is from the Gaia-HQ output processed through the below AviSynth script. A residual bottom black border is cropped, the gamma is increased by .02, resizing to final dimensions is done via Spline144, and the Limited Sharpen Faster Mod filter is applied as described above:

ClearAutoloadDirs()
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\LoadDll.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\ImageSeq.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\SplineResize.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\masktools2.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\VariableBlur.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\warpsharp.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\RgTools.dll")
LoadDLL("C:\Program Files\Hybrid\32bit\avisynthPlugins\libfftw3f-3.dll")
Import("C:\Program Files\Hybrid\32bit\avisynthPlugins\Zs_RF_Shared.avsi")
Import("C:\Program Files\Hybrid\32bit\avisynthPlugins\LimitedSharpenFasterMod.avs")
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
# loading source: E:/USER/fin/wip/GaiaHQ/%06d.png
#  input color sampling RGB32@8, matrix:Rec.709, scantyp: progressive
#  input luminance scale tv
ImageReader(file="E:\USER\fin\wip\GaiaHQ\%06d.png", start=0, end=147528, fps=23.976, use_DeviL=true, pixel_type="RGB24")
# current resolution: 1412x936
# cropping
Crop(0,0,0,-8)# 1412x928
# color modifications
Levels(0,1.02,255,0,255)
# scaling to 1920x1080
Spline144Resize(1920,1080)
# sharpening
ConvertToYV12(interlaced=false, matrix="Rec709")
LSFmod(Smode=5,Smethod=3,Lmode=4,preblur="ON",soft=-2,edgemaskHQ=true)
PreFetch(4)
return last

Finally, the fourth image is the same as the above, but the Rec.2020 color space that Video Enhance AI outputs has been converted back to Rec.709 which matches the source DVD. I feel the Rec.2020 output by Video Enhance AI is an improvement in this case:

ClearAutoloadDirs()
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\LoadDll.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\ImageSeq.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\ColorMatrix.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\SplineResize.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\masktools2.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\VariableBlur.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\warpsharp.dll")
LoadPlugin("C:\PROGRA~1\Hybrid\32bit\AVISYN~1\RgTools.dll")
LoadDLL("C:\Program Files\Hybrid\32bit\avisynthPlugins\libfftw3f-3.dll")
Import("C:\Program Files\Hybrid\32bit\avisynthPlugins\Zs_RF_Shared.avsi")
Import("C:\Program Files\Hybrid\32bit\avisynthPlugins\LimitedSharpenFasterMod.avs")
SetFilterMTMode("DEFAULT_MT_MODE", MT_MULTI_INSTANCE)
# loading source: E:/USER/fin/wip/GaiaHQ/%06d.png
#  input color sampling RGB32@8, matrix:Rec.2020, scantyp: progressive
#  input luminance scale tv
ImageReader(file="E:\USER\fin\wip\GaiaHQ\%06d.png", start=0, end=147528, fps=23.976, use_DeviL=true, pixel_type="RGB24")
# current resolution: 1412x936
# cropping
Crop(0,0,0,-8)# 1412x928
ConvertToYUY2(interlaced=false, matrix="Rec709")
# color modifications
ColorMatrix(mode="Rec.2020->Rec.709",interlaced=false,hints=false)
Levels(0,1.02,255,0,255)
# scaling to 1920x1080
Spline144Resize(1920,1080)
# sharpening
ConvertToYV12(interlaced=false)
LSFmod(Smode=5,Smethod=3,Lmode=4,preblur="ON",soft=-2,edgemaskHQ=true)
PreFetch(4)
return last

To best see the differences between these images, it’s best to load each of them in 4 separate instances of Irfanview, then switch between the open windows. Here’s the files uploaded to the above Imgur gallery in a zip:

I’m not 100% sure, but I believe VEAI supports AMD video cards only in Mac OS using CoreML in Catalina and above. And then, only for Artemis and Theia models (not Gaia). I also use an eGPU in Bootcamp Windows, but my card is Nvidia, and it works fine.

I wish it would support Gaia HQ

Still looking forward to some Nvidia Tensor-core action. THAT would be a game changer.

I bought a laptop with an Nvidia 2070 card SPECIFICALLY for the AI cores.

TY

3 Likes

If you fellas can’t do H.265, VP9 is always free! Then again, the program already includes ffmpeg, and that program already has H.265 licensed!

1 Like

I agree with my fellow customers. H.265 and utilizing Tensor Core would make this program in a whole different league. I have Dual 2080 TI cards in my PC and Tensor Core is one of the main reasons why I bought them.

1 Like

Are you sure it’s Rec.2020 ? If you use the Rec.601 matrix instead of Rec709 when converting to YUV you normally directly should match your source. Also are you sure your DVD has been mastered with the Rec.709 colorspace ? because DVD is SD and most SD sources use the Rec.601 matrix.

I decimate the DVD to h.264/mp4 first using constant rate factor of “1” (the best setting possible). I believe the Rec.601 gets “copied” to Rec.709 settings as it matches the source after converting. But Video Enhance AI changes colors. If I take Video Enhance AI’s result and run the AviSynth ColorMatrix filter on it, which actually does do a conversion, the results of that then match the source DVD almost exactly. That leads me to believe that Video Enhance AI is converting input color matrices to Rec.2020.

It appears that saving the default is not saving the output type. I always use mp4, but it defaults back to jpg.

h.264 should allow crf of zero, for uncompressed output.

Also, Gaia HQ seems to take longer to process the same clips, than the previous 1.3.8, after testing, take almost 2x the time per frame to process after the new update. I am on Mac.

Looking closely at the problem where an MP4 default does not hold I found that all of the others will. They are controlled by the registry value outputVideoFormat, where MP4 is 0. When MP4 is selected and saved as default, it will work for any others loaded in that session, and the registry value is changed to 0. When you close and load VEAI again, the registry value changes to 1 (jpg) during the program load process. If the value is 2 or any other it does not change, and the default is respected. So, zero becomes 1 at load.

At the download page why showing only 1.4.1 version?

1 Like

I notice a slow down too on a non GPU Windows machine with OpenVino.

It’s like, 2 or 3 frames will save, then there will be a 10 second pause, then 2 or 3 more frames will process. It’s worse when the application is maximized when it is processing video.

I rolled back to 1.3.8 and this problem is gone.

I know it sounds dumb, but, I think it might be the new “throbber” animation that now plays on the video thumbnail when processing is taking place that is affecting this. To troubleshoot I’d need a way to disable animations when the app is running.

No that’s a bug i experienced and which happen on some machines, this issue has been fixed int he beta 1.50b, that will be available to everybody soon. My rendering time are now back to normal !

I’m still on 1.2.3. I primarily use Gaia-HQ for the long stuff and Artemis-LQ for more CG type stuff. When I tried 1.3.8, processing slowed by about 50% so I went back to 1.2.3. Is there any reason for me to update to 1.4.1 if I’m sticking with those models?

Problema risolto: si trattava di scheda grafica difettosa.