VEAI Command-Line Alpha 3.0.0.1a

Topaz Video AI Video AI Beta
1

Downloads:
Windows | Headers, Libs, & DLLs (for custom builds)
Mac | Headers & Dylibs (for custom builds)
Models
Source Code

Hello everyone,

The next release of the VEAI 3 alpha is now available! Please note that these initial releases only utilize a subset of FFmpeg features - some features available in general FFmpeg builds may not be available in this build.

Changelog

  • Mac support
  • (Mac) Added OpenH264 encoders
  • (Mac) Added libvpx encoders
  • (Windows) Run scripts now default to H.265 hardware encoders. This should resolve some quality issues with HD videos.
  • Model versions bumped

All Platforms

  1. Please download the additional set of models at the top of this post, and copy over the models included in the archive.
  2. Follow the remaining instructions for your platform.

Windows Installation

  1. Extract the downloaded zip file to a location of your choice
  2. Run the “run-me" file appropriate for your graphics card.
  3. You will be prompted to confirm the auto-detected models location. In most situations, you should be able to just enter “Y”
  4. Follow the on-screen prompts to login to TopazLabs.com and enter your VEAI options
  5. VEAI will run on your files
  6. Once complete you can either exit the window, or type additional commands (see Advanced Usage)

Mac Installation

  1. Extract the downloaded txz file to a location of your choice
  2. Using terminal, run ./installer.command file to set appropriate environment variables.
  3. Run veai-ffmpeg/bin/login email password
  4. Run veai-ffmpeg/run-me.sh input.mp4 to upscale using Video Toolbox.

Advanced Usage

  • If you have an existing models folder, you may select it by setting the VEAI_MODEL_DIR setting in your system environment variables. If you do not already have downloaded models in this folder, you will need to copy the model JSON files from the download into this folder. It will then be detected by run-me.bat
    • Mac users: this variable is automatically set in .bashrc. If you are using ZSH you may want to move this variable to .zshrc.
  • (Windows) After run-me.bat finishes, you can execute additional VEAI sessions by running FFmpeg. See the end of run-me.bat for an example.
    • As an example: ffmpeg -hide_banner -i "input.mp4" -vf veai_up="scale=2:model=amq-12" "input-2x.mp4" would upscale input by 2x using AMQ 12, saving the output to input-2x.mp4
  • (Mac) The ffmpeg command can be invoked similarly to perform additional VEAI sessions. Our setup scripts currently assume bash will be the terminal used. ZSH users should set the appropriate variables in .zshrc.

Available Models

  • aaa
  • ahq
  • alq
  • alws
  • amq
  • amqs
  • ddv
  • dtd
  • dtds
  • dtv
  • dtvs
  • gcg
  • ghq
  • prob
  • thd
  • thf

Licenses

  • FFmpeg is licensed under the GNU Lesser General Public License version 2.1 or later (LGPL v2.1+). A copy of this license can be found here .
  • OpenH264 is licensed under the BSD license. A copy can be found here
  • libvpx is licensed under the BSD license. A copy can be found here
3 Likes
4

Hi,

Mac M1 Max

I had zero joy using my 3 minute 50 second mp4 test video and the dtv model (the colorspace yuv420p to bgr481e error) so I changed the model to ghq-5 and it again shows the colorspace error but it does continue. But a a snail pace of 4.7 fps!!

So you know :wink:

5

While the 2.6.3.3.b thread is closed already and I am not too familar with a command line interface anymore (last I used one was back in the 1980’s I am afraid) I just wanted to mention that the 2.6.3.3.b still suffers from some sort of a memory leak still if not the Nvidia driver itself. After running two instances of VE AI well I started a game in addition. When I ended the game, both instances of VE AI ran very slow and never speeded up again until I reboot the whole PC and opened two fresh instances of VE AI to continue the work.

6

We are aware of the memory leak situation on windows with v2.6.3.

We are moving to a different architecture and code base for 3.0, the first GUI alphas/betas should be out in a few weeks. So far we have not seen any memory issues with the new version.

2 Likes
7

Hi, can you provide something like a “/build/ffmpeg_extra.sh”

So we can include all the libx264, fdk_aac, nlmeans_cuda and nonfree stuff in a single ffmpeg build ?

8

Is there a tutorial for a noob with command lines like me on how to get started with this? Do I need to set the output file to a folder or something, how does that work?

9

My command is like this

/Users/ibobby/Downloads/veai-ffmpeg-mac-3.0.0.1a/veai-ffmpeg/bin/ffmpeg  -i born.AVI -vf veai_up="scale=2:model=amq-12" born.mov

and the error it threw was

[Parsed_veai_up_0 @ 0x7fecc6106a00] Failed to configure output pad on Parsed_veai_up_0
Error reinitializing filters!
Failed to inject frame into filter network: Invalid argument

What does that mean and how to fix that?
Thank you.

10

First try at making a long conversion (50 min video ~ 12 Hours)

Command
ffmpeg -hide_banner -i "input.mpg" -vf "bwdif=mode=1:parity=0:deint=0, veai_up=scale=2:model=ahq-12" -debug 1 "output\%06d.tiff"

Finished processing frame 20699 ahq-12 2 413.900000 413.900000
[image2 @ 0000029DFF08F500] Opening 'E:\ZTopazOutput\BarrioSesamo19ahq\020695.tiff' for writing
[file @ 0000029DF71850C0] Setting default whitelist 'file,crypto,data'
[AVIOContext @ 0000029D9178D9C0] Statistics: 10037879 bytes written, 0 seeks, 39 writeouts
About to filter frame 20700 ahq-12 2 413.920000 0.000000 0
Image Model inference failed:  OpenCV(4.5.3) C:\Users\Adam\.conan\data\opencv\4.5.3\_\_\build\eae637997668fbc59b3bd2e17a93770838eb99b2\source_subfolder\modules\core\src\alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 13271040 bytes in function 'cv::OutOfMemoryError'

Error Handle Output:  OpenCV(4.5.3) C:\Users\Adam\.conan\data\opencv\4.5.3\_\_\build\eae637997668fbc59b3bd2e17a93770838eb99b2\source_subfolder\modules\core\src\alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 13271040 bytes in function 'cv::OutOfMemoryError'

Status: 2 message: "OpenCV(4.5.3) C:\\Users\\Adam\\.conan\\data\\opencv\\4.5.3\\_\\_\\build\\eae637997668fbc59b3bd2e17a93770838eb99b2\\source_subfolder\\modules\\core\\src\\alloc.cpp:73: error: (-4:Insufficient memory) Failed to allocate 13271040 bytes in function 'cv::OutOfMemoryError'\n"

Happens with keepColor 0 and keepColor 1.

12

working on NVIDIA 2070 Super, first.2 attempts with amp seems to indicate stutter (duplicate frames) issue is fixed. Will test more. Very interesting!

13

Trying to document what I understand of the ffmpeg command VEAI is launching, your feedback/corrections are welcome of course!

ffmpeg
-hide_banner Suppress printing banner
-strict 2 allow non standardized experimental decoders and encoders
-hwaccel auto hardware acceleration
-i “input.mkv” input file
-vf veai_up=“threads=2:scale=2:model=amq-13” VEAI parameters
-c:v hevc_nvenc envoder = nvidia, hevc_nvenc: yuv420p nv12 p010le yuv444p p016le yuv444p16le bgr0 rgb0 cuda d3d11
-rc vbr variable bitr rate
-cq 24 Set target quality level (0 to 51, 0 means automatic) for constant quality mode in VBR rate control (from 0 to 51) (default 0)
-qmin 24 Set min video quantizer scale (VBR). Must be included between -1 and 69, default value is 2.
-qmax 24 Set max video quantizer scale (VBR). Must be included between -1 and 1024, default value is 31.
-profile:v main10 Set the encoding profile
-pix_fmt p010le 10 bit pixel format (other options available for hevc_nvenc?)
-b:v 0K video bitrate
“output.mp4” output file

1 Like
14

Thank you for posting. We need some further information:
Is this on windows 11 or windows 10?
How much memory do you have on the system?

15

That’s correct! If you want to see more of the options available for the encoders, such as what other pixel formats are valid, ffmpeg has some built in help you can access: ffmpeg -h encoder=hevc_nvenc

16

image

17

Hi!

Is there any additional output before that error message?

18

Thank you, we are noticing a memory leak on long videos. Will resolve it soon.

2 Likes
19

There is no more gamma/color differences compared with V2.6 but there are still diffrences with the outputs whatever model I have tried at least.

GHQ have more chroma noise in dark areas in v3.0a and on contrary with AMQ-12 seems more denoised with sharper edges.
But overall v2.6 outputs are still better quality for the same model.

20

Just spotted the Mac version was here - I’ve been a little slow off the mark. It seems to work nicely (once stopped doing stupid things on installation, and then figured out how to add the Proteus params)

Not done much with it as yet - just a couple of tests of Proteus. I did notice though that it’s only using a small part of the GPU capacity (I’m on a 6800 - it was using around 16% GPU)

Results seem very similar to the 2.6.3 beta.

21

Changed the 2-threads in the run-me-nvidia.bat to 4 and increased the upscaling speed to around 6.8fps (RTX 3070 on DVD deinterlaced video, model ‘amq-13’).
It downsampled the audio from 256kb/s to 192kb/s, could be worth to add an option to skip encoding audio (-acodec copy).

One more problem is the scaling multiply. Going from 720x576 video to 1920x1080 is not possible with this method, as x2 will yield a >1080 pixels height result.
Tried manually adjusting the parameter but it seems the veai plugin expects only multiply.
Could be worth adding a way to upscale to specific resolution with black bars like in the normal VEAI.

Result looks awesome nontheless, so thank you for the hard work!
Much appreciated :smiley: