Video AI 3.0.0.40a

nipun.nath · July 19, 2022, 9:46pm

Hello everyone,

Another alpha is available today. Please let us know the issues you found and your feedback on this. Thanks.

Downloads
Windows
Mac - update pkg | dmg
Headers, Libs, Dylibs, etc.

Changelog

Export extensions, meta data, audio copy should be fixed
Added option to output without moov_flags (used for auto-recovery)
Updated preferences dialog and all options should work
Fixes issues with processes running even after deleting or closing
A new model (Apollo) is available for slowmo and framerate conversion
Hide/show relevant options in customized output
FFmpeg command added in the context menu for preview/exported videos
Fixes bitrate not working for some encoders
Cleaner progress status and other UI appearance improvements
Fixes intel encoder failures
Other misc bug fixes

Known issues

Preview sync, play/pause laggings
Extension change in Preferences doesn’t take effect until re-opening the app on Windows
UI sluggishness
Bitrate is irrelevant for prores
Color is inconsistent
Apollo model is very slow on Mac

Please upload videos that are not working to this DropBox folder.

thedlkr · July 20, 2022, 1:13am

VEAI 3.0 is going to be a game-changer! Excited about the possibilities!

Questions:

If we have previously installed the alpha on a Mac, which should we download — PKG or DMG? I’ve tried searching online but found the answers confusing…
Is there a way to queue multiple jobs to run sequentially rather than in parallel?
Is there a way to apply the same settings to multiple jobs at once instead of individually?

suraj · July 20, 2022, 2:33am

Use the pkg file for upgrade and dmg for fresh install.
Set the max processes in preferences to 1.
Use the … menu to copy settings and then select other videos and paste them. We will add back presets and multi-select in future for faster pasting of settings.

thedlkr · July 20, 2022, 3:16am

Thank you very much.

One more question:
Custom Output > Audio Settings: What is the difference between “Copy” and “Transcode?”

xl5 · July 20, 2022, 4:04am

In Customise Output/Bitrate, I know that you get a precise size for your final video, but what results to others get when leaving it on Zero?

xl5 · July 20, 2022, 4:12am

I just tried using the new Apollo setting to preview. I had it on 100% with a fps output setting of 50 (original is 25) and received the following error? Seems to be related to my previous question regarding Auto Bitrate.

Unable to parse option value “0” as video rate

No accelerated colorspace conversion found from yuv420p to bgr48le.

Invalid value 2 for device, device should be in the following list:

Failed to configure output pad on Parsed_veai_fi_0

Error reinitializing filters!

Failed to inject frame into filter network: Invalid argument

Error while processing the decoded data for stream #0:0

xl5 · July 20, 2022, 4:29am

I find that when I preview videos the original doesn’t play alongside the preview properly, only giving me two frames?

maaru12315 · July 20, 2022, 4:30am

I’m getting similar errors too, in fact using any model gives me an error. Proteus, artemis or even gaia.

maaru12315 · July 20, 2022, 4:40am

Only using Gaia and nothing else on. I get the following error.

No accelerated colorspace conversion found from yuv420p to bgr48le.

Invalid value 2 for device, device should be in the following list:

Failed to configure output pad on Parsed_veai_up_0

Error reinitializing filters!

Failed to inject frame into filter network: Invalid argument

Error while processing the decoded data for stream #0:0

I’m using windows 10. Just did more testing with other models and output options, Getting the same error.

kjell · July 20, 2022, 7:05am

I also get the same error

@nipun.nath
Could you explain how Apollo is different from Chronos?

kjell · July 20, 2022, 8:08am

In the last Alpha I tried it was possible to customize the VEAI command, but I cannot find this option now. If this was removed, I hope it comes back, I thought it was very useful (Especially when trying to work around errors).

I noticed that when I only enable convert fps it is not possible to choose the model one wants to use.

I hope the colorspace conversion error is fixed soon, so that I can actually try to render a video.

Martyprod · July 20, 2022, 8:15am

Noticed that too !

tomwesterman12321 · July 20, 2022, 9:48am

Good update, however, I found some issues/bugs that need to get fixed:

When previewing MPG videos, the left preview window glitches out constantly. The preview rendering is going on, but the original video’s preview is all weird. Especially with MPG files.
Would love to see the “add film grain” option return
Where did the preset manager go?
MPG videos, or 50fps videos of MP4/MKV, are quite sluggish when scrolling through them.
Can we get a new way of using Dione? For example, for my 25fps video, the TV model is the best with its quality, but the video will always be 50fps as the output when using that model. Would it be possible to select the model first, then the output framerate (25fps or 50fps)?
Will there be an option added to pause rendering?
Will the hotkeys come back, such as “I” and “O”?

vista.joel · July 20, 2022, 10:24am

@suraj
I did a lot of experiments with FFMPEG, finally understood the filters, and believe some things could be optimized. For example, having a stereoscopic 360-degree video with a resolution of 4096x3840 I want to upscale to scale 149% (to be within the 35 million pixel limit of the Meta Quest 2). The GUI creates the following relevant parameters:
-vf veai_up=model=alqs-2:scale=0:w=6104:h=5722,scale=w=6104:h=5722:flags=lanczos

Passing w=6104:h=5722 to the AI filter, the filter itself decides based on the desired resolution, at what factor it should upscale (based on the filter description in FFMPEG). With that video, it unfortunately is scale 1, so it doesn’t really upscale, it just improves the quality. I believe it should be scale 2 in this case. That’s probably why the beta version was so much faster.
The FFMPEG-scale-filter is very CPU-intensive. scale_npp or scale_cuda should probably give it a speed-up. But it should be switchable because it might fill up the VRAM with those large resolutions. Or could FFMPEG at least be compiled with these filters also?
Enhancing said video with scale 1 (alqs-2), the GPU can be fully utilized when running 2 concurrent jobs, but when using scale 2, utilization constantly moves between 0-30% and 80%-99% every second

Even running NVDEC and NVENC on a second nvidia-card instead, the GPU is not fully used.

Is there a way to improve that? Maybe it happens because of those large uncompressed unbuffered 8K frames transferred between the eGPU and the notebook with the limited PCIex4 speeds? (~4 GB/s). Could a buffer be implemented maybe? Maybe the filter pipeline could be optimized to use CUVID for example so that the frames don’t get unnecessarily transferred to system memory but stay right in the VRAM for NVdec, veai_up, scale_npp and even NVenc. Theoretically it wouldn’t even have to leave VRAM.

If that’s not possible, would it be possible to process two frames simultaniously in the filter pipeline? Specifying filter_threads=2 or threads=2 in the AI filter doesn’t do that yet in FFMPEG unfortunately.
Specifying an additional thread count in the AI filter parameters (threads=3) doesn’t seem to make any difference in neither the VRAM usage, nor in utilization. Is that still in development? It is listed as a supported parameter in the FFMPEG filter help.
Would it be possible to compile FFMPEG with x264-/x265-support also? I would like to use it on ultrafast preset and high bitrate to be able to convert 2 videos at the same time without maxing out the VRAM. NVENC takes away around 1 GB VRAM for that. I can’t use the second nvidia card for that in the long run because it makes the eGPU have microstutters (a common problem with EGPUs).
Will it be possible to scale up to a non-standard-defined resolution with the AI filter itself without the need to downscale afterwards?

Edit: I corrected some mixed up numbers and typos.

xl5 · July 20, 2022, 10:40am

I’ve been trying to double the framerate of a 1080P video with Apollo but it is taking forever compared to the released version. Could it be anything to do with the AI Processor settings (I have mine on NVIDIA GeForce RTX 3070)? Should I change it to Auto?

I am now trying the very same video with Chronos and it’s moving fine. I think Apollo may have problems with GPUs?

Martyprod · July 20, 2022, 11:39am

almost all your questions have already been answered by the dev in the previous (34a) thread.

some example :

preset will be back at some point : “The goal is to provide good enough presets so users don’t have to configure filters directly as much.”
they already answered that there wil be pause rendering, it has been said on day one on the 3.0 back to the future thread.
I think they are aware of the issue with the preview as it’s written in the 40a change log.
“Known issues : Preview sync, play/pause laggings”

maaru12315 · July 20, 2022, 11:44am

Update: I fixed the issue by changing the AI processor from auto to gpu…now everything works.

suraj · July 20, 2022, 1:44pm

Choose model in slow motion and apply it at 100% and then set fps at desired number.