Topaz Video AI Early Access v3.0.0-3

Hello Everyone,

We have another update for Topaz Video AI 3.0 with lots of fixes and improvements. Please provide your feedback.

Downloads
Windows | Mac
Headers, Dylibs, & Libs for custom FFmpeg builds

Changes from v3.0.0-2

  • Added TensorRT models for Deinterlace and Enhance models
  • Added grain/noise filter
  • Added Reduce Jittery Motions for stabilization (iteratively reduces inter-frame jitter-like motions)
  • Export can be opened in external players or editors regardless of the Auto Recovery setting (previously needed it to be turned on)
  • Fixed Gaia failing on Mac
  • Fixed model download failure
  • Shows warning if output resolution is not supported by the selected encoder
  • Fixed Apollo model failing with Auto device setting
  • Improved zooming sensitivity in the preview
  • Misc. fixes and improvements

Known Issues

  • Users may experience delayed switching of videos when previewing
  • Some videos can still have some seeking issues in Preview/Export
  • Preview mismatch for certain files
  • Color space / HDR issues
  • Keyboard navigation missing
  • Intel codec not working on some machines
  • Some quality mismatch between 2.6.4 and 3.0
  • Multiple selection for source videos is missing
  • Output bitrate is not reset to Preferences bitrate when importing a new file
  • Different view mode is still missing
  • Failure can happen when resizing to a big resolution like 4k or 8k without a warning if machine runs out of memory
  • Speed can be slower than 2.6.4 with some models on some machines. We’ll be working on speed optimization in the next weeks
  • Proteus with Auto + Stabilization may cause artifacts
  • Preview with image sequence doesn’t work or plays at 25fps

Problems and Solutions:

  • Make sure Max. Memory Usage in the Preferences is set to a high number
  • If the Gaia model fails on Mac, try re-installing the app using the DMG file
  • If export fails, try using Transcode/None/Auto in Audio Settings in Preferences

Proteus Auto Caveats
The adjustments will not be visible and the sliders will show values of “0” when in auto mode. However, each frame is being analyzed for optimal settings while processing. If you keep Auto enabled and modify the sliders, the settings you choose will be value-adjustment percentages. For example, if Auto estimates the Sharpen slider with a value of 50, and you manually select -50, then the actual value applied will be 25.

CLI Usage
CLI mode has returned for Topaz Video AI 3. Detailed usage instructions are available here, on our support website.

Frequently Asked Questions can be found here.

Please upload videos that are not working or having any type of conflicts to this DropBox Folder.

5 Likes

The side by side preview playback has improved. It’s still not great but it’s noticeably better in this one.

3 Likes
  • please offer a single frame preview
  • please offer zoom steps via mouse of fix +/- 5 % or 10 %
  • please make the app remember my zoom level
  • please make the app remember my pause on preview
  • please offer a cut selection by frame number
  • please offer different types of previews again
  • please do an anti zombie training for small faces
  • please make the vertical scroll bar wider
  • please offer .jpg image output again
  • please offer a live preview of the last export frame
  • please offer a custom setting for the audio bitrate
  • please make ve ai take into account the Windows scaling
  • processing a second instance failed: out of memory
  • the preview during play repeatly jumps back to frame one
  • recover details causes bad patterns when set too high
  • add noise and add grain: what is the difference?
  • cpu/gpu usage is 50/70 % now (8700K + 2070S)
  • “recover details” does not recover details but makes existing ones more visible
  • to recover details better ramp up the dehalo option slightly!
  • the gui crashed during processing. here are the log files:
    logs.zip (643.8 KB)
  • after crash of the gui please enable the gui to continue after restart when ffmpg is still processing
  • during processing the preview uses a servere amount of processing power: on a 10 hours processing time you can save one (!) hour by switching the preview to pause.
11 Likes

Hope to take part in testing VAI 3

1 Like

I’m working on compiling a custom ffmpeg with veai and other GPL sources, but it would be really cool if you guys could just make an avisynth plugin so that we don’t have to muck around with this; compiling ffmpeg on Windows is a mess at best; adding a custom patch is pretty tough.

4 Likes

Excited to see the speedups, especially on the frame/CPU encoding time because I’ve seen strange situations where slower GPUs (ie RTX 3060 + Ryzen 3900X) are faster than my 3070ti + 5800X in the same 480p / 720P to 1080p upscaling tests.

I do hope we can get more info / content added to the pane and the easy 'click-to-change-view for quad comparison- dropdown box like on 2.6.4, and a larger indicator for the speed / seconds per frame indicator instead of in the mini-box when doing previews.

For speed tests, I have a Precision 5570 (12800H + A1000, 64GB DDR5 + NVME) which I ran both 2.6.4 and 3.0 on. The results, both using the same settings and source video (Big Buck Bunny MP4, 30mb 720P sample):

Topaz Video AI 2.6.4:
CRF 17

720P → 1080P (No Crop)
Speed: 0.28 → 0.30 (150frames rendered)

720P → 720P 0.18-0.19~ (150frames)

Topaz Video Enhance AI 3.0:
Artemis Medium/Normal, Nvidia H264 High, copy audio, mp4)

720p → 720p 0.12s
720P → 1080p 0.19s

Which is ‘basically’ a 50-75% speedup on the same hardware if the reported speed is accurate. This A1000 is only a tiny bit slower for 720P → 1080P than my full blown 235W UV/OC tuned RTX 3070ti at 2GHz, which is crazy and promising for when I can try this on my desktop later.

But I did encounter that issue / bug when trying to re-render / do another preview. I think it may be because the speed limit options revert to the older runtimes/encoding methods which are much more inefficient, especially on a clockspeed limited laptop?

For example:
720P → 1080P took 0.28s a frame with: 98% Memory limiter, ALL GPUs, 12800H Iris Xe + A1000
(0.19 → 0.28 which is 2.6.4 speed, and a LOT slower ofc)

To reproduce, while still in the session / application open from an ‘All GPU’ run, revert to your good/proper GPU (ie my) Nvidia A1000. Despite telling it to use your primary GPU only, a speed penalty still in effect.

What I found is to revert to 100% (and only 100%) memory usage / close and open and speed is restored. Atm using any speed limit, even if you’re nowhere near to VRAM usage (600mb VRAM usage reported by GPU-Z out of 4GB) incurs a large penalty on speed and slows it down to 2.6.4 levels on my laptop.

Another bug is that the default container (mov) is selected for the ‘defaults’ in the config menu even though on previous runs I selected MP4 in that area. Strangely the video itself pre-selects MP4 as desired (I believe) which is an inconsistency.

See below screenshot for render/AI runtime differences between 100% and 98% memory limit selected with the same everything else (Nvidia A1000 laptop, 50W max, 1300-1500MHz & 2048 cores)

5 Likes

I’m on my 5800X (3600MHz RAM) + 3070ti (2055 MHz - 235W) and testing the 3.0.0-3 release:

I’m experiencing low FPS in the application itself on my desktop while playing/previewing the video, I never had this problem with any previous versions of Topaz Video Enhance AI. There’s no reason why it shouldn’t be locked to my monitor’s refresh rate (144hz) given how simple the UI is, and we’re accelerating video decoding with GPUs and CPUs now?

MSI afterburner is saying while playing a 30fps (video) preview (720P → 1440p) & upscaling that the Topaz Video Enhance AI’s window is (1920x1080) is running at 44-45fps, jumping to 53-65 fps and sometimes down to 1/2fps while idle.

This behavior is causing noticeable flickering and weirdness on my Freesync/GSync display, hence me discovering this regression.

For performance, I do not see any speedups for 720p → 1080P or even 1440p (Both taking 0.15-0.16s a frame) video, BUT I have noticed that my 3070Ti’s power consumption while rendering the video is significantly lower for the same results/models/speed etc. From 190-215W consistently to 65 / 85/ 125W (peak) jumping around. This efficiency increase for the models definitely explains why my A1000 laptop was seeing a huge performance benefit, but awkwardly it seems past that my GPU remains underutilized.

Despite me selecting NVENC H264 High for my encoder, while running the Preview the encoding speed on 3b is the same as 2.6.4, and the CPU load is around the same for both versions. I don’t suppose there are ways to further reduce CPU usage on future updates, as for single-video upscales to 1080p/1440P max, I’m definitely still limited by something unrelated to my GPU.

3 Likes

Many videos have duplicate frames for whatever reason. Is there or will there be the possibility to remove those duplicated frames?
it is not visible that every sixth picture is a duplicate if you play the video at 30fps but if you change the fps to 60 or do a slow motion the video starts to stutter.

2 Likes

Duplicate frame removal could be incorporated with Chronos, based on the initial frame rate?
A setting for the number of consecutive duplicates to be replaced with newly generated frames!
Options for how app should deal with variable frame rate and areas of continuous duplicates!

2 Likes

Is it possible to implement version 2.6.4’s UI?
Because this new one is so clunky.

2 Likes

some source videos may have doubled frames already. this might have happened for example if someone took a 23,976 fps movie and encoded it by mistake with 30 fps. in this case it often helps to just convert the original file back to 23.976 fps first to remove the doubled frames.

Will support for AVS files be added to version 3? I don’t seem to be able to use an avisynth file from the command line or by dragging the avs file onto the app like I could in previous versions.

I have excessive flickering too, unless I turn off g-sync. I got an older g-sync display with the physical module (Asus PG348Q) and a 3080ti

Thank you for your feature requests :slight_smile:

Grain adds texture which can give the output a more lifelike appearance.

Adding a small amount of noise before converting media to any codec is a good way to reduce compression artifacts.

4 Likes

What about v3.0’s interface do you find to be clunky? The more specific you can be the better.

Thank you!

1 Like

If I’m being honest. All of it. It’s harder on the eyes, the options now all make less sense. I loathe the fact you can no longer just tick a box to select all the files for bulk jobs.

The old UI is just so seamless. Go back to it and keep the new AI models. Don’t fix what isn’t broken. Pleeeeeeeeeease.

2 Likes

Hi, unfortunately, we are not able to support AVS due to a licensing issue.

Thank you for your feedback :slight_smile:
The ability to select all and batch process will be added back in prior to release.

1 Like

avs is the file extension for avisynth. I’m not sure what licensing you’re referring to?