Topaz Video AI Beta v3.5.3.0.b

Hello everyone,

We have another beta available for your testing.
3.5.3.0.b

Changelog from 3.5.2:

  • Adds “Black and White” toggle to video input options.
    • This option will desaturate output to prevent models from adding color to Black & White videos.
  • Adds “Telecined” toggle to video input options.
    • This option will reverse the telecine process and deinterlace your video to 23.976 fps before applying AI models.
  • Adds detailed tooltips to Output Settings.
  • Enables ProRes, H264/5 and AV1 encoders for Linux.
  • Fixes many color-banding issues seen in the in-app player.
  • Fixes update issue related to frame interpolation / ffmpeg that some users experienced after updating to 3.5.0/3.5.1 and was not fully addressed by 3.5.2.
  • Fixes tooltips so they never go off-screen.
  • Fixes UI issues in video settings when information displayed is multiple lines.
  • Changes filter tooltips so they are now also seen when you hover on the filter name and not just the icon.
  • Changes “os” field in video-encoders.json to a regular expression (regex) evaluation to give more flexibility.

Known Issues:

  • Currently not all the models are supported on Intel ARC (model support will be added week to week).
  • Videos with mismatched metadata and streams will display incorrect duration
  • Frame number preview length may shorten on app restart
8 Likes

Is there any update on the color shifting issue?

2 Likes

You Topaz guys are gonna have to start sending me a check for testing this thing lol

There is a bug here somewhere that leads to high initial VRAM spikes in Linux. I can’t figure out if it’s Topaz or Nvidia’s drivers. Some of the models are well behaved. If it just so happens that the effected models do not exceed the cards VRAM in the first 3 seconds, it levels off and runs fine at a reasonable memory usage. I’ve wiped out all my models and let it download new ones. I’m running a 4070 12gb and I don’t even run a monitor on this GPU, this one is in a gen4x16 slot and I have another Nvidia GPU running the display. I’m only selecting this GPU in Topaz and nothing else is using it.

The VRAM usage setting in Linux seems to do absolutely nothing. I’ve seen reports of issues with other programs on Windows too on the Nvidia forums but TVAI may be getting away with it in Windows because there’s actual control over the VRAM usage in Topaz on Windows, but that’s just a guess.

Test clip is 640x480. All of these tests were done with just the single model running, original resolution and framerate output. If there is an initial spike over 10gb the export will fail. If it spikes but stays under 10gb it will likely settle out and be fine with reasonable VRAM usage for the rest of the export. Some models are completely well behaved from start to finish.

-Gaia CG, spikes > 10gb and fails
-Gaia HQ, spikes > 10gb and fails
-Artemis Alias or Moire, spikes > 10gb and fails
-Theia detail, spikes > 8gb and fails (usually)
-Theia fidelity, spikes > 8gb and fails (usually)
-Apollo, spikes all over the place and completes, produces nothing but black frames
-Apollo Fast, spikes all over the place and completes, produces nothing but black frames
-Chronos Fast, spikes all over the place and fails

The Theia models will die on a higher input resolution due to a higher VRAM spike, they just barely make it @ 640x480. All the models that work work fine at least upto 1080P input clips (highest I tested). All the older versions of the bad models do the same thing.

On rare occasions during a 15s preview the bad models may actually succeed, they just so happened to spike a bit under the point where they fail, if they get through the spike they level off and are fine These are all the same 640x480 non-scaled clip.

This is Gaia HQ when it works, every bad model does this but they usually just fail rather than level off:

This is Iris, Iris always works. Same exact clip:

This is Nyx, Nyx is fine too:

The high GPU load is good, that’s what we want to see. The high/erratic memory usage is the problem.

I have never seen it mentioned that Linux uses a different model set than Windows or that the models are incomplete so what is it about these models that is different? If these models are incomplete you Topaz guys should make a list so nobody wastes there time messing with them. Like the Intel Arc Models, why don’t you guys just make a list? Nobody has any way of knowing whether the model actually exists from using it on the GUI so nobody knows if a bad model is the problem. Intel ARCs already have enough problems.

The same models that have issues in an export also error out on a benchmark.

Topaz Video AI Beta  v3.5.3.0.b
System Information
OS: Linux v6.2
CPU: AMD Ryzen 9 5900X 12-Core Processor  31.254 GB
GPU: NVIDIA GeForce GTX 1080 Ti  11.24 GB
GPU: NVIDIA GeForce RTX 4070  11.994 GB
Processing Settings
device: 1 vram: 0.6 instances: 1
Input Resolution: 640x480
Benchmark Results
Artemis		1X: 	88.07 fps 	2X: 	48.62 fps 	4X: 	11.37 fps 	
Iris		1X: 	114.08 fps 	2X: 	71.93 fps 	4X: 	15.14 fps 	
Proteus		1X: 	96.78 fps 	2X: 	54.93 fps 	4X: 	10.98 fps 	
Gaia		1X: 	  ...    fps 	2X: 	ERR fps 	4X: 	ERR fps 	
Nyx		1X: 	06.82 fps 	
4X Slowmo		Apollo: 	ERR fps 	APFast: 	ERR fps 	Chronos: 	12.90 fps 	CHFast: 	ERR fps 	

The “vram: 0.6” was just me playing with the slider, it doesn’t make a bit of difference whether it’s @ 100 or less.

I process all the time Black & white videos and I never saw a situation where it added color to it.
can anyone elaborate what this is for? maybe i misunderstood this feature.

This it not speciffically related to this beta version but applicable to the latest versions in general:
Iris Face Enhance has hughe problems differenciating between foreground and background in shots like this:

Poor Tuvok.

4 Likes

The Iris 2x speed anomaly still is there with that model being abnormally slow on Apple Silicon (while it was about twice as fast until only a few weeks ago).

This goes for Iris V2 but also for V1 (only there already since quite some time). It would be nice if the former speed-up - announced with the first 3.4 alpha - would return.

1 Like

I have seen this happen in previous versions.
This is a 400% zoom from 3.3.3.0.a

2 Likes

COOL, I’ll be testing to see if the interpolation issues crop up during stabilisation and deblur while encoding.

1 Like

Can someone tell me why Artemis 2 X with NYX 1 X does correct the faces while Iris cannot do this

Is

Huh, I never knew this, I’ll test it as soon as I’m done un/installing the prog, as I have to use windows uninstall troubleshooter due to a deleting shortcut freezing bug during uninstallation that has been plaguing me for a while yet!

Not to forget again:

  • the preview and/or preview slider and/or processing progress bar gets temporary or fully stuck at some point.

A personal note: I really can’t understand why the most common issues with Video a. i. are regulary not mentioned in the “known issues” section. Are they hidden intentionally? :eyes:

5 Likes

How about having some way to recover failed videos? Right now you can have a video spend a couple of hours just to fail halfway through and be deleted. How about TVAI just leaves the video in a separate folder or something?

2 Likes

You could process it to frames and then continue with the next frame for the case that the frame number issues were fixed and the saved image numbers and the in application image numbers would match. Recently they didn’t in version 3.x but in version 2.x.

I’m not interested in processing images but the unfinished video is perfectly useable in video editors. I just don’t want hours of processing to be deleted against my will for no reason.

A couple bugs I’m finding that have been around for a while, but I failed to point them out.

  1. On macOS The file order when adding multiple titles at once instead of 1,2,3,4,5 they load 1,3,2,4,5
    Details: I’m unsure if this was ever truly correct, if I have a series of videos, and I import all of them, either by File Import, Clicking the Import button, Clicking the interface, or Drag-n-Drop from the Finder, all methods seem to randomly load the files in different orders. Subsequent attempts to load the same files, seem to load them in the same order, never in alphabetical or numerical order.

  2. Trim resets your position instead of beginning at the position you preview
    Details: The old trim tool before the upgrade a few versions back, began the trim mark where you had scanned the file before. So an example is you have scanned a 1 hour clip, and you are at the minute mark 00:25:49:209 of a 01:05:19:189. If you then click the “trim” option, the position of the play head is reset to 00:00:00:000. It would be ideal if the trim position picked up the current play head position as the start, and the end of the film file as the end of the trim. Does anyone else feel this way?

  3. Strange FPS (Original) getting multiplied to 240:00fps instead of 29.83fps.
    Details: Some time back, maybe during the 3.4.x releases, some of my old archive video files that upscaled just Fien at 29.83fps now when attempting to load them say that their FPS are 240.00 FPS (Original). If I get the old _prob3.mp4 file created by TVAI3.3.x version it loads correctly at 29.83fps, and you can “copy the settings” and “paste the settings” onto the incorrect original, which somehow changes the (Original) fps, but as soon as you attempt to either PREVIEW or EXPORT than the FPS reset to the incorrect 240.00fps causing the Frame Interpretation to engage and causing a 30second clip to take 3hrs instead of 3min.
    Handbreak says that the file is a 29.38fps, and but if I convert the file via Handbreak using the “Same as Source” option for FPS, the resulting file, nothing else changed, appears to load correctly into TVAI 3.5.x. Very strange behavior.

1 Like

What happened to the height/width, and scale adjustments!?
Screenshot 2023-10-07 221945

2 Likes

@jacob.muchow Error on loading Video
Archive.zip (47.1 KB)

untick the “Lock Aspect Ration” and you will have them both available

1 Like

The anamoly is also in Nyx … so it is not iris when a close up is done then Nyx is better more natural with eyes then iris …

When i choose iris model and choose manual (Everything set to 0) then the image gets processed too. Is that the way it’s supposed to be? Can remember back in time when i choosed manual and all to 0 then the video was the same than before.

Unfortunately, I can’t edit the video well that way. Because the filter is so strong that there are a lot of artifacts in the image and everything is wobbling and moving where there should be a smooth image.

It would be nice if we can upload small preview rendered video clips from topaz here in the forum. So i could show the artifacts i mean. A photo cant show this.

Theia model is creating strange artifacts too: