Topaz Video AI Beta v3.5.2.0.b

Topaz Video AI Video AI Beta
1

Hello everyone,

We have another beta available for your testing.
3.5.2.0.b

Changelog from 3.5.1:

  • Adds Iris options to select between Medium/Low Quality. Medium will use Iris-v2, Low will use Iris-v1.
  • Adds 1x & 2x deinterlace options for Interlaced videos when keeping original fps.
  • Adds detailed tooltips for all preferences.
  • Adds info about Iris & Nyx models to Progressive models tooltip.
  • Adds detailed metadata for Iris & Nyx enhance models to exported video file.
  • Adds “Noise” value for enhancement models to video metadata when not zero.
  • Fixes ffmpeg issue causing exported video to sometimes get stuck in playback.
  • Fixes for rotation issues for files with rotation metadata. This was causing many of the issues like “file opened upside down.”
  • Fixes issues selecting “original” fps option in fps selector.
  • Fixes FPS display to drop extra "0"s when the fps is a whole number.
  • Fixes filter order when deinterlacing which should lead to higher quality visual results.
  • Fixes metadata for exported video when using Iris to only say “Deinterlaced” when the original video was actually Interlaced.

Known Issues:

  • Currently not all the models are supported on Intel ARC (model support will be added week to week).
  • Videos with mismatched metadata and streams will display incorrect duration
  • Frame number preview length may shorten on app restart
12 Likes
2

I still can see some weird stops and the playback of a completed preview is completely wonky (the seeking triangle just runs to the end of the line). I can also see the difference between the “fluidity” of frame interpolation in v3.5.1.0.b and v3.5.2.0.b - it seems it got worse, I checked it on that Steam Deck 10 fps animation (I brought it to 60 fps with Apollo V8).

image
image1920×1032 102 KB

This image stays since I started typing this report, and the progress of the preview doesn’t move. yet the preview file increases its size, which means FFMPEG writes stalled frame into the file.
This is my GPU usage during this stall- just 8-11%:
image

Edit: I had to interrupt the preview, since the file rose to impossible size while not progressing any further. The preview file is unplayable, I was using 16Mbit/s H.265 NVENC MP4 output.
Edit 2: The issue might happen during second or third video ‘played around’, while the others are opened or even one of them is being ‘previewed’. It might not necessarily appear with just the first video dropped there and previewed. Also, previous custom presets are read wrongly, I think, so that could also have something to do with it. :slight_smile:
I’ll upload the log file from the ‘stall’ time.

uploaded! To the DropBox from this thread. :slight_smile:

Edit 3: The playback issue - I’ll upload it to the DropBox, too, since I can’t put .mp4 AVC file here. I’ll also show, what this playback issue means for inconsistency of the preview (left click/release). :slight_smile:

Edit 4: Frame Interpolation now doesn’t stick/work with original frame rate, why? It was a nice option to make some animations fluid smooth by using Replace Dup Frames, while preserving their original FPS.

3

I tested an alpha build for this on Ubuntu with Intel ARC.

You should probably make it a “known issue” that ARC absolutely does not work with more than 1 GPU right now.

So far, the only model that looks like it actually works on an ARC GPU is Gaia, and it fails on everything above SD input quality and uses CPU anyway. The video I initially tested and got to work with Gaia was 624 × 352 upscaled to 1920 x1080. GPU selected runs twice as fast as CPU, which really isn’t great but it’s something.

I test an input video of 1920x1080 and get this error in log:
2023-09-30 02:21:27 140534189125632 CRITICAL: - Running CPU fallback because of this error [ [ GENERAL_ERROR ] Check ‘false’ failed at src/plugins/intel_gpu/src/plugin/program.cpp:401:
GPU program build failed!

I haven’t found a single other model that even tries to use the GPU. I asked for a running list of models that are supposed to work in the last beta thread but didn’t get an answer.

I picked up 2 ARC A770s because according to the last beta thread these were supposed to work in Ubuntu. I know Nvidia 10-series GPUs also do not work on Ubuntu and AMD GPUs aren’t even supported yet so can someone please tell me what GPU is actually tested and know to work in Ubuntu?

4

Thank you so much! :slight_smile: :+1:

4 Likes
5

Thanks very much!

Now I have to test the speed as Iris V1 and V2 did get dramatically slower on Apple silicon in the last release(s) compared to earlier versions.

3 Likes
6

This variable was added to the desktop launcher for Ubuntu because ARC cards are not detected correctly as it seemed to help:
“OV_USE_DEVICE_INDEX=1”

However this causes Nvidia cards to not be detected correctly. After removing it a 10-series is at least detected correctly, still has the Cuda issue though.

7

f957fa086f69334b2304c6fe12297ae6
f957fa086f69334b2304c6fe12297ae63778×1886 367 KB

{90A6397A-4418-4b18-A826-31CCA3BAC222}
{90A6397A-4418-4b18-A826-31CCA3BAC222}1402×136 7.73 KB

Some clips can’t be playbacked and the 3.5.2.0b still gets stuck causing the cpu to fill up!

8

@suraj . You asked to let you know about the frame interpolation
 and I did. No reply from you.

Every 15 to 60fps conversion result in first 4 frames being dupes. Like a freeze. Then seems to have no dupes.

It shows in the preview of VEAI when you play it back. It holds then shows the preview which are subsequently not in sync with eachother


EDIT. its on all videos! 4x = first 4 frame tilt. 2x is first 2

BTW why cant I upload videos here?

9

The Iris 2x “anomaly” unfortunately carried over into this Beta:

Bildschirmfoto 2023-09-30 um 14.03.38
Bildschirmfoto 2023-09-30 um 14.03.381580×738 137 KB

Something happened between the 3.5.1.0.b and the 3.5.1 release version with a drastic speed loss of Iris on Apple silicon with the exact same task taking about 6 mins in the 3.5.1.0.b but 10 mins in the 3.5.1 release version and now also on this 3.5.2.0.b.

And 2/3 slower really is a dealbreaker.

10

Preview still does not work correctly

2 Likes
11

Strange enough, after reinstalling the earlier 3.5.1.0.b the speed of this version is as slow as the new one.

So I guess this performance drop is due to changes to model and while 3.5.1.0.b formerly used the old model that was already downloaded it now (after deleting the whole folder) was forced to redownload the newer model and thus is slower as well.
Iris V2 while being much faster than V1 before (except for the 3.4.0.1.a where Iris V1 simply flew) now is down to the exact same speed as V1. And I cannot say that this is due to better quality or such, just the same task at 60% of the possible speed.

SO: Which changes were there to the Iris V2 model in the background and also WHY (as they’re really hampering performance on AppleSilicon)?.
Same goes for the V1 model but there already earlier somewhere between the 3.4.0.1.a and the first 3.4 beta versions.

For some figures Iris now runs at 12-13fps while it was 20+ fps before on the exact same encode.

1 Like
12

[Topaz Video AI Beta v3.5.2.0.b]Adding apollo and other Frame Interpolation models can easily cause the export program to get stuck and the cpu to be full !!! Please fix it

13

Testing out a 4070 now on Ubuntu


That is definitely still a problem. I always use manual, estimate, then I tweak the settings. The export ALWAYS fails at different points if I use “estimate”, it has been fine as long as I don’t use estimate. This is a “known issue” why is it not listed? Now I wondering if some of my ARC issues were caused by this.

EDIT: Actually I don’t know what’s going on with this. Once I’m using this beta and it’s pretty obvious that “estimate” is causing the problem. Next time I use it estimate doesn’t cause any issue.

14

Problem on Linux too, doesn’t matter what GPU is in use. Chronos gets stuck 90% of the time and Apollo generates black frames according to the preview

This issue was reported in the last beta and isn’t a “known issue” either.

15

The aspect ratio dropdown randomly stops becoming visible on anamorphic sources. It’s there sometimes but at some point it stops showing. I think it’s when you add an input video that is not anamorphic.

16

How is this different from the “Interlaced Progressive” option?

Also FPS detection doesn’t seem to work correctly in this version. Any MPG files I have seem to show double the FPS they actually have. This can be reproduced fairly easily by just generating files with FFmpeg:

ffmpeg -f lavfi -i testsrc -t 30 -pix_fmt yuv420p test.mp4
ffmpeg -f lavfi -i testsrc -t 30 -pix_fmt yuv420p test.mpg

The MPG file will show up as 50 instead of 25 FPS.

17

I’ve noticed that both the preview and export still works, it’s just that they’re completely out-of-sync so even when the viewer stops at 100% the program is still processing the footage as evidenced by the small timer still going
 ⇐ EDIT: For clarities’ sake, It does finish as the little fps counter is still going past 100%


_(As far as the broken motion deblur, it can still be used if you use my earlier workaround by multiplying the original frame rate of the footage up to 3, for example: 28.081 × 3 = 84.243


*If you want more than 84.243 fps, simply multiply the resulting frame rate by either 2 or 3 depending on how high your results are


So: 84.243 × 2 = 168.486


and if you want to have more frame rates while staying within the 240fps limit, add the original frame rate to your earlier results


*28.081 + 168.486 = 196.567)_⇐ EDIT, I’m still doing some testing, so this info may change.

EDIT: It has indeed changed a bit, not only will you get duped frames from using motion Deblur, and Reduce Jittery Motions, but also if you use the stabilisation model, again, if you want to interpolate without dupes you need to multiply your frame rate up to 3, any more and you get super-dupes!

I hate maths, but it needs to be done if you want to use motion deblur, and as far as reduce jittery motions, it’s pretty broken as well, as it completely blurs out fast motion from the camera, I’ll need to do more tests


TVAI VIEW OF VIDEO WITH RJM
TVAI VIEW OF VIDEO WITH RJM2563×1134 388 KB

Without RJM:

TVAI VIEW OF VIDEO WITHOUT RJM
TVAI VIEW OF VIDEO WITHOUT RJM2564×1150 467 KB

NOTE: Animation and slow motion benefit greatly by using high frame rates.

18

those are two different things.
Interlace x1 or x2 means to deinterlace an interlace source either by keeping the original fps, by dropping half the interlaced fields (odd or even frames).
Or by bobbing (x2 fps), making each interlaced field a full progressive frame (with interpellation).
This results in the video having a frame rate identical to the original field, but each frame having half the vertical resolution, or resolution equal to that of each field that the frame was made from.

progressive interlaced is an AI meant to fix progressive source you feed TVAI with, that was once an interlaced source that was converted to progressive at the time / in the past without deinterlacing them with any deinterlacing method.
that is the only time you should be using interlace progressive.

image

2 Likes
19

That’s an understatement! It’s a HUGE MESS is what it is


20

Got an error at the final ffmpeg using Iris2 2x. I have the full log available if you can tell me where to submit it. Thanks.

[mov,mp4,m4a,3gp,3g2,mj2 @ 0x1347185c0] stream 1, offset 0xffffffffccc48418: partial file
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x1347184b0] Error during demuxing: Invalid data found when processing input
[in#0/mov,mp4,m4a,3gp,3g2,mj2 @ 0x1347184b0] Error retrieving a packet from demuxer: Invalid data found when processing input
[out#0/mp4 @ 0x134726390] video:0kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
[out#0/mp4 @ 0x134726390] Output file is empty, nothing was encoded