Proteus 3 black splotche blotting

Here are the test results with 3.2.2.
VEAI_error_input_lagarith.avi is a good test video because it is relatively reproducible.

CLI
“C:\Program Files\Topaz Labs LLC\Topaz Video AI\ffmpeg” -y -i %1 -sws_flags spline+accurate_rnd+full_chroma_int -filter_complex tvai_up=model=prob-3:scale=4:compression=0:details=0:noise=0:blur=0:halo=0:preblur=0 -an -c:v prores_ks -profile:v 3 -vendor apl0 -pix_fmt yuv422p10le “%~n1_xx.mov”

1 Like

logsForSupport.tar.gz (8.1 MB)

box-00.01.05.047-00.01.18.264.zip (30.8 MB)

Dropbox didn’t exactly work, but I attached the logs + a small sample of the movie I used. These were its settings:

As you can see, I tried a ‘pan & scan’ zoom (crop to fill), and started a preview at 24 seconds into the clip.

That is when things always go wrong: when you try to ‘enlarge’ the image. The process seems to run out of video memory somewhere.

N.B. This ‘enlargement’ is an issue by itself, btw. And Topaz is doing it wrong. :laughing: No offense. Seriously. What Topaz appears to be doing (which you can deduct from the time needed to create the Preview), is that they upscale the image first, and only then ‘crop to fill.’ This is bad (regardless of blotches), as it means the resultant image, pre-crop, is way larger than 3840 x 2160 (both in height and width), and only thereafter is cropped to fit the 4k constraints. And processing that surplus area takes a lot of extra time. Best is just to pre-crop first. On a 1080p source file, with say typical 140px black bars, you’d start by cropping 248px (140x1.77) from both sides, plus 140px from top and bottom, and only then do you upscale to 3840x2160. By pre-cropping, you avoid wasting time (and memory!) processing a much larger image first. This is a bit beyond the scope of this post, though, but I thought it was interesting to already mention.

Thank you! @meimeiriver @TicoRodriguez @ForSerious

This is all very helpful!

2 Likes

Look at this one: v3.2.3 goes besers on ‘Auto’, even without any cropping;

Or

Curious, if you do not use Auto does this still happen with this file?

Yes, it does. I posted more instances of this occurance at:

new blotches weirdness

If you want, I can create a short sample again.

3 Likes

Thank you, no need at this time. We are investigating this.

1 Like

I’m enhancing some old DVDs and I noticed occasional large, black splotchy artifacts generated by the Proteus model at settings of 300% scaling or above. (I couldn’t reproduce it in Artemis or Gaia, or Proteus at 200% or 100% scaling settings. But 300%+ shows the artifacts.)

If you’d like to try it for yourself, here’s the input file. It’s a lossless AVI in the YUV420 colorspace using the free “Lagarith” codec.

Thanks for looking into this. Proteus is the model I use 9 out of 10 times. I’ve seen other instances of these black splotches too, so this isn’t a one-in-a-million case.

1 Like

Looks like the exact same issue I had in 2.6.4: Seriously wrong blotting - #3 by meimeiriver

I tried, but no error occurred.
What settings did you use to convert?
My setup is as follows: output to ProRes with CUI.

“C:\Program Files\Topaz Labs LLC\Topaz Video AI\ffmpeg” -y -i %1 -sws_flags spline+accurate_rnd+full_chroma_int -filter_complex veai_up=device=-2:model=prob-3:scale=4:compression=0.24:details=0.24:blur=0:noise=0.16:halo=0:preblur=0.08,scale=w=2880:h=2160:out_color_matrix=bt709 -color_primaries 1 -color_trc 1 -colorspace 1 -color_range tv -an -c:v prores_ks -profile:v 3 -vendor apl0 -pix_fmt yuv422p10le “%~n1.mov”

It’s happening only in zoom 300% - bug work on my computer.

For my case (see above) it happened, for 1080p (to be upscaled to 4k), at ca. 270% zoom. Looks to be some sort of issue with running out of memory.

I take it you meant ‘without’ any problems? At any rate, the problem never occured for me with Proteus 2, only 3. And, as you can tell from my post, support clearly acknowledged the issue, and said it was on their todo list.

yes, meant ‘No’ problems.

And @TicoRodriguez is, I believe, a user like us who has signed on as a beta tester. I don’t think he’s in Topaz Support

I don’t use the command line directly; not sophisticated enough for that. I drop the file into the UI of Topaz Video AI v3.0.5, upscale to 300%, Proteus model with all of its parameters set to 0, and export to ProRes. Simple. That’s enough to produce the error. Here’s a copy-paste of “Show preview command” in the UI if you want it:

E:\ffmpeg "-hide_banner" "-nostdin" "-y" "-nostats" "-t" "2" "-ss" "1.2929583333333334" "-i" "C:/Users/someguy/Desktop/Dropbox/VEAI_error_input_lagarith.avi" "-flush_packets" "1" "-vsync" "0" "-avoid_negative_ts" "1" "-sws_flags" "spline+accurate_rnd+full_chroma_int" "-color_trc" "2" "-colorspace" "2" "-color_primaries" "2" "-filter_complex" "veai_up=model=prob-3:scale=0:w=2112:h=1440:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:device=0:vram=0.9:instances=1,scale=w=2112:h=1440:flags=lanczos:threads=0,split=2[out1][out2]" "-map" "[out1]" "-c:v" "tiff" "C:/Users/someguy/AppData/Local/Temp/somenumber/temp/_tvai_867486382/%06d.tiff" "-map" "[out2]" "-c:v" "prores_ks" "-profile:v" "3" "-vendor" "apl0" "-bits_per_mb" "8000" "-pix_fmt" "yuv422p10le" "-map_metadata" "0" "-movflags" "use_metadata_tags+write_colr " "-map_metadata:s:v" "0:s:v" "-an" "-metadata" "videoai=Enhanced using prob-3 with recover details at 0, dehalo at 0, reduce noise at 0, sharpen at 0, revert compression at 0, and anti-alias/deblur at 0. Changed resolution to 2112x1440" "C:/Users/someguy/AppData/Local/Temp/somenumber/previews/VEAI_error_input_lagarith_prob3.mov"

FWIW my system is
Windows 10
AMD TR 1950X
64GB RAM
Nvidia RTX 3090 (24GB)

I know this. :slight_smile: But my confirmation came from Topaz Labs support directly:

“I have shared all of this with our development team and they are working on a solution, however, this fix will take some time. In the meantime, please use v2, you should not find any advantage to using v3.”

So, they are definitely aware.

Dunno, is Proteus 2 still a thing in VEAI 3? If so, maybe you could try 'veai_up=model=prob-2’?

Thanks, but no model manager in VEAI 3’s UI, so idk how to access that. (The black-splotch issue is not a show-stopping issue for me personally, so I’ll just live with it until it’s fixed. I just wanted to report the issue.)

Had it not been for Proteus 2, yeah, those blotches would have definitely been a deal-breaker for me. :crazy_face: Removing Proteus 2 (at least from the UI, haven’t tried the CLI yet) is another area VEAI 3 oddly regressed, for no good reason.

Yes, I am a beta tester.
I’ve never seen that happen at my side, but I may have just not noticed it, so I wanted to make sure it was reproducible.
Also, if it is a reproducible bug with a lot of users, I think it would be a big request for a fix.