Besides, Proteus 3 is still suffering from its infamous black-blotches-syndrome (when upscaling big), making it mostly dangerous to use.
1 Like
Thanks. Didn’t know that existed. 
I heard that and tryed out, but for me it was to invasive. I spend much time now into source preparation. since I realized that is particularly important.
First I check if its interlaced or not. Virtualdub/Avidemux are very handy to have a look at this.
Deinterlace/Scale (Hybrid)
When needed I do QTGMC deinterlace and sometimes resize a little bit in order to bring it to an even divide by 2 number and/or correct wrong form factors, it’s sometimes necessary to cut off a few pixels. Then I encode this with ffv1
Denoise
Now I have for example a 768x576 prepared source, then I check the noise. For denoising I import it into Handbrake set x265 and type “lossless” in the encoder option field.
In Handbrake register “Filter” choose “NLMeans” set filter mode (most it’s Film or Grain) and set noise reduction level from ultralight to strong. (most light or medium). It’s a shame I’ve been using handbrake for so long now and never used this great noise reduction filter.
TVAI
Until here all tasks were lossless compression done. Now you can import into TVAI or before optimize contrast/gamma/colors in you’re video cutter software, do not resize and export it lossless.
Then in Topaz I check results “relative to auto” without user presets first on Proteus, Iris, Rhea, Gaia and decide then. I use only Sharpen + Deblur, both optimally matched. For models Iris & Proteus additional sometimes I use “Rec. Orig Details”.
Downscale
When it’s Rhea I always do 4x and downscale with Hybrid to the desired resolution, here not lossless, it’s the final encode. Then I kill all my generated huge processing data (the lossless thing).
it shouldnt be too invasive 
, if anything for some clips theres barely any change, it only marginally enhanced whats there. just to make sure, it is recovery original detail 100, not improve detail. improve detail is at 0. in my case a 720x1280 source i upscale it to 1080x1920 in hybrid before importing it to topaz. this pre-upscale or not changes how much details the model can detect accurately, it changes whether its generating or enhancing.
Thank you for sharing your approach. You brought me some new ideas. (especially the scaleup prior to upscale)
Could you elaborate on your source a little? I suppose it is not off camera, but old clips with some/significant compression noise and blocking?
I have a love/hate relationship with Proteus4, apart from the hair which i noticed too, but what is your say on the skintone smearing (paint over). All little (i mean very little) color changes in skintone are gone after proteusV4 touched it.
And what do you mean by dehalo -100, the lowest setting is 0 or am i mistaken?
As far as i know “recover detail” is nothing more then mixing in the original frame back into the enhancement output “recover detail @ 100” is a 50:50 mix. That’s what i read in the documentation, though a long time ago or am i mistaken?
if u wondered, and i dont know why myself too, a negative deblur value actually sharpens when used during proteus v4 upscale lol.
1 Like
does also invasive things, yes I use it too, when you have “fluffy” too undefined structures, rec. orig details can make sense, but fix-compression I rarely use and “Dehalo” does wild things I never use it again.
It’s just that this upscale is not enhanced and when I used Rhea 4x with pre-upscaled, this is not desirable, I must do more down scale then, because I want 4k and not 8k. But if you have good experiences with it, everything is fine, we are all trying it out and there is no one way
1 Like
To be honest i have the same experience. On high quality material it delivers significant blur. I never use it. on low res material it might work but never tested. On high res, high quality material its generally a no-go
I cant agree more, noise control on your source is in my experience key and your noise pre-processing varies dramatically depending on source origin. Noise preparation before upscaling is an art in itself.
I wouldn’t use Topaz for noise reduction.
Speaking of upscale, look at what I found (by accident):
ffmpeg "-hide_banner" "-hwaccel" "cuda" "-i" "C:/Volumes/mount.vpy/mount.avi" "-sws_flags" "spline+accurate_rnd+full_chroma_int" "-filter_complex" "tvai_up=model=prob-:scale=0:w=3840:h=2160:preblur=-0.103699:noise=0.11:details=0.129344:halo=0.0276502:blur=0.17:compression=0.23:blend=0.2:device=0:vram=1:instances=1" "-c:v" "hevc_nvenc" "-profile:v" "main10" "-preset" "p7" "-tier" "high" "-tune" "hq" "-pix_fmt" "p010le" "-bf:v" "0" "-b_ref_mode" "disabled" "-tag:v" "hvc1" "-g:v" "0" "-strict_gop" "1" "-rc-lookahead" "0" "-rc" "constqp" "-qp" "1" "-2pass" "1" "-multipass" "qres" "-spatial_aq" "1" "-temporal-aq" "0" "-aq-strength" "1" "-b:v" "0" "-map_metadata" "0" "-map_metadata:s:v" "0:s:v" "-movflags" "frag_keyframe+empty_moov+delay_moov+use_metadata_tags+write_colr" "-metadata" "videoai=Enhanced using prob-4; mode" "H:\veai/recall.mov"
[swscaler @ 000002063A90D3C0] [swscaler @ 000002063AAC0600] No accelerated colorspace conversion found from yuv420p to rgb48le.
Output #0, mov, to ‘H:\veai/recall.mov’:
Metadata:
videoai : Enhanced using prob-4; mode
encoder : Lavf61.3.103
Stream #0:0: Video: hevc (Main 10) (hvc1 / 0x31637668), p010le(progressive), 5664x3184, q=2-31, 23.98 fps, 24k tbn
Wait, wut?! Dafuq is TVAI always doing this idiotic upscaling on crops?! The source is a mounted, pre-cropped movie,
import vapoursynth as vs
core = vs.core
vid = core.dgdecodenv.DGSource (r'c:\jobs\recall.dgi')
vid = core.std.CropRel (clip=vid, left=252, right=252, top=142, bottom=142)
vid.set_output(alt_output=1)
For some reason TVAI feels the need to upscale this to 4x it’s size (3184 / 4 = 796). Where, pray-tell, did I ask for such a monsterous upsize?! Cropping has always been horrendously broken like this!
/rant
1 Like
Between iris+proteus and proteus+iris do have any difference?
I mean enh1+enh2 vs 2+1… ???
From what I remember, I tried both combinations and I seemed to get better results with Iris + Proteus. That’s why, when I use this combination, it’s the order I have them.
I cant reproduce this, think it’s a decoder issue on avi file or an issue originating from vapoursynth
I never use these tools, i always script the things in bash (WSL)
O, I tried it without VapaurSynth first. I only used the AVFS mount so as to pre-crop. Before that, TVAI straight-out qudruples the output:
Stream #0:0: Video: hevc (Main 10) (hvc1 / 0x31637668), p010le(progressive), 7680x3184 [SAR 1:1 DAR 480:199], q=2-31, 23.98 fps, 24k tbn
Sigh. Okay, so here is the direct command line, as produced by TVAI:
ffmpeg "-hide_banner" "-i" "F:/jobs/recall.mkv" "-sws_flags" "spline+accurate_rnd+full_chroma_int" "-filter_complex" "crop=w=1920:h=796:x=0:y=142,tvai_up=model=prob-4:scale=0:w=3840:h=2160:preblur=-0.103699:noise=0.11:details=0.129344:halo=0.0276502:blur=0.17:compression=0.23:blend=0.2:device=0:vram=1:instances=1,scale=w=3840:h=2160:flags=lanczos:threads=0:force_original_aspect_ratio=increase,crop=3840:2160" "-level" "3" "-c:v" "ffv1" "-pix_fmt" "yuv422p10le" "-slices" "4" "-slicecrc" "1" "-g" "1" "-an" "-map_metadata" "0" "-map_metadata:s:v" "0:s:v" "-movflags" "frag_keyframe+empty_moov+delay_moov+use_metadata_tags+write_colr" "-bf" "0" "-metadata" "videoai=Enhanced using prob-4; mode: manual; revert compression at 23; recover details at 12.934399999999998; sharpen at 17; reduce noise at 11; dehalo at 2.76502; anti-alias/deblur at -10.3699; focus fix Off; and recover original detail at 20. Changed resolution to 3840x2160" "H:\veai/recall_883117904.mov"
If you notice, they deploy an extra correction trick, after instances=1
instances=1,scale=w=3840:h=2160:flags=lanczos:threads=0:force_original_aspect_ratio=increase,crop=3840:2160
So, no problem then you think, right? No, because TVAI performs the initial x4 upscaling, internally, after all: the extra command just masks it. (You can tell by the same drop in fps).
I am not the first one to notice a significant drop in fps on pre-cropping. It’s only now, that I’ve been using the command line since recently, that I saw TVAI ‘admit’ it does the 4x upscaling first.
But thanks for trying to reproduce this, and testing it.
Still i haven’t seen this, i never use crop. I do however remember the proteus upscale model by default first upscales to 4x. I previously expected proteus would by default output 4x and the last scale step was to resize to target resolution. Only last year i noticed that last scale step is not required to get to the target resolution. Obviously you now discovered this behavior is not valid for cropped input and proteus defaults to 4x again.
1 Like
Why do you feed the original input frame back into the output result?
1 Like
You tell me! LOL. That is the command line TVAI always outputs. But yeah, that’s obviously no good.
This is normal:
import vapoursynth as vs
core = vs.core
vid = core.dgdecodenv.DGSource (r'c:\jobs\recall.dgi')
vid = core.std.CropRel (clip=vid, left=252, right=252, top=142, bottom=142)
vid = core.resize.Spline64(vid, width=3840, height=2160, format = vs.YUV420P10)
vid.set_output(alt_output=1)
Regardles of the VapourSynth preprocessing, you get the idea: crop appropriately, then upscale to 3840x2160 dimensions. Now let’s look at what TVAI does:
ffmpeg “-hide_banner” “-hwaccel” “cuda” “-i” “F:/jobs/recall.mkv” “-sws_flags” “spline+accurate_rnd+full_chroma_int” “-filter_complex” "crop=w=1920:h=796:x=0:y=142,tvai_up=model=prob-4:scale=0:w=3840:h=2160
filter_complex is what ffmpeg does: the pre-crop. Then tvai_up apparently first does does an upscale of the 1920x976 source to 5664x3184 (x4), essentially completely ignoring its own initial scale=0:w=3840:h=2160 directive!
Stream #0:0: Video: hevc (Main 10) (hvc1 / 0x31637668), p010le(progressive), **5664x3184** , q=2-31, 23.98 fps, 24k tbn
Only the second scale command ‘fixes’ the original instruction to do 3840x2160:
,scale=w=3840:h=2160:flags=lanczos:threads=0:force_original_aspect_ratio=increase,crop=3840:2160
Instead, it should obey its own initial scale=0:w=3840:h=2160 directive!
See, it looks like Proteus v4 first does the x4 upscaling (5664x3184), works on that, then ignores its own initial scale=0:w=3840:h=2160 directive! (And then adds a later commend to correct for the mistake, as it were).
Anyway, going to try some hardware accelerated pre-crop going, and hope Proteus v4 won’t go into x4 mode.
So Proteus does 4x upscale and then down to user set resolution, always? If it’s the case, Proteus does was Rhea does, but this makes no sense, Proteus is not 4x optimized like Rhea, or maybe it is, but they don’t tell us.
