Video Enhance AI v1.5.1

Thanks.

I was in the middle of a batch and didn’t have the time to re-launch the program.

I see it is working as expected now :smiley:

I should mentioned that I would deinterlace my video using handbrake or ripbot264 and I made sure that the framerate was 23.976 before I processed the video. I’m going to try 1.5.1 again overnight using the deinterlaced video from Hybrid and I’ll update.

you seriously might be the most helpful responder on a forum i’ve ever seen. Thanks so much for such detailed write ups! Hopefully this works.

edit: so far it still has the jittering issue. One thing I should say is MediaInfo shows me that my copy that has the jittering runs at 23.976 (23976/1000) while another version of the show I have runs just fine with no jitter at 23.976 (24000/1000) , Could this be one thing causing the issue?

I just had a suggestion! A paper has recently come to my attention, titled TecoGAN which is geared towards video Super Resolution which maintains fluid motion and uses moving pixels as a context to add further data. I believe that this model, or an adaptation of it, would be graciously at home in the VEAI interface.

1 Like

big files are fine for me as long as quality is good :stuck_out_tongue:
although they should add configurable QP or bitrate settings

Discovered today that VEAI ignores the metadata item “Rotation” when it is present, so a cellphone video with that set to 90 when the conversion was played back did not rotate. Simple enough to use ffmpeg to write the metadata value, but I thought I would point it out.

Just updating that I’m still having the audio syncing issue even after trying deinterlacing with Hybrid. I should mention that I ran the same file using 1.4.2 and I’m able to sync the audio without issue.

did you accidently use a variable frame rate?

I followed the directions posted by Chase above.

I didn’t help much if it didn’t solve your issue. :stuck_out_tongue:

So you have two files, one with the issue, one without… I guess I would ask, what created each file? Does the original have the issue? Are they all mp4 files?

24000/1001 is typically the more “correct” way to specify 23.976. If you have .mp4 files, you can try putting the one with the jitter through MKVToolNix, and specify the bitstream timing (I usually do the resolution as well):

2 Likes

I second that

No dice :confused:
The non jittery file came from a dvd source I believe (low quality, never got a proper western release), and the jittery file (which is much higher in clarity) came from a fan project online; that one is in mkv format. Do you think realistically this kind of thing is fixable? I’m quite new to upscaling/dvd encoding things like this, but everyday im looking for ways to fix this.

These settings didn’t seem to fix it in MKVToolNix either - It’s really a shame, It’s an old anime that never got a proper release, and most official DVD sources online and the one I have are poor quality, but I found this strange copy online that has terrific quality, but has the jittering problem.

Edit: here are two gifs, this one is my previous upscale attempt, that used the source with no jitter and then this is the current source file with better clarity, with jitter

Regardless I’m sure all of us really appreciate your help Chase =)

That doesn’t look like a jitter caused by framerate “problems” (assuming we are looking at the same bits of the image). It looks like a temporal artifact in the image itself, caused by how the source was originally processed. The “no jitter” version has it too but less noticeable due to the reduced sharpness.

If it means anything, after a ton of time and many chinese forums, I found the encoding settings used in the so called “jittery” version.

Format                      : Matroska
Format version              : Version 4
File size                   : 264 MiB
Duration                    : 22 min 48 s
Overall bit rate            : 1 615 kb/s
Encoded date                : UTC 2019-11-01 07:58:49
Writing application         : mkvmerge v33.1.0 ('Primrose') 64-bit
Writing library             : libebml v1.3.7 + libmatroska v1.5.0

Video
ID                          : 1
Format                      : HEVC
Format/Info                 : High Efficiency Video Coding
Format profile              : @L4@Main
Codec ID                    : V_MPEGH/ISO/HEVC
Duration                    : 22 min 48 s
Bit rate                    : 1 165 kb/s
Width                       : 1 304 pixels
Height                      : 720 pixels
Display aspect ratio        : 16:9
Frame rate mode             : Constant
Frame rate                  : 23.976 (24000/1001) FPS
Chroma subsampling          : 4:4:4
Bit depth                   : 10 bits
Bits/(Pixel*Frame)          : 0.052
Stream size                 : 190 MiB (72%)
Writing library             : x265 2.9+8-27d8424c799d:[Windows][MSVC 1900][64 bit] 10bit
Encoding settings           : cpuid=1111039 / frame-threads=4 / numa-pools=16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=3 / input-res=1304x720 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=6 / crqpoffs=6 / rc=crf / crf=16.5 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=10240 / vbv-bufsize=4800 / vbv-init=0.9 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=31 / qpmin=0 / no-const-vbv / sar=0 / overscan=0 / videoformat=5 / range=0 / colorprim=1 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei
Default                     : Yes
Forced                      : No
Color range                 : Limited
Color primaries             : BT.709

Audio
ID                          : 2
Format                      : AC-3
Format/Info                 : Audio Coding 3
Commercial name             : Dolby Digital
Codec ID                    : A_AC3
Duration                    : 22 min 48 s
Bit rate mode               : Constant
Bit rate                    : 448 kb/s
Channel(s)                  : 2 channels
Channel layout              : L R
Sampling rate               : 48.0 kHz
Frame rate                  : 31.250 FPS (1536 SPF)
Bit depth                   : 16 bits
Compression mode            : Lossy
Stream size                 : 73.1 MiB (28%)
Service kind                : Complete Main
Default                     : Yes
Forced                      : No

Encoding settings           : cpuid=1111039 / frame-threads=4 / numa-pools=16 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=3 / input-res=1304x720 / interlace=0 / total-frames=0 / level-idc=0 / high-tier=1 / uhd-bd=0 / ref=4 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / open-gop / min-keyint=23 / keyint=250 / gop-lookahead=0 / bframes=4 / b-adapt=2 / b-pyramid / bframe-bias=0 / rc-lookahead=25 / lookahead-slices=4 / scenecut=40 / radl=0 / no-intra-refresh / ctu=64 / min-cu-size=8 / rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=2 / dynamic-rd=0.00 / no-ssim-rd / signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=3 / limit-refs=3 / limit-modes / me=3 / subme=3 / merange=57 / temporal-mvp / weightp / no-weightb / no-analyze-src-pics / deblock=0:0 / sao / no-sao-non-deblock / rd=4 / no-early-skip / rskip / no-fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / no-splitrd-skip / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=1.00 / no-rd-refine / no-lossless / cbqpoffs=6 / crqpoffs=6 / rc=crf / crf=16.5 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=10240 / vbv-bufsize=4800 / vbv-init=0.9 / crf-max=0.0 / crf-min=0.0 / ipratio=1.40 / pbratio=1.30 / aq-mode=1 / aq-strength=1.00 / cutree / zone-count=0 / no-strict-cbr / qg-size=32 / no-rc-grain / qpmax=31 / qpmin=0 / no-const-vbv / sar=0 / overscan=0 / videoformat=5 / range=0 / colorprim=1 / transfer=2 / colormatrix=2 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / no-opt-qp-pps / no-opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / no-idr-recovery-sei / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 / no-lowpass-dct / refine-mv-type=0 / copy-pic=1 / max-ausize-factor=1.0 / no-dynamic-refine / no-single-sei

Try running the “jittery” version through the QTGMC deinterlacer, with these settings:

Sometimes, even if the source is progressive, it can clean things up.

After changing Source Matching (number 6), click the preview button (number 7) and seek some frames with the jitter. You can toggle the check box at number 2 to turn the QTGMC deinterlacer off and on, to compare before and after frames in the previewer.

If it doesn’t seem to be doing anything, try changing the Input Type (number 8) setting to 2, and see if that’s any better. Also try setting 3 here. Choose the Input Type with the best results (0, 2, or 3). You can also try changing the Input Scan Type (number 2) to “bottom field first” and observe the results.

I actually did have a cartoon with similar issues that was “helped” with QTGMC at the above settings, with the Input Type (number 8) set to 0. It wasn’t completely corrected, but when scaled with Gaia-CG, Video Enhance AI did some further anti-aliasing that improved the result quite a bit. But it didn’t fix everything 100%. It was the best I could get it.

I also had a live action DVD that was progressive scan, but came from an interlaced source, and it still had some visible artifacting left over from however they deinterlaced it. This was also fixed with QTGMC with the Input Type (number 8) set to 0, but for live action, I also set the “Lossless” setting (underneath Source Matching (number 6)) to “before resharpening”. In that case, everything related to interlacing issues were fixed near 100%. (I also had to run the Derainbow filter (found on the AviSynth tab then the Denoise tab) with the Threshold set to 5.)

How difficult.
Unless i’m doing something wrong, nothing seemed to help it. Once again appreciate the big write up; I figure a last ditch effort (or something I should have done before) was upload a sample file, maybe it would be easier to come to a solution that way? This is a small cut from the file of just a scene with jittering in it.

P.S. even if we don’t come to a solution, I hope you have something like a kofi so I can pay you for putting in so much effort :smiley:

At this point, I think I’m just going to wait for the next one or two releases and try again. I’m ok using 1.4.2 for now.

I think they should also add a YUV/RGB option, since back when people would rather encode a png sequence, they would often do so in RGB to retain the often beautifully recovered chroma.

OK, after looking at the sample, I can see that’s just how the frames are, unfortunately. If this came from a DVD, then it should be 720x480 for NTSC. DVD standards don’t allow 1304x720. I can see from searching on Google that it’s supposed to be 480. It was probably either deinterlaced or detelecined incorrectly from the source, then cropped (black bars removed) and scaled to 720p. Or, it’s possible that the “jitter” is simply is what it’s like on the DVD (unlikely but possible).

If it were me, I would seek out the original DVD to purchase. I know that for this particular series, that is going to be expensive, as the DVD’s are now rare.

Thanks for checking the sample, a shame that’s how it is. This leads me to one question; If I got a hold of the DVD and applied the same encoding parameters they did (I posted them above) do you think I would reach the same level of clarity as their release?

The release of this upscale was around 2019, could they have used similiar upscaling programs?
(waifu2x, video2x etc) or was it done entirely through the encoding settings?