Input 1560x1080, output 3120x2160.
Latest version 3.0.11 installed.
When I work with stabilization, artifacts are added to the finished movie.
These are dazzling appearances, fast rotations, foggy effects in sporadic places.
If you add rolling shutter corrections and/or jittery, it gets worse. Arms/legs disappear during fast movements.
The stabilization is therefore not applicable in my test (video of 18 min.).
I hope you can still improve something.
The program is getting better every time. I donāt know any other team that works so quickly on improvements and releases patches/updates as fast as you do. Great!
1 Like
Iām brand new to TVAI and this is the first version Iāve ever used. I was really impressed with what I saw in the trial version and decided to pick it up during the recent sale. As Iāve processed a few different videos now with various models, I was wondering about something in particular. I notice that when TVAI isnāt the active window in Windows, the fps seems to tank significantly compared to when it is. Given how long videos can run, I would think many people are running TVAI in the background while they work on other things, but if that is truly causing the fps to further slow down an already slow process, I wonder if that can be fixed somehow? I am using a 4090 coupled with an i9 12900k with 32GB of RAM FWIW. Really impressive product though!
1 Like
On my Mac Studio (base spec. 32 GB, 24-core GPU), I can already get faster than realtime upscaling from SD (480/576 25fps) to 1080 HD using Artemis. The caveats:
- I need to split up a single source video into frame-accurate segments. I use Handbrake for that as well as deinterlacing / decombing
- I need to run 6 parallel upscaling processes (4 isnāt enough on my machine but might suffice with higher spec)
- I then need to join up the upscaled segments afterwards
Iām currently working on some scripts to use the HandBrake and ffmpeg CLIs to handle all of that, finally outputting to a WebM file so I can start watching within a few seconds while itās being written. So there will be 8 processes running in total - 1 x Handbrake (to start with but will finish fairly quickly) and 7 x ffmpeg. The HandBrake output segments will be distributed among the 6 upscaling ffmpeg processes and the 7th ffmpeg process will concatenate the upscaled segments and add the original source audio, outputting to a WebM (or MKV) file for āstreamingā.
The thing is, I canāt see why Topaz couldnāt easily achieve the same result, in a much more elegant way, if they wanted toā¦
Thanks.
Andy
3 Likes
The problem in a nutshell is that there is no magic bullet for deinterlacing. Even the best scripts in programs like Avisynth require that you have knowledge of the content of the file you are working with, because how it is ādeinterlacedā will vary from file to file.
It is nice when you have a straight, easy to deinterlace file that has no variable framerate issues and a is just straight deinterlacing, but if you have a mixed frame footage, which a lot of DVDās from the 90ās people try to upscale have, you have to make decisions on how to handle the changes in framerate that are buried in the telecine.
Do you drop a frame from the 29.97 sections and make it look more jerky so it plays back at 23.976? Do you motion interpolate the 23.976 sections to increase their framerate back to 29.97? What do you do when you just so happen to have genuine interlacing mixed in with the variable frame rate and telecining?
Add in Anime VFR and most people throw in the towel. There was even a script developed specifically for how painful Anime was, though its hard to find now and most people just use combinations of QTGMC and TFM/TDecimate scripts.
You can also if you choose find the common denominator and make everything effectively 120fps. This is the lowest number where both framerates coincide to avoid any dropped frames.
Currently I am IVTCing back to a 59.94 to have a general smooth playback throughout regardless of changing sections, but doing so required an extremely custom script that took some of the most experienced users on the Doom9 forums over 20x pages of forum threads to work out - and there was still disagreement between users as to which way to handle it.
I like it - but it increases the number of frames for upscaling drastically so slows down the upscale process a lot.
At the end of the day, Topaz can handle straight deinterlacing with no complications, but if you plug in anything else and donāt pre-process it, you will always have issues in Topaz, which can include strange framerate problems, interlacing artifacts not being removed, and sound de-sync to name a few.
To give you an idea, this is my current IVTC script I am using on the DVDās presently:
A=Tfm(field=1,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
B=Tfm(field=0,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=A,d2v="D:\Video\NTSC Season 4\Disk 6\Episode 23\VTS_03_1.d2v",flags=1,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=B,d2v="D:\Video\NTSC Season 4\Disk 6\Episode 23\VTS_03_1.d2v",flags=1,micmatching=0)
interleave(C,D)
But the first challenge is always identifying what type of content you have as things like Mediainfo just donāt help enough. Also, Joel Hruska did several DS9 posts on the challenges of upscaling the VFR in those DVDās and gave several different options to pre-process which you can check out here, aptly titled āVariable Frame Rate DVDās can burn in hellā
As an extract, if not wanting to read it all:
Star Trek: The Next Generation, Deep Space Nine, and Voyager werenāt recorded in anything as sensible as a steady 23.976fps. When these shows aired in the 1980s and 1990s, they were constructed from at least two different types of content: 23.976fps progressive content and 29.97fps interlaced content. Episodes of these shows do not run at one frame rate; they run at different frame rates depending on whatās happening on-screen. A show that runs at one frame rate, start to finish, uses Constant Frame Rate encoding (CFR). Deep Space Nine, TNG, and VOY all use VFR. Stargate SG-1 and Babylon 5 do the same thing. Most of the best nerd content of the late 1990s and early 2000s is locked behind media that go for this kind of manipulation.
Topaz Video Enhance AI is not the only application that gets confused by VFR content. Iāve been using StaxRip as a front-end for AviSynth+ and have been mystified as to why the program constantly detects a frame rate of 24.66fps when loaded with MakeMKV source. It turns out that 24.66fps is the average frame rate over the entire episode if you average together the relatively small number of 29.97fps scenes with the much larger number of 23.976fps scenes.
1 Like
The ā.ā still works for me.
Very smart idea! It may be a solution for the future. Every pre touch tho with encoding may steal you from that final touch. You can try 4-sec clips and do a blind test for your self after Topaz. I often use losslesscut, it may work for this aswell.
I think this is the Nvidia driver, so that nothing jerks while you do something else it brakes compute. Or was it windows? I do not know exactly.
Genius!
Try Resloves AI deinterlance, it was the best iāve used.
It did even remove deinterlance from a PAL extreme compressed video.
So that i was able to make a good video out of it after deblocking with TVAI.
2 Likes
when I try to render a video whether it be a preview or an export. the video will hang for about 5 to 10 seconds at the begging of the video after it has rendered.
Interresting! Can you drop VOB files in there? Davinci 16 donāt like them.
It probably depends on if the VOB has encryption or not. Just guessing here, but the fact that you have a VOB file at all probably means it was copied off a DVD. Most DVDs have encryption that stays on such files, but some donāt.
Yeah, resolveās AI tools are not always the headline features of the Resolve since its so much more, but some of their AI tools are not only really good, but they keep adding them to do all kinds of little things. There is one for upscaling, which is not on the Topaz level, but its good enough for some footage and better than some of the standard upscaling interpolation. They have one for adding more frames to the footage as well, they have one for masking and depth masks, they have tools now with AI for audio that does amazing job etc. I wonder if it would be possible to add Topaz Video AI as a plug in that integrates with Resolve or Premier pro or After Effects etc. Not sure what would be best way to do it, but maybe something like we have seen Mocha from Boris FX or their Silhouette Paint program being added.
2 Likes
Interesting information. Didnāt realize that. I donāt work with that kind of TV show content myself, but it sounds challenging to even prepare videos for Topaz. Interesting and potentially useful info for me. Thanks.
VOB is a container. Its still not able to read them.
I did export the video with vlc into h264.
I was exporting to AV1. Maybe thatās the issue? Iāll have to try going back to H264.
Yes, and a TVAI OFX plugin that would encompass quite a few additional video editors too. The Boris FX model of standalone and plugin versions may benefit Topaz too if they do it, I hope this will appear in a future update of their timeline.
After all, they already do this for some of their photo apps so why not for TVAIā¦ once itās debugged of course.
2 Likes
Good point. Yes. I would love to be able to do some things without having to export and import videos and have a more streamlined workflow, akin to Boris FX or even what Topaz is doing with their photo apps. Yes. I forgot they do that with photo side of their AI apps.
