Can someone explain v1.7.1 and its massive CPU use? It has started using huge amounts of CPU resources, through ntoskrnl.exe. This is a huge increase over v.1.7.0, I’m talking 50% CPU usage in v.1.7.1 compared to 5% in v1.7.0
The 3070 can only go faster if the rest of the system can keep up with it. With gpu utilisation going up and down it seems like the gpu is waiting for new data to process.
What file type are you outputting (mp4, mov, jpg, png or tif)? Encoding speed to h264 might be a limiting factor (I think VEAI does that on the cpu, but I’m not sure). Writing speed to disk might be an issue (especially with an image sequence output to the same disk that the source is on). You might find that another file type can be output quicker. These are things you can test.
There are a lot of different ways to deinterlace, and you will definitely see a difference between dropping half frames and blending. Neither way is guaranteed to be the best way for all sources, so you need to experiment. QTGMC has a lot of options, and so does YADIF. I find that with almost all video things there is no one-size-fits-all.
Understood, I will have to do some test with both methods, but I think I’ll keep working with half frames because result doesn’t seem to be bad.
Thank you very much for your help!
I understand that the new gpu can only work as fast as the rest of the system. But I just wouldn’t have thought a “highend” CPU from last year and a SSD with 2500MB/s + IOPS from 450k would be such a bottleneck.
I’m outputting to png and I will try another file type. Thanks for the tip 
I,m getting those odd horizontal lines too with Artemis LQ or HQ and only when enhancing the source. The only way that I found to minimize the problem is to do a small upscale… Don’t know if we can run multiple versions on the same machine at the same time…
https://ibb.co/L0RqYDW
Thank you for the response. What is the small upscale you do to try and minimize the problem? like 125% or even smaller?
You’re welcome 
122% it’s enough to minimize the problem… at least in my case.
1920x1080 122% upscale to 2342x1318…
1920x1080-Upscaled-to-2342x1318 hosted at ImgBB — ImgBB
As you can see the problem it’s still there but not so strong.
I’m seeing about 80% CPU usage on a Xeon quad core 3.6Ghz CPU when processing. I think most of that is being used to compress the .pngs.
On my new box for testing 1.7.1, I scaled the CPU’s power to match what may may be in the box eventually, a RTX3080 or AMD RX6800XT, and bought a i7-9700k 8 cores at 4.9GHz (only $259 at Newegg). I may have to get a 12 or 16 core CPU if that bottlenecks. I made sure I can scale the LAN too. On Newegg, there’s a USB C to 5Gb Ethernet adapter as well as SPF+ 10Gb E PCIe cards. I plan to have a hybrid copper/fiber 2.5Gb E to 10Gb E LAN at some point, using a switch with SPF+ and Gb E ports (Newegg has one with 18 ports). I’m seeing about 15% utilization now, so may be ok with another 4x faster box, but a third one and I’ll for sure have to get that switch and 10Gb E for the server.
It sounds like you are reading/writing on a local drive, correct? Is it the same drive Windows 10 is on? Ideally you have a different drive for you videos. What CPU do you have, like how many cores and what GHz? Outputing to .tiff should cut your CPU load. To .mp4 would max it out unless you have like 16 cores.
That’s messed up. You can’t miss seeing those. I’ll be testing 1.7.1 soon.
Developers, if you read this, please check out the the image:
blocks at 100% upscale
1 Like
Voyager has been like a quest for the Holy Grail. I will sometimes get to a point I am happy with it, and then I will see someone else do something better and I start again.
To avoid a really long post, I will try to be brief. I started with the PAL DVD sources and spent weeks trying to deinterlace them to the point I was happy. I even had an ok script that was really close - but I then saw other peoples examples online that were better than I could get and that made me start again.
The NTSC DVD source has all the filmed segments in24P telecined to 30P, not interlaced. This means that the vast majority of scenes, particularly with just people in them (which is a lot) has no deinterlacing artifacts or conversion artifacts in changing it to the PAL 25. So far as I have been able to tell, the NTSC source is imply a lot better suited for the upscale and beats everything I have tried from PAL so far.
This is a clip from current “best” process - it is still not perfect in my eyes, but i may be really critical at the moment in search of “perfection” so am always open to changes or alternative processes.
If you want the full process and scripts for this (if it helps) I can post them, but it is not the most user friendly process and I currently finish with Premiere so it is not all free either. Let me know how this compares to what you are doing.
You do understand that a 1.22x upscale with VEAI does a 2x upscale, then does a standard downscale to the final? If you missed it, VEAI only does 1x, 2x, and 4x, with the breakpoints at 1.2x and 2.4x
Anything that is not 1, 2 or 4 that has problems you can’t tell whether the AI is at fault or not.
1 Like
Look at the : indicating ratio and flops as well. From TechPowerUp
2060
FP16 (half) performance - 12.90 TFLOPS (2:1)
FP32 (float) performance - 6.451 TFLOPS
1080ti
FP16 (half) performance – 177.2 GFLOPS (1:64)
FP32 (float) performance - 11.34 TFLOPS
The 2060 (and 20 series) gets 2 FP16 per core/clock, while the 1080ti gets it’s FP16 cut by 64 times it’s normal per core/clock. That’s artificial and done to segment by Nvidia. As FP32 cores doing FP16 should be at least 1 for 1.
@virtutis
I’ve got a question: if I understood correctly from your post Star Trek Voyager on NTSC DVD isn’t interlaced, right? So it’s normal to have about 66000 frames to be processed and not to have them doubled right?
I started working on the NTSC source too and I’ve got good results now, (seems that drive starts it at 720p so I suggest you to download it and then watch):
https://drive.google.com/file/d/1c7b_y9v3GfTYr4xN9lsArMu3JBqxgS0m/view?usp=sharing
What do you think about those results?
In your clip I noticed there’s the same issue I’m having: when people are far from the camera they look blurry and faces aren’t perfect, I already tried sharpening a bit but without results, your clip is still impressive considering the quality of earlier seasons.
If you want to share your full process and scripts I would be very happy because I can find other useful information, if you want to know how I’m working now feel free to ask!
Thank you very much!
Ah ok.
Faces and such that are further away are always going to be an issue. The main reason is that the original image file looks like this in one of those scenes:
The number of pixels is just getting too thin so the smallest changes in scripts and adjustments pre-processing can have wild changes in the end result.
Its actually doing an ok job on the current extract, which looks like this, but it is struggling on the eyes:
If I use an alternative script, and process the same image, I get this:
Which is better around the eyes, but everything else becomes more noise as well. As I mentioned, it usually comes down to running script after script and adjusting things and then deciding what looks best - and for me, I am still looking. It is like a very long rabbit hole that never ends.
The process at the moment is essentially this:
- Extract using DVD Decrypter to VOB
- DGIndex the file to D2V with Honour Pull Down Flags
- Run this script in AvsPMOD64 with Avisynth64 (note the final cropping is optional and based on the areas of no image per extract)
D2VSource("D:\Video\Voyager\S01E02 Demuxed\S01E02.d2v")
AssumeTFF()
TFM(D2V="D:\Video\Voyager\S01E02 Demuxed\S01E02.d2v")
TDecimate()
QTGMC(InputType=1, Preset="Very Slow", NoiseProcess=1, NoiseRestore=0.5, STABILIZENOISE = TRUE)
Vinverse(sstr=2.7, amnt=255, uv=3, scl=0.25)
ChubbyRain2(th=10, radius=10, show=false, sft=10, interlaced=false)
BilinearResize(640,480)
Crop(4,0,628,480)
- Open the script in VirtualDub64 and extract as Image Sequence
- Import as Image Sequence into Both VEAI 1.7.1 and 1.6.1 (or a similar pre release version)
- Process CG shots in the older VEAI, process the new in Gaia HQ in 1.7.1 (same folder). Process into PNG Lossless files
- Import the image sequence once complete into Premiere. Interpret footage as 23.976 FPS.
- Import the sound file that was demuxed in step 2
- Combine two sets of files and export media (I am using H265 atm, makes a 12GB File).
As a note, when I lick on that link it requires access to the file so I haven’t been able to check your link.
2 Likes
@virtutis
Sorry, please use that link instead:
I suggest you to firstly download it, what do you think about my results?
Looking at your code I understand that the source is already progressive and not interlaced as the PAL version, right?
About your workflow I’ll definitly try it because it’s really interesting, only one thing: I didn’t understand the “process CG shots”, are you processing CG parts in 1.6.1 and recorded parts in 1.7.1?
Thank you very much!
My Ryzen 3700 has also a usage of 80-90% and it has 4.40GHz with 8 cores/16 threads. But as you guessed correctly on the SSD was also Win10. I just installed another M2 SSD and now I get much better results. I would never have thought that those new SDD could struggle with anything. Reminds me on the days where you couldn’t record your raw gameplay in 60fps because the hard drives were to slow.
Thanks everybody for the help
NTSC version of Voyager, like a few Sci Fi shows, actually have a hybrid or variable frame rate. The reason is that the filmed parts (the parts not primarily CG) were shot as actual film (24p) but then telecined to 30p to match NTSC Format. Note I am abreviating here in terms of actual Frame Rate, as for NTSC we are talking 23.976 and 29.97).
So when you look through those shots frame by frame you will see the 3:2 pattern of 3x progressive frames and 2x interlaced ones. However, this is a soft Telecine at least on my DVD’s so far, and it is really easy to recover the original progressive framed content and export as 23.976.
The problem is that the CG shots were not shot in 24p and then telecined they are 30i or 29.97 interlaced. So Voyager episodes are made up of varying types of content. If it was all filmed/telecined content it would be dead easy to IVTC, but it gets a bit more complicated when the footage varies throughout.
I am still experimenting with ways to “fix” this. My method above is using the D2V file fed into the IVTC script to assist in exactly matching where it needs to be fixed and then running a deinterlace cleanup filter to remove any after effects from interlacing. So far it has worked ok, but up until a few weeks ago I was working on PAL DVD witha nice constant 25i and all made up really of only 25i.
This line:
TFM(D2V="D:\Video\Voyager\S01E02 Demuxed\S01E02.d2v")
is De-telecining the input based on the full index from the D2V file (instead of the IVTC looking itself at the footage and trying to guess what each is). The decimate is removing any duplicate frames.
The end result at the moment is Progressive 23.976 footage.
One of the downsides of reducing everything down to 23.976, which the above does, is that in both formats of the show you are watching in either 30 telecined or 25i which is closer to 50 frames per second (albeit I see a lot of second frames with little motion) so stripping back to straight 24p does have the potential to be less smooth on the eyes.
Josh did quite a bit of work on DS9 with the same issue and he had a long discussion - along with many ways he was experimenting with fixing the frame rate issues - but I don’t think even he had a foolproof method and mostly no one does as its an absolute pain the ass.
In relation to your clip, it looks very good as well. There are some aspects on the close ups and image generally that I still don’t think I am matching, but I have also suspected that the later seasons may have a bit better quality. I only have season 1 in NTSC and I haven’t actually tried later seasons on PAL as I started at the beginning. I really need to try some later episodes and see if it makes a difference.
I can see what you mean in relation to the faces as they go away from camera. It is the same issue I have and I am still experimenting. Can I also ask was this from a PAL source or NTSC? If it is PAL I may pull out my season 5 discs and have a look and see how my process works on that same clip.
For the CG, I found (so far) that processing the actual CG in Voyager often produced a better output using the Gaia CG option. However, in 1.7.1 it is still not working properly, so the clip I posted used both 1.7.1 in Gaia HQ and 1.6.1 in Gaia CG. Frame start and end time were selected (manually) telling VEAI where to start and stop processing. Usually I try to do this in one window and I end up with a series of tasks along the bottom where it flips back and forth between HQ and CG but I can’t use CG in 1.7.1 atm, so I have to run the processes separately (which is irritating, but the only other option is to revert back to 1.6.1 for everything instead).
1.7.1 Gaia HQ seems to be processing a bit faster, albeit not by large margins, so have been using it for the moment for the filmed parts of the episode.
The reason I use Virtualdub to export the series as an image is so that the input is image based and the output is also image based. This is a blessing when it comes to both 1: having a process fail mid process and 2: stitching together separately processed footage into a single file.
I guarantee if you try to separate the outputs into a series of video files and then try to “put them back together” at the end, you will have a nightmare of a job. Exporting as an image sequence means that the frames are numbered, the frame numbers never changed and when you are done, you can just import or process the entire sequence in one go into a video file.
@virtutis Thank you very much for that detailed description which gave me important information.
Very useful the information about telecined source! For now I am converting to 23.976fps CFR, then after a light denoise I process with Gaia HQ (upscaling in FHD, not 4K) exporting as image sequence and then I convert to video increasing speed at 25 fps, to avoid modifing PAL audio speed with gave me some issues.
In the clip of my previous post I used NTSC source, which gives me far better results than PAL source. The following clip has been made with PAL DVDs, took from season 5:
Also you should consider that from season 5 quality is far better than earlier seasons, mostly for the blurriness reduction I think. Considering this I still find your results really impressive because you achieved a good quality for the second episode in my opinion.
Regarding the CGI shots I find that Gaia HQ enhances them very well, but it may depend on the source: considering the PAL source I think they would benefit from the stronger sharpener of Gaia CG, but moving to NTSC it seems that Gaia HQ does a good job on them also(for now I’m working on season 5 so about earlier seasons I don’t know from my experience).
