My Star Trek DVD Upscale Project

Greetings fellow tech nerds! For those of you also using the software to upscale Star Trek DVDs, I’ve written a blog post detailing, if a bit verbosely, the workflow I’ve come up with over the last year. I would be interested in hearing the approach others are taking. Live long and upscale! :vulcan_salute:

11 Likes

Nice. I’m doing something similar but am using a bunch of bash scripts to automate it. If everything works right I can do a single script to run the entire thing:

  1. call dgindex via CLI interface;
  2. take my avs script for ivtc and write it out to a temp file, adding the filenames (it’s really odd there’s absolutely no way to pull CLI env arguments into an avs script, maybe too OS specific?)
  3. call temporary avs script with ffmpeg
  4. call veai via CLI which as of yesterday’s release works great
  5. call ffmpeg to encode the image sequence to h.265, merging subtitles and audio back from the VOB

There is some tooling for defining project profiles; it’s a constant work in progress but is getting stable.

I’m interested in your 2-pass ivtc process. I am using a variation on what Joel Hruska described in his articles about his Deep Space Nine project, that converts everything to 24fps

TFM()
TDecimate(hybrid=1)
QTGMC2 = QTGMC(Preset=“Very Slow”, TR2=4, SourceMatch=3, Lossless=2, InputType=2, noiserestore=0.1, NoiseDeint=“Generate”, grainrestore=0.1, MatchEnhance=0.75, Sharpness=0.5, MatchPreset=“Very Slow”, MatchPreset2=“Very Slow”)
QTGMC3 = QTGMC(preset=“Very Slow”, TR2=4, SourceMatch=3, Lossless=2, InputType=3)
Repair(QTGMC2, QTGMC3, 9)
AddGrain(var=1.0)

It looks pretty good but i feel like it could be better, I am interested in trying your two-pass process. Are you saying that you leave the output as VFR? Is VEAI ok with that, how does that translate to PNG sequences and the subsequent encoding?

This is all for Voyager which is where i’m starting. I have a totally different ivtc process for Stargate SG-1 which is a lot simpler and looks way better (I guess the DVDs or the source are just better/better encoded), but I wonder if that too wouldn’t benefit from the two-pass technique. I do see occasional motion stutter, like dropped frames, but very inrequent.

That sounds like a great workflow! Much less pointing and clicking at the very least. I had occasionally seen mention of CLI and batch processing like this but I haven’t done any research into how it’s implemented.

The TLDR is, yes, I end up with a VFR .mkv. As far as the two pass process and VFR here’s what I’ve learned.

The first pass references the d2v file that DGIndex creates to analyze which sections are film and which are video. The second pass uses that analysis to reverse telecine and then outputs a timecode file for use later.

Instead of saving the second pass to an intermediary file I open the .avs directly into VEAI which will show a weird averaged frame rate. For the time being though the frame rate is inconsequential because AviSynth has output a timecode file which will set the correct frame rates later. AviSynth is still serving all the frames to VEAI. The only important thing is is that the input and output frame counts are the same. This is why I decided to export as an image sequence as it has no inherent frame rate. I found that exporting as mp4 or ProRes left me with files that had different frame counts than what was input.

After upscale the image sequence is encoded back to a video file at 29.97 as that’s the initial frame rate that the the timecode file assumes. Then I multiplex the video, audio, and the timecode file with MKVToolNix which results in a VFR mkv that plays the film and video sections at their respective native frame rates.

I am going to give this process a try next to see how it works out as comparison to what I have done so far. I have had a slightly different workflow - It also doesn’t help that I only have one season of Voyager in NTSC as I am in Australia and getting NTSC versions are really hard it seems. I have Season 1 at the moment and Seasons 2 and 3 arriving next week. Most of my upscales have been on the PAL versions as I have all 7x seasons.

With the NTSC one, I read a lot of posts on doom9 (and arguments) from the thread where Joel’s method was being described. There were several other scripts that were being thrown around as being “very good” for handling the VFR without having to re-encode it back to VFR. I say “very good” as it is always interpretation as to what looks good.

What I am currently experimenting with is this one:

video = Bifrost(D2VSource("D:\Video\NTSC\DEMUXED\VTS_03_1.d2v"))


A = Video.Tfm(mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
B = Video.Tfm(field=underpants,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
C = Video.Tfm(mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=underpants,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
QTGMC(InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)

BilinearResize(640, 480)

Which I save and open directly into TVEAI.

100% Pass as Dione Robust V3
225% Pass as Artemis Medium

It results in a 59.94fps file and is definitely larger (so takes longer) than 24/30fps, but it netted this test result which I thought looked pretty good: (download and watch it, watching it live may compress the stream)

Dropbox - Test.mp4 - Simplify your life

Note no sound in this test clip. I was, at the moment, processing the whole episode so when I am done, can go through the whole thing and see.

I also processed it using a variation script also from that same thread which worked out pretty good too but the filmed sections looked a little jerky - the above one smooths that out, albeit there are interpolated frames so it kind of ends up being personal opinion on which is better.

Once I have done these tests, I will see if I can follow your guide and look at the result as well to compare.

hola! dione robust is bad… :frowning: it transforms the images, and how does your video come out at 60 fps while dione robust deinterlaces without doubling the frames?

Dione Robust or Dione in general have so far achieved the best results in terms of upscaling Voyager from all the tests I have done so far. It just depends on what you feed into it, and your preferences on the output end.

It is not perfect - none of models are - but it is all a matter of taste as well. I tend to find that Dione can drastically increase the level of detail in the images, but it also causes artifacts on certain scenarios and if there is artifacts in the source, it tends to amplify them, so its not always the best in all situations. To give an example, these images are:

  • Top Left is Original
  • Top Right is Dione Upscaled
  • Bottom left is Artemis Medium Upscaled
  • Bottom right is Dione 100% then Artemis upscaled

You can’t beat the clarity of image when Dione becomes involved - BUT - you do get the issue that it may over process the image just a bit too far to the point where it becomes potentially unusable - hence my experimentation.

Dione Robust does not double frames, but the issues with the NTSC Voyager is that it is Hybrid footage with (typically) a combination of 23.976 progressive and 29.97 progressive footage. To make the 23.976 filmed sections match, these are telecined. The problem is if you IVTC the footage, you either to recreate the hybrid file at the end (which incidentally is what the OP did and hence I will test his script later), or make some kind of processing change to the footage to try to make it a constant frame rate.

This script will leave the 30P sections as they are and then interleave the remaining to reduce the jerkniness of missing frames in the 24P sections. Its not perfect and is one script I am experimenting with:

D2VSource("D:\Video\NTSC\DEMUXED\VTS_03_1.d2v")

Bifrost()

A=Tfm(field=1,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
B=Tfm(field=0,mode=0,slow=2,pp=2,mchroma=false,cthresh=-1,micmatching=0).converttorgb().generalconvolution(matrix = "0 -1 0 0 4 0 0 -1 0",divisor=2,auto=false).converttoyv12()
C=Tfm(field=1,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=A,d2v="D:\Video\NTSC\DEMUXED\VTS_03_1.d2v",flags=1,micmatching=0)
D=Tfm(field=0,mode=0,slow=2,mchroma=false,cthresh=-1,clip2=B,d2v="D:\Video\NTSC\DEMUXED\VTS_03_1.d2v",flags=1,micmatching=0)

interleave(C,D)

BilinearResize(640, 480)

This script interpolates between frames to fill in gaps to avoid the jerkiness:

video = Bifrost(D2VSource("D:\Video\NTSC\DEMUXED\VTS_03_1.d2v"))


A = Video.Tfm(mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
B = Video.Tfm(field=underpants,mode=0, PP=1, MI=20, Slow=2, CThresh=4, Micmatching=0).QTGMC().SelectEven()
C = Video.Tfm(mode=0, MI=20, Slow=2, CThresh=4, Clip2=A, Micmatching=0)
D = Video.Tfm(field=underpants,mode=0, MI=20, Slow=2, CThresh=4, Clip2=B, Micmatching=0)
Interleave(C,D)
QTGMC(InputType=1, ShutterBlur=3, ShutterAngleSrc=180, ShutterAngleOut=180, SBlurLimit=8)

BilinearResize(640, 480)

In both cases, to reduce the chance of jerkiness from bad IVTC, they both increase the framerate from the original 29.97 to 59.94. TVEAI isn’t increasing the frame rate from Dione, these scripts are.
Note I did not write any of these scripts - just read pages and pages of arguments where people far more experienced argued over their own interpretation of the “best way to deal with the hybrid footage” and in general, these two scripts tended to be the end result of two of those people.

They really didn’t agree with each other though, so hence I am testing for myself which I prefer.

yes indeed, it’s a big job that you do too! I understand better, for me, dione tv distorts the images of my sd sequences quite a bit, I just uploaded an original / modified comparison test. I only have the neatvideo plugin left to tweak

See at the moment, I am so focused on Star Trek - especially given the time it takes to run through everything - I just don’t have a lot of time left for much else with the program at the moment. That is to say, I have not tested very many different kinds of footage with the models yet so I don’t know how well Dione works for other kinds of footage.

I know, for instance, that Voyager is actually not bad at its core. There are issues yes, but the dvd quality is not atrocious and doing a Dione 100% and thats it (no upscale) produces a very watchable result.

But it does exacerbate a lot of artifacts in the footage. So the more artifacts the footage has, the worse Dione tends to be. Your test footage is in a lot worse shape than the Voyager originals. I am not sure the best method would be for that kind of footage as I have not really used it for that yet sorry.

yes the original sd is worse than original Voyager, because it’s old hi8 cassettes that have been digitized, with a lot of noise. But I finally managed to deinterlace perfectly with virtualdub and topaz, passing the original in hd with a good deinterlacing, neatvideo takes care of the rest :slight_smile: which ultimately gives me a more beautiful image than the original, with a little more detail. Only I have to go through 4 video renderings to get this result.

Well dbasthemer, first let me say how great it is to see that there are several (talented) people online that are trying to restore the last remaining Trek-Series (in this case Voyager and Deep Space Nine) that unfortunately won’t get a studio remaster.

Second, hats off to your results, they look pretty good.

I know from personal experience (I happen to have my own Star Trek project going that includes Topaz Software in an attempt to restore those shows) how difficult it is working from the commercially available DVDs (both NTSC and PAL).

I’d like to throw my two cents in here and let you know about what I have learned working on my own project.
To give you a little heads up: I am working with the PAL DVDs (I also worked with NTSC, but there are too many issues related to vfr mostly and trying to establish an “acceptable” stable frame rate - just my opinion) and have started with Star Trek - Deep Space Nine.

In general: The DVDs (both NTSC and PAL) are a b**** to work with/from (speaking quality-wise here, not content-related). That assessment of mine stems from all the problems that source material poses to anyone trying to work on it. DS9 and Voyager were (as far as I know) produced with analog cameras during their early seasons (I believe DS9 switched from analog to digital somehwere around season 4 and fully with season 5, while Voyager had the same happening during season 3 and went fully with season 4 onwards) - you can notice that when comparing the image quality from seasons 1-3 with season 4 onwards in both shows … to put it simply: The image became more focused/sharper, the starship/space scenes had more complex motion (due to the producers using vfx/cgi models instead of real static models) and the color representation was a bit more vibrant/looked less “washed out”.

For me, the (PAL-)DVDs presented the following challenges: Uneven/inconsistent picture size/dimensions (depending on how much cropping was required to remove black bars left/right), terrible combing/interlacing issues that probably resulted from the pulldown they did to handle the vfr (23,976 fps for live action and 29,97 fps for space scenes) to conform to a consistent 25 fps for the european market (which also resulted in a terrible audio pitch shift, making voices sound looney tooney), banding due to the limited 8bit color depth, block artefacts due to the MPEG compression of those days and ringing/haloing around edges, not to mention the bad rainbowing on starship structures (which many scifi-series of that era suffered from).

I guess it’s not the fault of the production companies back then, since technology in those days couldn’t handle anything better and compression algorithms weren’t as “good” as they are now.

But when the DS9 documentary “What We Left Behind” came out and after re-watching TOS and TNG on Bluray as well as the advancements in consumer-level-tech (including AI software) I thought: We can do better - not perfect of course, but better.

So here is what I came up with:

First I use StaxRip with a looooot (many) adjustments and filters to get the source material ready to be fed into Topaz VEAI. That includes:

  • Individual cropping (auto crop isn’t always enough, since sometimes there are lines left over like the ones we knew from old vhs-tapes on the top and/or bottom, so I do a little additional crop there)
  • Resizing the entire image (actually adjusting the size slightly to 720x540) - that happens because once I upscale to 4K, I want consistent measurements of 2880x2160p (which is what 4:3 would look like in 4K)
  • Using insane (!!!) presets (including lossless encoding) to make sure the quality loss is minimal (since a lot of filtering is required and more rendering passes are coming up, that’s important) - currently that includes using quality factor based encoding, 10bit color (better color representation) and 444-10-intra profile as well as full motion search
  • The filters include: Derainbowing, QTGMC Progressive Full Repair with a custom made script (I finally arrived at my current settings after 18 months of work, so you can guess how frustrating that was) which includes slight denoising and grain restoration, DeBlock, DeHalo, DeRinging, Anti Aliasing, Sharpening, Handling of Color Banding and finally Frame Rate adjustments towards a (mostly) stable 23,976 FPS - I also slowed down the audio to restore the original pitch and syncing that up later on
  • I then use a three-pass approach (!!!) with Topaz in the following order: Gaia HQ at 100% to get a little more detail out, Artemis LQ with upscale to 1080p (to handle remaining artefacts and get things a little sharper yet and finally Gaia CG to 4K as finishing touch in order to also have something done for the vfx/cgi scenes (which thankfully doesn’t hurt the live action shots).
  • The then produced uncompressed 16bit TIFF-images are being fed into Adobe Premiere Pro, where I use a unsharpening mask, a very small amount of sharpening and grain and a custom made LUT (Color Grading) to conform towards a color palette that was shown in the DS9 Documentary (colors are a bit cooler, less shifted toward the red spectrum).

Now, my version is certainly not perfect, I guess none of our versions will ever be, since every AI model comes with strengths and weaknesses and I have learned that with the source material we have available, it’s all about compromising on the different aspects of the series (live action, vfx/cgi, color representation, soft/sharp image, fluidity of motion and so on).
My version for example sometimes suffers from problems with artefacts and grid patterns during flashing lights and nature shots (which is caused by Artemis LQ mainly), but since they are not happening that much, I concluded I could accept that watching episodes, since it is more important to me that actors, faces and cgi/vfx looks good instead of the occasional planet side vista or explosion, which happens so quick that you would have to hit pause anyway to clearly notice it.

I would be willing to provide sample shots if anyone is interested, it could just take a little time, since I am still rendering out the final version of the DS9 pilot and my connection over here is not the fastest.
Anyone interested?

Sorry for the long post. Live long and prosper.

4 Likes

waha! this work! I don’t know staxrip, with all these filters but it must be impressive and especially long, can be longer than my renderings to me until the final result. I say well done, and it is true that I will be curious to see the result, when you can.

Thanks. Just to give you a sort of time frame: I can only judge from my work on the DS9 pilot (which is a double episode, resulting in about 90 minutes).

Please be advised that the rendering times below can vary, depending on whether I have other applications running.

StaxRip: 20-24 hours.
Topaz Run 1 (Gaia HQ 100%): 10-12 hours.
Topaz Run 2 (Gaia ALQ 1080pHD): 6 hours.
Topaz Run 3 (Gaia CG 4K): 48-56 hours.
Adobe Premiere Pro (Putting everything together): Easily 72-80 hours.

I know, it’s crazy, but that numbers/times result from me using a gaming laptop with a GPU for Topaz (RTX 2060) and CPU (Intel(R) Core™ i9-9900K CPU @ 3.60GHz with 32 GB RAM) for everything else, because I saw quality differences between using GPU and CPU in the various applications, so I stuck with what worked.

@virtutis I’m sure we’ve read some of the same doom9 threads. There are certainly some strong opinions out there. :joy:

That clip looks great!

I started a bit out of order. I first upscaled the last two seasons of DS9 as that’s where I was in a re-watch and just couldn’t go back to SD. Once the last two seasons were done I moved on to season 1 of Voyager so I’ve already encountered the quality difference between the early and late seasons.

I’m sorry I can’t offer any insight at upscaling or restoring PAL. TIVTC has worked rather well for me on the NTSC version. Someone else mentioned to me that they are using a newer printing of the NTSC DVDs than I have and are getting some slightly different frame counts than I am. In the end I opted for figuring out the VFR method for faster processing, truer motion for all sections, and smaller file sizes.

@CodeGrey This is great! And no need to apologize, my blog post was certainly not brief.

I’ve already encountered the less than stellar quality of the earlier seasons. I actually started with the last two season of Deep Space Nine as that’s where I was in a re-watch. Then starting the first season of Voyager was a rude awakening. :joy: The video is decidedly not centered in the DVD frame nor is it consistent throughout an episode. I had to weigh my desire for clean sides of the frame with the amount of work it would take to crop literally every shot. I chose a good enough method. It sounds like you have much more patience than I ever could! I applaud your efforts!

I have no experience with PAL so I’m sorry I can’t offer any insight on their conversion. I opted for a variable framerate approach with the NTSC DVDs for a lower frame count (ie faster processing), truer motion for both the VFX and live action, and smaller file sizes.

My method is a series of compromises and contractions but there are a couple of your suggestions that piqued my interest. I really wanted to be able to do at least 4 episodes a day so I have kept it to a single pass but I did notice A LOT of rainbow artifacts in the first season of Voyager. I will have to look into the derainbow filter you mentioned.

Again, great work!

1 Like

@CodeGrey

One of the things I realised early on was that if I sank days into each upscale, the process would be never ending. Your script sounds impressive and would definitely be interesting in seeing results as well.

The downside is that there is 172 episodes of Voyager (and I don’t think that separates all the double length episodes). At 80x hours per episode that’s 573 days of constant processing or about 1.5 years. It is of course personal opinion, but I couldn’t justify sinking 1.5 years of endless processing into the series. Maybe if a lot of people pooled their resources and say had 172 people do an episode each and upload it would be a lot more interesting - but that brings us into murky pirating territory.

So I have been focused on reducing the time spent per episode. I recently realised I can run two episodes at the same time in two Topaz windows, so that almost halved the episode processing time. My actual process takes about 16x hours per episode now (depending on which method I choose) so that is around 8x on average given I can do two at once (ish) on my 4x year old PC with a 1080.

As someone with everything on PAL, it is definitely easier for me to process PAL instead of NTSC but I do know what you mean about the quality of footage. I can also see the quality difference just in general between season 1 and season 7.

So my philosophy has been “good enough in a reasonable time frame” as opposed to perfection. I am not saying perfection is bad, just I have gone down the rabbit hole of perfection and I don’t think I have the patience :rofl:

Also streaming in 4K to my set at home is always an issue, so I didn’t see much point for me personally going to 4K and settled on 1080p - but that’s my own preference for how I am watching it ^^

So I start a new season with a cross section of tests to confirm which I am happy to watch - and the proceed to process them all the same way. I try a bunch of things to see, even if I think they may not work.

@dbasthemer

I have season 1 on NTSC and season 2 and 3 arriving shortly. I have had a lot of issues getting the seasons on NTSC however - is there a convenient online store you can point me to? I figured if I can at least do the early seasons from the NTSC masters it will help with the quality downgrade as my Season 7 I am looking at I don’t think I really need everything. But then season 1-3 may be enough…

There are also definitely a lot of rainbow and dot crawl issues early on and getting rid of those - especially in the PAL versions - have been the bane of my upscale tests. The pre-processing methods that work really well always seem to make the ouput in TVEAI worse.

As a current example, this is my cross section of Season 7 DVD (PAL) that I am using to decide which process I am most happy with for Season 7 while keeping in my above philosophy. After watching I am noticing artifacting in the eyes which seems to be interlacing residuals, so I have to experiment a bit further, but I am leaning kind of towards the Dione TV with 0.8 grain into 1.5.3 Gaia CG at the moment… But I shall see and probably watch over and over again :sweat_smile:.

Dropbox - File Deleted

As an edit, it appears the combing in the eyes is in the underlying progressive frames, which means that this has been caused by whatever process was used to convert the original NTSC version to PAL making it very hard to remove. It is un-noticeable in the original footage, but upscaling it is making it very noticeable. I suppose another reason to use the original NTSC footage ^^.

My workflow may actually change, since I am working my way into Da Vinci Resolve as of late.

There were/are two reasons for this: Slightly faster encoding and less issues with banding in the final exported/rendered video file (Adobe seems to have some problems there).

I’ll keep you updated.

Yes indeed! my current movie is about 80 minutes long

1st pass: vegas (10h)
2nd pass: virtual dub (50m)
3rd pass: video enhance (32h)
4th pass: vegas (26h)
this is an average, without the fact that I took about 5 years to find my filter settings with vegas and that I am currently modifying since I use video enhance .

i have ryzen 7 2700, gtx 1660 and 16 gb

1 Like

Yeah, finding “what works for you” is always a matter of compromise: You certainly want quality, but you are always paying for it with rendering time.
And then there is aspect of “perceived visual quality” (at least that’s what I like to call it): Some people prefer sharpness and film grain, others want things smooth and softer, then there is the issue of the level of motion fluidity (for some known as “film frame rate” vs “soap opera effect”) … there is a lot to consider.

I have been looking thorugh the web, comparing upscales on different materials of all sorts, when it comes to Star Trek, there is plenty out there. Unfortunately, I am a bit of a perfectionist and nothing convinced me so far, so I wanted to try myself.

Here are the things I like: Detail-retention (if possible), film grain (personal preference), little to no color banding, accurate motion, original fps.
Here are the things I dislike: Oversharpening, oversmoothing, super fluid motion (soap opera effect), fps modifications through bobbing/interpolation.

That’s that for me.

exactly, I am of the same opinion as you! I prefer my sd sequence at 50 fps instead of 25 fps because it is not fluid enough 25 fps. With a slight stabilization, the movements are clean, but in my case it is a personal camera, it is not the same movements as a pro camera. And then the grain in addition to being aesthetic, and it brings a slight sharpness

Please feel free to comment on the quality. Any tips or ideas are most welcome.