Experience With DVD (SD) Video Upscaling

Topaz Video AI General
1

Over the past 2 years, I’ve had the opportunity to use TVAI extensively, primarily in the upscaling of old concert videos for my own enjoyment. The source material is typically SD and interlaced (e.g., 480i); I generally aim for an upscaling to 1080p. IMO, upscaling of SD material to 4K is a time-consuming fool’s errand. I’ve seen a number of newcomer posts related to DVD upscaling, so I wanted to share my experience with this particular task using TVAI and additional tools.

I do not rely on current, high-end hardware. My laptop is approaching 6 years in age. I’m fortunate if my upscale settings allow me to encode at 0.9 fps (or about 24 hours per one hour of footage). I’ve also seen a number of post from frustrated users stating that they’re abandoning the idea of using TVAI because of slow processing speed. Unfortunately, there is no similar yet faster software for this purpose that I’m familiar with. It requires patience. In my case, the reward comes in seeing a long-forgotten concert from, say, the 1970s, upscaled and improved for show on the big screen, supported by a quality audio track. Simple pleasures . . .

Whether you use TVAI or another software for this purpose, your DVD/SD video should first be de-interlaced and the pixel aspect ratio (PAR) should be converted to 1:1 or “square pixel.” The conversion to square pixel helps to prevent later problems with header encoding (e.g., “original display aspect ratio”) that may affect the program display (i.e., problems with stretching or squashing of the video on playback). There are a number of posts which claim one de-interlacing method to be superlative. In general, both QTGMC and TVAI de-interlacing provide good results. QTGMC offers the benefit of speed over TVAI; however, in rare cases, I have found source material which did not interact well with QTGMC. In the few instances where this has occurred, QTGMC appeared to create artifacts which looked like coarse aliasing. My go-to QTGMC settings include a source matching of “2xrefined” and match presets at “medium.” If the source SD video is of exceptional quality and I have the time, I might select QTGMC “Bob,” to effectively double the frame rate. In most cases, however, I choose to keep the native frame rate. My processing chain involves several steps, and I generally encode to lossless (FFV1, 10-bit) format until I’ve reached the end of the chain. FFVI encoding is supported by both Hybrid encoder and TVAI. If you choose to use the Hybrid encoder for pre-processing, prior to upscaling, you will find that, even with the basic install (i.e. no installation of dependencies), it offers PAR conversion and Vapoursynth QTGMC de-interlacing. Until the final step in my processing chain (to be discussed later), I do not crop video or touch audio.

TVAI upscaling typically follows – or occurs simultaneously with – PAR conversion and de-interlacing. I have also read numerous posts claiming one AI model to be superior to the rest for any particular purpose. From my experience, there isn’t a case for exclusive use of one model for DVD/SD upscaling. In my first year of using TVAI, I found that Proteus generally provided the best results with my DVD upscaling; however, I would occasionally come across a source video which did not react well to the Proteus model, and no amount of tweaking the settings would produce a desirable result. In those few instances, I found that another model, such as Artemis Medium or Artemis High, provided a more pleasing upscale. Over the past year or so, Iris Medium has become my go-to model for DVD/SD upscaling. In most cases, I am able to achieve a more visually pleasing upscale than with Proteus. Still, rarely, a source video will announce its incompatibility with Iris and show a preference for the Proteus or Artemis models. Although the Dione models are those most associated with de-interlacing in TVAI, I have chosen not to use these, as I’ve never been pleased with results of Dione on upscaling. In more recent versions of TVAI, a de-interlacing function has been paired with Iris, and I have found that this provides good de-interlacing results. For many upscales, this provides an opportunity to eliminate a pre-processing step using another software such as Hybrid, considering that TVAI also provides an option for conversion to square pixels.

Concerning the TVAI upscaling itself, I always use the “manual” option, and I have learned that there is no “rule” related to individual settings. I recommend against using the “Auto” feature. Let your eyes be your guide. Some source videos require minimal Fix Compression values; some require values over 80 to remove the funk. Noise always seems to skew the models and predispose them to artifacts. If you encounter unexpected artifacts, especially if they appear as parallel, vertical or horizontal striations, crank up the Remove Noise slider, in tandem with the Fix Compression slider, to find minimal values at which these disappear. For the DVD source material that I most often upscale, I always keep the Anti-aliasing/Deblur slider in the realm of “Deblur,” with values typically ranging from 8 to 24. I’m still trying to figure out the purpose of the Remove Halo slider, since it doesn’t seem to remove haloing at any value, in the videos that I upscale. If the video has been significantly de-noised in the process, I typically add conservative Size/Amount values for grain, often 1/1 or 2/2. Everything here is season to taste.

I do not process audio in TVAI, as I’ve never been satisfied with TVAI’s audio handling. I’m also not currently satisfied with the encoding options for H.264 and H.265 in the TVAI “save as” menu. In TVAI, I save the output as lossless in FFVI, 10-bit format. I then pass this file to a free software like Handbrake, usually for H.265 encoding, since it offers better control of the encoding parameters. I also crop appropriately in this final step. If PAR conversion to square pixel was done at the outset and cropping saved until this last step, this generally prevents any display distortions. Since audio processing wasn’t included in the previous steps, I then use MKVToolnix to remux the H.265-encoded file with the source audio tracks. If I’m not satisfied with the format of the source audio tracks, the source file can be loaded into Hybrid for audio-only processing. This allows for convenient conversion of, say, PCM audio to FLAC or Opus.

I hope that this is helpful. I didn’t address 8-bit vs. 10-bit encoding or color palettes, since I’m only focusing on the upscaling of older SD videos. I keep my processing chain at 10-bit until/if I decide to change it.

3 Likes
2

It will make no difference if you encode it in 8-bit, as your source DVDs were captured and stored at 8-Bit color space. encoding it with 10-Bit will not magickly enhance coloration or make it any better.

Transferring your image beans from the original smaller container into a larger container will not give you more beans…

2 Likes
3

hi, well it seems to me that once I tried 10-bit encoding on one of my SD videos and the quality was slightly better than 10-bit. I’m going to try this again. the one where I didn’t see a difference was the ffv1 encoding versus the lossless mp4 encoding

4

And yet others will claim that H.265 encoding to 10-bit will reduce color banding. The decision to use 8-bit vs. 10-bit H.265 encoding wasn’t the focus of my post.

5

I think it would be better to convert the PAR after it is processed by TVAI to get a somewhat higher quality image.

For example, in NTSC (4:3), 720x480 (704x480) is a 10:11 PAR, which becomes 640x480 when converted to 1:1.
Converting the PAR first would throw away 64 horizontal dots of information.
Also, TVAI can only process 2x and 4x, and other scaling is adjusted by FFMPEG’s Lanczos filter.

The comparison is as follows

704x480(10:11) → (scale) → 640x480(1:1) → (TVAI) → 1280x960 → (scale) → 1440x1080(1:1)
704x480(10:11) → (TVAI) → 1408x960 → (scale) → 1440x1080(1:1)

I think the latter has a slight advantage in terms of image quality, since more pixels are sent to TVAI and the resizing is done once.
However, I usually do the above process with CLI, so I don’t know how to do it with GUI.

It will make no difference if you encode it in 8-bit, as your source DVDs were captured and stored at 8-Bit color space. encoding it with 10-Bit will not magickly enhance coloration or make it any better.

I think this is wrong, TVAI processes internally in 16bit.
When TVAI was ver 2.0, it was 8bit, but since 3.0, it is 16bit.
High bit processing is because there is generation of intermediate colors by noise processing and scaling,
useful to avoid overflow, tone jumps, etc.
Considering that the image is converted from 16 bits, saving at 10 bits preserves tones better than saving at 8 bits.

1 Like
6

Or you could do what I do and convert to PAR: 1x1 by Hight. 720x480 to 720x540
Upscale x2 will bring you a perfect 1440x1080
720x480(10:11) → 720x540 (1x1) → (scale x2) → 1440x1080(1x1)

1 Like
7

In NTSC 4:3, the visible pixels is 704x480, so that calculation seems incorrect.

8

you are talking about something else, about the extra space on the sides that are NULL. that is not unique to NTSC, it is also PAL. as the matter of fact it is all interlaced content that was captured for CRT TVs. but still you can 1x1 it by highest. I do it that way and it comes out great and less steps.

9

Thanks for input on where the PAR conversion might better fall in the sequence. That’s certainly given me something to think about.

I loaded a video file with 720x480 resolution and a PAR of 8:9 into Hybrid encoder. On the Crop/Resize tab, when PAR conversion is set to 1:1, the default settings adjust height, to bring the output resolution to 720x540. Fortunately, I have pre-processed most of my videos with this default setting.

That said, what is the case when all steps (de-interlacing, PAR conversion, upscaling) are carried out in TVAI, in one operation? Prior to this conversation, I had not checked to see if the width or the height was adjusted to achieve the 1:1 PAR.

10

you already did it in Hybrid as I see in your comments. so all is left for you to do is Upscale (x2) as a progressive scan video.
If you skip Hybrid altogether then you can just Upscale Directly to 1440x1080 when you set the Square Pixel Type. it will do all this automatically (converting to PAR: 1x1 then Upscale).

Behind the scenes there will be 2 processes

  1. the 1st process, TVA will call ffmpeg to deinterlace and convert to PAR:1x1 (it is not TVAI engine that does deinterlacing and/or PAR conversion, it is pure ffmpeg Bwdif)
  2. once step one is completed, TVAI engine will kick in and start the upscale process.

NOTE: In any can or workflow you pick, deinterlacing and conversion to Square Pixel (PAR:1x1), it is always done by ffmpeg and not TVAI itself.
So if you are debating where to do the deinterlacing and PAR conversion in Hybrid or TVAI, You question is more accurate to ask where is it better? FFmpeg vs. Hybrid?

11

As described in wikipedia here, 10:11 is commonly used for 480i.

704x480(10:11) → 640x480(1:1) or 1440x1080(1:1)
720x480(10:11) → 654x480(1:1) or 1472x1080(1:1)

Therefore, I think that stretching to 720x540 and then enlarging by 2x is an incorrect calculation for PAR.

12

I would love this to be true, but I suspect TVAI just treats the non-square pixels as square—since the square versus non-squareness is stored in metadata only.
I would love to throw together a test script (Just kidding—modify my current script), to compare the two ways with that being the only change. I’m not really sure it’s worth the time figuring out how to transfer my PAR calculation to work after TVAI though.

EDIT: Oh never mind. TVAI always does scale down before processing, if you pick the square pixel option. So yeah, there will be a little more information to work with if you don’t do that first.

You can ignore me here.

1 Like
13

Thanks, Akila. My process involves using both pre-processing in Hybrid (PAR conversion and de-interlacing) and de-interlacing in TVAI, though obviously not for the same video. While I do like QTGMC de-interlacing, which works 95% of the time, I rarely come across video sources that don’t respond well to QTGMC, leaving visually displeasing results. In those cases, it is preferable for me to do everything in TVAI. That said, I was not aware that TVAI was using bwdif for its de-interlacing. For the QTGMC de-interlacing failures, the bwdif de-interlacing in TVAI did a fine job, as far as my eyes can tell. Since Hybrid offers control of the PAR conversion parameters, I will continue to de-interlace in that software for most projects.

14

If it helps, I’ll upload a capture of the Hybrid PAR/Crop work screen. In this case, Hybrid detects an input PAR of 8:9. Perhaps it’s an oddball video.

hybrid
hybrid1408×641 28.1 KB

15

Ah, I think I figured out what was tripping me up earlier. If the DVD is NTSC 16:9 then converting to square pixels first will give more for TVAI to work with.

16

That is how I convert my videos to Square pixel (PAR: 1x1).