Real-time SD to HD with TVAI v3.1.x and VLC

Real-time SD to HD with TVAI v3.1.x and VLC

I’ve now finally found a way to upscale MKVs from interlaced DVDs (576i) to 1080p fast enough to view in real-time using VLC. The following modified TVAI ffmpeg command does the trick on my Mac Studio (lowest spec M1 Max). I use Artemis but Proteus gives similar speed:

ffmpeg “-hide_banner” “-nostdin” “-y” “-i” “/Users/Andy/Desktop/S1E01.mkv” “-sws_flags” “spline+accurate_rnd+full_chroma_int” “-color_trc” “2” “-colorspace” “2” “-color_primaries” “2” “-filter_complex” “yadif=0:parity=auto:deint=all,scale=w=720:h=528,tvai_up=model=amq-13:scale=0:device=0:vram=0.1:instances=1,scale=w=1440:h=1080:flags=lanczos:threads=0” “-c:v” “hevc_videotoolbox” “-profile:v” “main” “-tag:v” “hvc1” “-pix_fmt” “yuv420p” “-allow_sw” “1” “-b:v” “5M” “-map_metadata” “0” “-movflags” "frag_keyframe+empty_moov+delay_moov+use_metadata_tags+write_colr " “-map_metadata:s:v” “0:s:v” “-map_metadata:s:a” “0:s:a” “-c:a” “copy” “/Users/Andy/Desktop/up.mkv”

The main modifications to the standard TVAI export command are:

  1. Add de-interlace already “built-in” to ffmpeg. I chose yadif but bwdif could be used. The output is 1 frame per 2 fields (so 50i to 25p)
  2. To get a significant speed increase I slightly downscale the vertical resolution first from 576 to 528. Without that, I could only get around 22 fps. With it, I get 33 fps

Because the output is an MKV file, it can be opened by VLC while it’s being written by ffmpeg. VLC is the only desktop app that I’ve found that carries on reading the file as it gets larger. Unfortunately, the mobile versions of VLC don’t. However, there’s another method where any version of VLC appears to work (including for Apple TV, iPad and iPhone) and that’s to use a network stream instead.

For streaming which works with VLC:

  1. Add “-re” at the beginning of the command after ffmpeg to set the frame rate to real-time
  2. Replace the filename at the end of the command with -f mpegts “udp://192.168.1.247:1234?pkt_size=1300”

The ip address is the device which VLC is running on and 1234 is the chosen port number. In VLC, open the network stream with udp://@:1234.

Please note the these are only examples which work on my Mac Studio (and UK DVDs as the source) but hopefully gives enough detail to try for yourselves.

Thanks.

Andy

2 Likes

Sorry if this is intentional, but it appears that that command uses AI for noise removal but upscaling is done by FFMPEG’s Lanczos.

For simplicity, remove the double quotation marks.Extract only the filter_complex part.

-filter_complex yadif=0:parity=auto:deint=all,scale=w=720:h=528,tvai_up=model=amq-13:scale=0:device=0:vram=0.1:instances=1,scale=w=1440:h=1080:flags=lanczos:threads=0

Filters are chained together by commas.Replace commas with return lines.

yadif=0:parity=auto:deint=all
scale=w=720:h=528
tvai_up=model=amq-13:scale=0:device=0:vram=0.1:instances=1
scale=w=1440:h=1080:flags=lanczos:threads=0

Scale is not set for tvai_up filter.
It is scaled by the FFMPEG Lanczos after that.

To upscale with AI, the scaling factor must be specified within the tvai_up filter.
You can either specify the size of the image after it is enlarged, or specify the magnification directly (2x or 4x).

tvai_up=model=amq-13:scale=0:w=1440:h=1080:device=0:vram=0.1:instances=1
or
tvai_up=model=amq-13:scale=2:device=0:vram=0.1:instances=1