Source video to 4K with Gaia:
1920x804 to 4K = 50% speed
1920x1080 to 4K = 100% speed
My solution so far: first insert the video in 1920x1080 with DavinciResolve, export it lossless and then edit it with Topaz AI. Worth it considerably in terms of time. However, there is certainly a better solution. A special command at ffmpeg? Add “black bar”?
You can see an example of how to add such black bars if you look deep into the command created from a 4:3 source to 16:9. I believe the video filter is pad=1920:1080:-1:-1:color=black.
I wasn’t sure how they were doing the cropping, so I tried several different ways to see what the FFMPEG command would look like.
A:1920x1080 to 3840x2160(4K)
ffmpeg -hide_banner -nostdin -y -nostats -i 1080.mkv -sws_flags spline+accurate_rnd+full_chroma_int -color_trc 1 -colorspace 1 -color_primaries 1 -filter_complex tvai_up=model=prob-3:scale=0:w=3840:h=2160:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=1:instances=1,scale=w=3840:h=2160:flags=lanczos:threads=0,scale=out_color_matrix=bt709
B:1920x804 to 3840x2160(4K)
ffmpeg -hide_banner -nostdin -y -nostats -i 0804.mkv -sws_flags spline+accurate_rnd+full_chroma_int -color_trc 1 -colorspace 1 -color_primaries 1 -filter_complex tvai_up=model=prob-3:scale=0:w=3840:h=2160:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=1:instances=1,scale=w=3840:h=2160:flags=lanczos:threads=0:force_original_aspect_ratio=decrease,pad=3840:2160:-1:-1:color=black,scale=out_color_matrix=bt709
C:1920x804 to Select Width 3840x1608
ffmpeg -hide_banner -nostdin -y -nostats -i 0804.mkv -sws_flags spline+accurate_rnd+full_chroma_int -color_trc 1 -colorspace 1 -color_primaries 1 -filter_complex tvai_up=model=prob-3:scale=0:w=3840:h=1608:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=1:instances=1,scale=w=3840:h=1608:flags=lanczos:threads=0,scale=out_color_matrix=bt709
D:1920x1080(crop 1920x804) to 3840x2160(4K) Letterbox/Pillarbox
ffmpeg -hide_banner -nostdin -y -nostats -i 1080.mkv -sws_flags spline+accurate_rnd+full_chroma_int -color_trc 1 -colorspace 1 -color_primaries 1 -filter_complex crop=w=1920:h=804:x=0:y=0,tvai_up=model=prob-3:scale=0:w=3840:h=2160:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=1:instances=1,scale=w=3840:h=2160:flags=lanczos:threads=0:force_original_aspect_ratio=decrease,pad=3840:2160:-1:-1:color=black,scale=out_color_matrix=bt709
E:1920x1080(crop 1920x804) to Select Width 3840x1608
ffmpeg -hide_banner -nostdin -y -nostats -i 1080.mkv -sws_flags spline+accurate_rnd+full_chroma_int -color_trc 1 -colorspace 1 -color_primaries 1 -filter_complex crop=w=1920:h=804:x=0:y=0,tvai_up=model=prob-3:scale=0:w=3840:h=1608:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=1:instances=1,scale=w=3840:h=1612:flags=lanczos:threads=0,scale=out_color_matrix=bt709
Filter tvai_up
Apply Topaz Video AI upscale models, parameters will only be applied to appropriate models
Inputs:
#0: default (video)
Outputs:
#0: default (video)
tvai_up AVOptions:
model <string> ..FV....... Model short name (default "amq-13")
scale <int> ..FV....... Output scale (from 0 to 4) (default 1)
w <int> ..FV....... Estimate scale based on output width (from 0 to 100000) (default 0)
h <int> ..FV....... Estimate scale based on output height (from 0 to 100000) (default 0)
The tvai_up filter has only 2x and 4x scale, and other scale are adjusted by scaling with FFMPEG’s built-in Lanczos filter.
The “scale=w=3840:h=2160:flags=lanczos” and so on are the adjustments made by the Lanczos filter.
There are also two ways to specify the magnification rate to the tvai_up filter.
ex1 tvai_up=model=prob-3:scale=2
Fill in the multiplier directly.
ex2 tvai_up=model=prob-3:scale=0:w=3840:h=2160
By filling in the size after enlargement, the magnification is determined inside tvai_up.
If the size is more than 1.2 times the original size, 2x is applied; if the size is more than 2.4 times the original size, 4x is applied.
Based on the above conditions, let us interpret A through E.
A:1080 as 2160 (2x), so 2x scaling is applied internally.
B:Since the 804 is set to 2160 (2.68x), 4x scaling is applied internally, slowing it down.
C:804 as 1608 (2x), so 2x scaling is applied internally.
D:After cropping to 804, the image is scaled to 2160 (4x internal = 3216) and adjusted to 2160 by Lanczos resizing and padding. The result is of course slower.
E:804 as 1608 (2x), so 2x scaling is applied internally.
Thank you for the detailed test.
Did I understand correctly that there is only my emergency solution mentioned at the beginning?
Which is “1920x804 to 4K”, 3840x2160(3840x1608 with padding) or 3840x1608?
If 3840x1608, then the C or E would be used.
If 3840x2160(3840x1608 with padding), it should be output in lossless by C or E and then padded when compressed to H.265 or other formats.
If FFMPEG can be handled directly, the D can be rewritten as follows.
Just padding after 2x scale up with tvai.
ffmpeg -hide_banner -nostdin -y -nostats -i 1080.mkv -sws_flags spline+accurate_rnd+full_chroma_int -color_trc 1 -colorspace 1 -color_primaries 1 -filter_complex crop=w=1920:h=804:x=0:y=0,tvai_up=model=prob-3:scale=2:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=1:instances=1,pad=3840:2160:-1:-1:color=black,scale=out_color_matrix=bt709
The target should be 3840x2160(3840x1608 with padding). I will test this. Thank you.
