Inadequate Compression in TVAI

I’m using TVAI version 3.3.2 and am getting undesirable output file compression. Using an SD source, upscaled to 1080p with the H264 encoder, the output file was ~30Gb in size. I ran this output file through Handbrake H264 with the settings constant rate, same FPS and RF/CRF of 18 to achieve a file of ~5Gb. I don’t want to have to re-encode TVAI-processed files through a separate encoder. Can this be fixed? Thanks.

what bitrate setting you pick in VAI when you export?

image

I left it on “Auto.”

try by lowering it to 8000kbps
you can use MediaInfo tool to see what is your current kbps when you leave it on Auto. that way you can adjust quality to size balance.

Example:

image

1 Like

The “Auto” mode typically just means TVAI will pick a bitrate for you. As for how this bitrate is calculated, I do not know, but a lot of the time the bit rate can be quite high since I assume the developers of TVAI are targeting high quality outputs, not good compression levels.

One side thing about this. TVAI by default always operates in a “bitrate target” encoding mode which many people don’t want. Instead many people prefer a constant quality mode (similar to CRF).

So the developers could offer the option to pick a “constant quality” (Similar to CRF) or “constant quantize” mode instead of a straight bitrate mode, but sadly they don’t let you.
And I think the reason they don’t let you is because TVAI only offers GPU hardware encoders for the main codecs (H264, H265, AV1), and the different GPU hardware encoders have inconsistent quality control options and inconsistent quality across each vendor. (E.G. When I configure what I thought was identical constant quality settings for hevc_nvenc (Nvidia h265) and qsv_nvenc (Intel h265), the Nvidia one used a much higher bit rate (E.G. 5x higher) and was thus higher quality).
So if they let users pick a “Constant Quality” mode, and someone on Nvidia sets the constant quality mode to 23, and some on Intel also sets it to 23, then you will end up with different bit rates, file sizes, and quality simply because they’re using different GPUs.

CPU encoders would fix this. Because CPU encoders will always offer the same quality, no matter what GPU you have. And as a side effect of that, if TVAI had CPU encoders, they could offer constant quality modes with greater ease.

As for why they don’t offer CPU encoders for h264, h265, and possibly AV1, it’s for two reasons:

  1. Upscaling performance is decreased when you use a CPU encoder instead of a GPU encoder. It’s not that the CPU encoder slows down the upscale, it’s just that all the stuff related to CPU encoding causes overall slow downs which results in the upscales being slower.
  2. I believe there are licensing issues stopping them from adding H264 and H265 CPU software encoders. I believe either TVAI would need to pay a licensing fee, or make more of their code open source which they probably don’t want to do. Although I could be entirely wrong about this one.
3 Likes

Interesting. TVAI used to have a CRF/RF selection slider, but it disappeared in more recent versions. I will try adjusting the bit rate rather than trust the encode to an arbitrary “auto” setting.

Got it. I used to get reasonable compression with “auto.” Recently, that went away. I’ll make certain to set the bit rate from now on.

Here’s the MediaInfo readout on the encode that I told you about. The “Auto” setting does pick a fairly high bit rate, even considering that it is a high frame-rate video.

yeah, that is to high bitrate for file efficiency

Or you can export your video in TVAI with Lossless Compression, then head over to an external tool (like I do), such as: Handbrake, VidCoder, Avidemux, Hybrid, etc. and compress it with your favorite CRF settings using CPU for your final video. that way you enjoy both worlds, Quality and size.

That’s a good idea. I’ll start exporting from TVAI lossless. Since you mentioned Handbrake, I was surprised at the bit rate, approximately 5,500 kbps for a Handbrake-encoded 1080p, 60fps file using a RF/CRF setting of 19. I know that HB, when using constant quality, is supposed to apply bit rate as needed, but I don’t know if I fully trust it.

You should, that is the recommended method almost by all experts these days.
read those and decide for yourself.

Thanks. Good articles.

1 Like

Akila, I meant to ask. Why did you specify CPU encoding (vs. GPU encoding) in an external program? I have been using nVidia encoding within TVAI.

@markwaltermd the reason being is that with CPU you can set CRF setting while with GPU you can’t (CRF is the recommended way going by experts in general).
Another reason , I personally experienced when using both (CPU vs. GPU), that CPU encoding provides smaller file size at the same quality level.

1 Like

Got it! I found the same thing when using the version of NVENC found on my older nVidia card. I also got poor size/quality when using Intel QSV. Until I get a better graphics card, I guess that I’m stuck using CPU calculations.

So, couple of things I’d like to mention here. I would never use TVAI to do any kind of long-GOP encoding because that’s not what it’s for, and it’s not very good at it. Intermediate codecs like ProResHD are the way to go, but remember that even if it’s “lossless” you’re still probably doing colorspace conversions and whatnot that aren’t good for quality. So try not to save it any more times than you have to.

Also, I would never recommend Handbrake for any kind of serious video encoding. It’s good for beginners, but it’s like a sledgehammer. There’s a big tradeoff in the amount of things you can control if you know what you’re doing. If you’re serious enough to spend the money to buy TVAI, you’re probably serious enough that it’s worth your time to get comfortable with ffmpeg and x264/5. on the command line.

This topic was automatically closed 60 days after the last reply. New replies are no longer allowed.