Dynamic compression for H264/H265 (August 2023)

[eric (Eric Yang)]

New Dynamic compression option for H264/H265

Why change the export options? Users frequently ask about the ‘best’ bitrate setting for their Video AI projects, and the answer depends on many different factors including the content of the video, its resolution, and the display size that it will be shown at.

A new solution: To simplify this step of AI processing, we’re introducing a more efficient way to export files in Video AI starting with NVIDIA and AMD GPU users on Windows.

• The new Dynamic mode in the Export panel offers three adaptive compression options for H264 and H265 videos. These presets are designed to help users create files that are only as large as they need to be and don’t waste disk space with high constant bitrates.

Quick Start: For NVIDIA and AMD users on Windows, just select Dynamic from the export options panel and try the medium preset as a starting point. It offers a good balance between file size and visual quality.

How it Works: Using Netflix’s VMAF video analysis tool, we’ve set three compression levels that balance file size and visual quality. For most normal viewing situations a score of 95 or higher is considered ‘visually lossless’, so we’ve set up the High preset to score over 95 while the other two settings are focused on file size optimization:

• High - 98.4 VMAF @ 391MB file size
• Medium - 88.3 VMAF @ 39.9MB file size
• Low - 79.3 VMAF @ 23.4MB file size

Increasing VMAF scores into the high 90s takes a significant amount of disk space, which is why the Medium and Low settings are so close in file size compared to the High setting.

• These settings use CQP rate control, which is a more efficient way of adapting the size of a video to the content that is being encoded. This means that simple scenes should use less bandwidth, while scenes with complex motion and detailed textures will avoid loss of detail.
• We’ll continue to adjust these settings based on user feedback and we plan to bring Dynamic Compression Levels to Apple Silicon and Intel ARC users very soon.
• For users looking for the closest setting to the previous “Auto” bitrate selection we recommend using the “Dynamic - High” mode to ensure file quality lines up with previous exports.

Examples of the three bitrate modes (zoomed to show fine detail):

In these images, you can see the low and medium settings show some blurring and smoothing on the skin texture, while the high setting is able to retain the finer detail of the original shot.

Video credits: “In the Hand” by Guido Pezz

Other improvements

Since last month’s roadmap update, we’ve also released a major internal rework of how filters work and fixed various model quality/performance issues:

• Improved visual quality on first few frames for almost all Enhancement models
• First few frames no longer repeat with Frame Interpolation
• Faster Iris 1x and 2x processing on macOS Ventura
• Apollo can now run on M1/M2 when used with HD+ inputs
• Improved preset management
• Create presets from anywhere
• Reworked custom output resolution flow
• Tesla GPUs now show NVENC H264 and H265 encoders
• Processing on Intel CPU machines (12th, 13th gen) no longer slows down when app is minimized or out of focus
• Fixes Mac issue where all memory would sometimes be used
• Exports / Previews now show filename when hovering over thumbnail
• Login now uses proxy settings
• Fixes image sequence issues when non-ASCII characters are in the path
• Fixed integer overflow issue for start frame number
• Added Iris to benchmark (Ctrl/⌘-B)
• Log file can now be opened when app is open (Ctrl/⌘-G)
• Added and improved tooltips around the app

We’ve also attempted to fix processing errors with Intel Arc / iGPU and when exporting a large number of frames. Please let us know if you experience related issues.

Please read the release threads for 3.4, 3.3.10, 3.3.9, and 3.3.6 for the full list of changes in each release.

---

Next

We have several foundational improvements in the pipeline that should significantly improve how useful Topaz Video AI is to you.

New Enhancement models

• Iris v2 - will offer better detail and improved slider responsiveness. Iris is targeted toward restoring low quality input videos with faces
• Improved results on medium-quality input videos, either through improving an existing model or training a new one
• New 1x denoising model to improve low light performance for high resolution input videos

Workflow improvements

• Direct integration with popular NLEs
• Improved preview experience that allows easier comparison between settings
• Allow offloading processing to a cloud backend
• Allow pausing and resuming processing
• Allow applying a second Enhancement pass without exporting and re-importing

We plan on shipping most of these improvements before the end of 2023. Please apply for the video beta program if you’d like to shape the development of these features. Thanks, and we look forward to hearing what you think!

[brumult]

When will AMD AI cores be used?

[UHD4KUser]

@eric

Iris v2

My hopes for Iris v2:

• Less, or no ghosting between frames, especially those with rapid motion
• Even better compression artifact removal
• Improved facial reconstruction
• Lessened color bloom with certain colors, or preferably no color bloom at all
• Improved overall detail reconstruction
• Better retention of input video’s original colors

[Martyprod (Marty)]

@eric

Direct integration with popular NLEs

very excited by this ! hoping open-cl version will be available to be compatible with Vegas Pro user !!

[david.123 (New Profile Picture!)]

Scene detection for interpolation models was mentioned in the June roadmap as coming soon, but has not been mentioned on the last 2 roadmap updates and we still don’t have it. Frame blending between scene changes is still an issue (see example below).

Can someone from Topaz please provide an update on this?

Other than that, thanks for all the good work on Video AI!

[ida.topazlabs]

Hey @david.123 Yes, this is still a work in progress and is a challenging thing to implement. Rest assured we are working on this!

[a.marschitz]

If 95 is virtually lossless - then HIGH should be 95.

The 98.4 could be named ULTIMATE or MAX or something.

Also: MEDIUM is to low with 88.3. Should be 90 or 91 or no one is going to use it.

Btw: the example pics are not very useful. You should use example pics with lots of detail.

[lhkjacky (Jacky)]

@eric

• High - 98.4 VMAF @ 391MB file size
• Medium - 88.3 VMAF @ 39.9MB file size
• Low - 79.3 VMAF @ 23.4MB file size

For NVENC HEVC,
TVAI is using
High = -rc constqp -qp 15
Medium = -rc constqp -qp 25
Low = -rc constqp -qp 28

The gap between High and Medium is pretty high, I think it should add a Medium-high setting, at around -qp 21

---

Also, I’m just curious. Is there any reason why using “Constant QP (CQP)” instead of "Constant Quality (CQ) "?

[tony.topazlabs]

We’re going to keep adjusting this settings based on user response, thanks for your feedback!

[tony.topazlabs]

CQP is the specific setting for NVENC, while AMD cards are using ‘-rc cqp -qp_i, -qp_p, and -qp_b’ with a separate CQP setting for I-frames and P/B-frames.

[lhkjacky (Jacky)]

@tony.topazlabs

CQP is the specific setting for NVENC

I’m just curious, why not using "Constant Quality (CQ) " ,
such as -rc vbr -cq 28
I think it act similar to Constant Rate Factor (CRF)

Quote from FFMPEG Encoder hevc_nvenc

-cq Set target quality level (0 to 51, 0 means automatic) for constant quality mode in VBR rate control (from 0 to 51) (default 0)

-qp Constant quantization parameter rate control method (from -1 to 51) (default -1)

The reason I ask this because it seem that most of the website recommend using CQ over CQP.
For example,
https://slhck.info/video/2017/03/01/rate-control.html

Quote from above link, (Click to expand)

Constant QP (CQP)

Unless you know what you’re doing and you explicitly want this, it is suggested not use this mode! Setting a fixed QP means that the resulting bitrate will be varying strongly depending on each scene’s complexity, and it will result in rather inefficient encodes for your input video. The quality will be good if you set a low enough QP, but in contrast to CRF, you may waste space, or you have no control of the actual bitrate.

Good for: Video encoding research, or if you have no CRF mode
Bad for: Almost anything else

[angrykrabba]

@eric

For users looking for the closest setting to the previous “Auto” bitrate selection we recommend using the “Dynamic - High” mode to ensure file quality lines up with previous exports.

The High preset uses a lot more bit rate than the previous versions Auto preset.

Auto for 4k encodes uses around 15Mb/s while the Dynamic compression High uses 53Mb/s.

[mpil]

H265 compression with dynamic bitrate does not work at all on my GeForce GTX 1650. It throws error “No capable devices found. Try a different encoder.” and your logs show error “Temporal AQ not supported”.
H264 dynamic works, H265 with constant bitrate works.

[JakSpoon (Jacopo)]

Is very simple: your card does not allow that kind of encoding with the HEVC encoder!

[mpil]

It only does not support one feature - Temporal AQ - which is not crucial for dynamic encoding, encoder can simply query API for NV_ENC_CAPS_SUPPORT_TEMPORAL_AQ and in case it is not supported, use spatial AQ. So even people, who do not own card with this support, can take advantage of dynamic encoding.

[lhkjacky (Jacky)]

I think GeForce GTX 1650 is a bit special, because according to Nvidia official Video Encode GPU Support Matrix, some GTX 1650 have 6th Gen NVENC, while some have 7th Gen NVENC.

[mpil]

I have some DDR6 version. But the point is, that the encoder feature “Temporal AQ” is easily detectable through API and encoder should use Spatial AQ, where Temporal AQ is not accessible. It still could be “dynamic”, only slightly different setting of encoder and suddenly a lot more supported GPUs.

[david.123 (New Profile Picture!)]

You can probably write your own entry for video-encoders.json and make it work with Spatial AQ, whatever that FFMpeg command is.

[johngshen]

I agree with previous users’ comments that the “high” setting is just way too high, 7 to 10x the ouput file size of medium setting. On the other hand, the medium setting seems too low.

480p DVD upscale to 960p with medium option only gives a 3.3mbps average which is low for FHD standard. The output file using h265 is actually smaller than the loseless DVD file. I appreciate the effort to contain file size, but most users probably can live with same or slightly larger file size than original but with the best possible quality.

Settings slightly higher and lower than previous auto setting would probably make a lot of sense for users to compare and adapt.

[lhkjacky (Jacky)]

@johngshen

I agree with previous users’ comments that the “high” setting is just way too high, 7 to 10x the ouput file size of medium setting. On the other hand, the medium setting seems too low.

If you want, you can edit the “cqpValues” in “video-encoders.json” manually.
I also found that the Quality Level High (qp 15) file size is way too large, therefore I manually change the cqpValues High to (qp 20) for hevc_nvenc and h264_nvenc.