This release includes two major updates focused on output quality and app performance:
1. All-new, GPU-accelerated video player
Video AI 5.4 uses an mpv-based video player that unlocks incredible improvements to playback performance. Scrub through highly-compressed input videos, or massive uncompressed media with near-instant responsiveness. This new player also supports HDR displays and audio playback! The old live preview behavior with the “Render” button is also restored. Right now, the new player is available on Windows only. We are working on bringing this player to macOS very soon.
Playback of 8K video with old player:
Playback of 8K video with new player:
Known issues:
Dragging the comparison bar in Split View during playback could have a lagging visual effect.
2. Hyperion SDR-to-HDR model
This model is designed to perform inverse tonemapping by increasing the bit depth, expanding the color gamut, and brightening the highlights of input videos. It improves contrast and recovers detail in highlight and shadow areas of standard dynamic range inputs, and produces HDR10 (BT.2020/PQ) output.
Controls:
Highlight threshold slider: This slider explicitly tells the model the lowest intensity level to be considered a highlight. This slider should be set to a value higher than the brightest skin-tone (or any other color where accuracy is vital) present in the video. In practical terms, we recommend starting with the default setting and raising the value if undesirable color shifts appear.
Saturation boost slider: The HDR color space has a larger color gamut and supports more saturated colors. Turning this slider up results in more saturated outputs in HDR without loss of detail. This slider will, however, lead to a color shift.
Changelog from 5.3.4
Improved timeline control, allowing to set in/out points more smoothly.
Adjusted macOS window margins to allow dragging on the window near the buttons.
Matched bitrate setting names across preferences and export pages for consistency.
Combo boxes will now close automatically if you scroll away, preventing them from getting stuck.
Resolved preview and export issues when both focus adjustments and cropping are enabled.
Added a quick way to copy settings from preview/export and paste them into input videos.
Fixed issue where “Auto” label not displaying correctly in Rhea previews.
Fixed changes in preset settings when switching between interlaced and progressive models, so settings are retained and input switching is smoother.
Fixed crashes that happened during large batch exports or when loading projects with multiple exports.
The minimum window size is too small so that the Rendering button is cut off. This has been mentioned before.
Video AI spams full the C:/ drive even though I told a specific installation path on a different drive including drives for working. Not only this: It spams in a hidden folder. This has also been mentioned before.
When I just encode a file without enhancements it successfully completes the task but the info line gets stuck/frozen at around 7 %. This has also been mentioned before.
I have noticed that the new player messes up often. Let me try to explain.
I am trying to test with an anime named Food Wars! right now. My starting frame has a character saying a line which is subtitled at the bottom (which is also weird, when did this start putting subtitles baked in?). I let the test loop a few times, which was indeed quite smooth. However, when I paused it then removed all groups, the image did not change, however the subtitles from my first frame showed back up, but also only in the OG side.
The new player is displaying subtitles in the preview. Not sure how that’s happening, since the srt file is not in the same directory where the source MP4 is located, but how do I turn them off?
First test on my side showed, that this beta doesn’t ‘honor’ the [duplicate frame replace] option either (like TVAI 5.4.0.8a). Could you fix this, please, before this bug finds its way into stable version?
Currently, only 5.0.4/5.3.5/5.3.6 (from what I can tell - I have these installed) version of TVAI are working with this option ON as intended.
It’s great to see mpv player, Rhea XL and SDR->HDR models to joinf the beta fray - thank you!
The other, long anticipated thing for me would be the ability to change the video’s aspect ratio, not by cropping, but by defining the pixel’s ‘ratio’ - I had seen some 4:3 videos that were totally incorrectly hardcoded into 16:9 format, and also vice versa…
There’s also the bug with TEMP folder being wiped still looming (just observed it)- this beta seems to ignore the ‘workspace’/‘projects’ folder setting to some extent - I’m noticing preview videos being saved next to the original file, too.
Yeah, this beta removes the manually set ‘workspaces’ folder at exit and doesn’t recreate it at startup.
I believe this beta might be placing temp files where the processed video destination folder is.
I like the work programmers are doing with the preview - now the app will try to adjust the previewed/processed frame as close as it can to the original one, even with frame generation. But as I mentioned, duplicate frame replacement option is turned off internally or broken.
Did I miss them fixing that? That has never worked. It might make things more blurry with the Chronos models, but it has never done what the name implies with Apollo or Aion.
From what I can see, the rdt value does change in the commands it generates: Off rdt=-0.000001 On rdt=0.029—so there’s no disconnect between the UI and ffmpeg. In the past there has been issues where models in ffmpeg ignore the input values. So changing rdt would make no difference, if that is the case.
Ah, it’s the mpv displaying the subtitles from either your separate subtitle file, or the video container.
The devs will tackle this a some point, I believe.
Makes sense, I just hope it is not hard coding them in when you export! That would… well… suck.
Would be interesting to see how it handles a .ass with multi-fonts. Maybe if I get the darn updates to properly apply to this HCI Stack sometime this year, I can go test that
I just downloaded 5.3.5 and did a pretty easy to recreate test:
Took a DVD and did not drop duplicate frames so that TVAI was fed the 29.97 frame rate with duplicate frames. I tried ‘Duplicate Frames Replace’ at off, 20, 70 and 100 with Apollo. I put the slow motion factor to X2. The only difference I can detect is that with it off, the duplicate frames can make motion take a step back. With it on, they are just duplicates. Either way, it takes the duplicate frames very heavily into account when calculating motion—resulting in jerky motion.
Using ffmpeg -r 23.976 to drop the frame rate (And on most DVDs, drop the duplicate frames), then running the same model with the same settings, produces smooth motion.
My conclusion is that it is still as nonfunctional as it has always been.
just like the alpha, this straight crashes when i try to load certain mp4 files…
After the crash, the video is already “loaded” so i have to start the program numerous times and try to close input file before the program completely crashes again.
Believe me I know, I was fighting for the longest for acknowledgement of when the interpolating feature was duplicating the frames, it was finally regarded and fixed, but yeah, the model is functionally useless for the task at hand.
I can’t get the bloody thing to display the UI, as it keeps flashing in and out, so this beta is useless for my system… I’m guessing the new player doesn’t work with my system, reverting to the previous beta…
