Hyperion SDR-to-HDR Alpha #1

[tony.topazlabs]

Hi everyone,

Today we’re introducing our first SDR to HDR conversion model, Hyperion.

5.2.1.0.hyperion

• Windows
• Mac - DMG | PKG

This model is designed to perform inverse tonemapping by increasing the bit depth, expanding the color gamut, and brightening the highlights of input videos.

This model improves contrast and recovers detail in highlight and shadow areas of standard dynamic range inputs, and produces HDR10 (BT.2020/PQ) output.

HDR playback is not currently supported in the app, and the output preview section has been updated for this model with an overlay informing users about this limitation. We recommend mpv player on Windows (mpv.io), or IINA on macOS (iina.io).

For optimal export settings, we recommend the H.265 codec’s Main10 profile for this model. Additionally, on macOS, ProRes 4444 XQ will result in higher fidelity exports (12-bit).

We’ve uploaded some samples of Hyperion’s output at different brightness settings to demonstrate the highlight expansion capabilities of the model:

SDR input
200 nit output
600 nit output

For this first alpha, we are seeking feedback and samples to determine the ideal default brightness level, along with edge cases that show banding or crushed black levels. We recommend starting with 200 nits and then testing higher levels at increments of 100-200 nits.

As always, thanks for your feedback as we step into the world of HDR!

[Alpheratz]

Definitely love the name you give to your AI models
There are some greek mythology fanboys in the team

[NerdyPhotos (Rob)]

YESSSS!!! I’ve been asking for this for months !!!

[riccot]

finally! thank you.
can’t wait to try it out.

[wesley.kao]

Just a preliminary test but it seems like I see fluctuating changes between colors of yellow hue and red hue

[Marc_Potocnik]

Love it, can’t wait to test it out! Question: is this model able to hallucinate some proper highlight rolloff in otherwise burnt out areas?

[SkodreragGaming]

I just tried it and for my part it is too slow for export and not fast. It reminds me of Rhea at the beginning in alpha, very slow for export. Why don’t you run the model directly in TensoRT directly? Why always put new models in a slowness at the beginning?

[annemartensa]

I just used rhea 5.2.0.3 it is slower than beta 5.2 but the optimization gives a better result now next step use the output from 5.2.03 alpha in HDR or use one of my old UK x factor movies

[zelenooki87 (Zeleni)]

It’s great that you implemented SDR/HDR conversion in the program.

Unfortunately my monitor doesn’t support HDR. And whichever player I try, I get different colors.

MPC-HC

MPV

Potplayer

Google photos web player

[Paul_Coddington]

This is very exciting news and I hope a still picture variant will be added to Photo AI down the track.

[Paul_Coddington]

Samples are blown out for me (have not had time to try anything else).

madVR reports the 200 nit sample has an average peak of 1037 nits.
The 600 nit sample has an average peak of 3478 nits.

Might be a metadata issue.

Setting monitor to 1000 nit (200 nit sample) and 4000nit PQ (600 nit sample) resolves highlight blowouts, but my monitor does not have the peak nits to render 1000nit and 4000nit PQ at true (calibrated) levels, so it looks a bit flat (non-dynamic). In practice, my calibrated levels conk out at 350 nits, so I am very limited (if the 200 nit sample was truly 200 nits, I could view it with accurate calibration, no trouble at all, but 600 would have to clip or roll off).

Color saturation mapping seems to have gone a bit off (oversaturated apple, clouds now more pink, grass a little neon).

Still early days, and this is not a trivial problem to solve, but off to an interesting start.

[Paul_Coddington]

Quite a few video players (and image viewers) are not yet able to display HDR content. They cannot even tone map it to display correctly on an SDR monitor.

I am using JRiver Media Center with madVR to passthrough HDR directly to an HDR capable monitor, bypassing the SDR Windows desktop. This setup can also tone map HDR video to display correctly on the Windows desktop in SDR on an SDR monitor. Added bonus, it plays back all video, SDR and HDR fully calibrated to the current monitor mode/profile at the correct frame rate.

As for picture viewers, ImageGlass can tone map HDR images to SDR but cannot display them properly in HDR. I use Affinity Photo to view HDR static images in HDR at this time (slow and clunky, but works).

Hopefully this information will help you serviceably view HDR content on your SDR monitor, but without an HDR pipeline and monitor it is not really possible to evaluate this new feature being added to Video AI.

My setup only caters for my interest as a hobbyist and photo restorer who wants a bit of calibrated home cinema on my desktop with headphones in my little rented room, but it does not compare to those in the industry with a proper HDR pipeline who are better suited to evaluate this beta. My tests are going to be limited to 300nits range, while the pros can no doubt do much better.

[topaz257 (Michael Karg)]

Great!
Will be intereseting to compare it to NVIDIA RTX auto video HDR available through VCL video player.

High Can See Clearly Now: AI-Powered NVIDIA RTX Video HDR Transforms Standard...

[reiner (nvidiajetson4christmas)]

as mentioned above, most video players can´t handle high bit depth playback or interpret the embedded HDR Data or even tonemap everything.

Try VLC for a quick and dirty solution.
MPC-HC needs some tweaking.

[riccot]

quick test on 480p looked promising, but I can’t use this model with 4k content and not even 1080p. it eats up all my vram (12GB) and slows the pc and processing to 0.1fps.
I hope this can be made to work with less than 16 or 20gb+ of vram.

720p: 4fps
1080p: 0.8fps
2160p: 0.1fps
(RTX 3080 12GB)

[deepzone]

logsForSupport.zip (406.9 KB)
I keep getting errors when previewing

[JesusSheep]

@tony.topazlabs Could Use Dolby Settings In Divinchi Resolve Studio P3 600 Nits D65 600 Nits.
Sony TVs have 600 nits plus some Vizio 1000.

[SkodreragGaming]

I deleted this alpha because it’s super slow to export. I will wait for them to put in the release version and which will be fast with nvidia tensort

[Grammy]

I played around a bit with a video clip from Avatar (2009) in 1080p and, to be honest, I’m quite impressed with the results, especially considering it’s still in alpha status. However, there’s still a lot to be done. This includes:

• Speed: Even with an RTX 4090 and without further enhancement, I’m achieving around 5 FPS. There definitely needs to be some improvement here.
• Preview: The fact that the video cannot be displayed in HDR in the comparison view is a major issue. This makes it very difficult to compare the results with the SDR video.
• Nits Specification/Selection: You can see a difference in the converted video, but without comparing it to the original video, it’s very hard for me to determine which value leads to a good result.
  Visually, a value around 400-500 nits seems to work well for my text clip from Avatar (2009). A lower value around 200 doesn’t seem to have enough “punch.” Values around 600 and above cause bright areas to be extremely overexposed, especially skin tones, which appear unnaturally pale.

[JesusSheep]

ColorSpace Error

Alpha Only Works on H265.