Downscale model to sharpen image

Dear community

Is there any model that perform well for downscaling (and sharpening without aliasing it) a 8K master ?

I did what Hollywood do with movie which consist in scanning the master in 8K, then downsampling in 4K to sharpen and fine tune it.
What I did is upgrade it with TVAI, for a kind good result. Now I have to HDR it, and sharpen it but with care, because bilinear downscale filtering often leads to aliased image.

I have seen some users use Theia with the Fine Tune Detail parameter and increase the Sharpen slider to some success.

1 Like

Thank you for replying. That was the Model I focused on, but also got good result with Proteus

Will compare both with A/B

EDIT: I finally settled on Proteus, but I’ll try Theai in my next encode
Proteus is so good at restoring very small BG things that have been scaled up to some hexagonal shapes (head for example). It rebuilds the head and body flawlessly, it’s cool

1 Like

(be sure to check out nyx3 with high compression reversion, sharpness, and possibly, degrain value)
it’s beautifully neutral

2 Likes

I passed over million of hours testing all models :slight_smile:
And indeed Nyx is one of my favourite
Both Proteus and Nyx are 5 stars.
Theai and Gaia are in bit a pain to tweak for allmost invisibles reults.
Rhea, still don’t know what it improves ^^

Also fan of Aion

Will post soon samples of my tries as my stuff is starting to be in place and optimized.

Thnaks for your reply and we allready talked about it, I found some very good params for ST, stay tuned

In my case, video is CG, so there’s absolutely no argentic noise nor grain on the picture, it is at most blurry. But as I was learning a bit dwonscling algos I found this that could be very interesting (not considering the app, but explanations and maths involved)

Then I considered playing directly my 8K master and post-process it with

  • Lanczos3
  • Calmut-ROM
    (hesitate … have to full view)

The result is very good and save me a lot of rendring time.

Resolutely, upscaling video from 1080p to 8K with a x4 intermediate gives awesome results, but the processing time is a pain in the in encoding time.

Format: HEVC
Format/Info : High Efficiency Video Coding
Format profile : Main@L6@Main
Codec ID : hvc1
Codec ID/Info : High Efficiency Video Coding
Duration : 1 h 33 min
Bit rate : 230 Mb/s
Width : 7 680 pixels
Height : 4 320 pixels
Display aspect ratio : 16:9
Frame rate mode : Constant
Frame rate : 23.976 (24000/1001) FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bits
Scan type : Progressive
Bits/(Pixel*Frame) : 0.290
Stream size : 150 GiB (100%)
Color range : Limited
Color primaries : BT.709
Transfer characteristics : BT.709
Matrix coefficients : BT.709

Don’t know what sh** I did but tone mapping turned into yellow/red :confused:
I used HEVC because was unable to output to FFV1 8b 4:2:2 or 4:4:4 to keep good small and smooth degraded. Am I missing stg ?

1 Like

Theia (nor other model) did not give the result I would expect.
All models smooth image too much :frowning:

Going to downscale to 4K first with ffmpeg and Lnaczos3 algo. will see what it gives, no quality loss.
Then I will AV1 compress it.

I would be played to keep 8K files, but 295 Mbps for video stream is too much for my NIC. My x86 runs it fine but not SHIELD. The advantage is that I can choose real time downscaling methos with Kodi and there’s much ! But the source file is so gigantic … 280 GB. I can’t jump to chapter or scene easily, even with 640 MBps file server.

i sharpen with high pass filter linear light method in davinci resolve, maybe a little texture pop if needed, then denoise with “fast” (better or enhanced is not strong enough). then import into topaz, focus fix standard (proteus 3), just by turning this on while all value is at 0, it will take care of the remaining noise. set dehalo 10 to dehalo the dark edges introduced from the sharpening before.

for 4k you can use focus fix strong instead.

Ok ! Thank your for these detailed advice.

Will give it a try :slight_smile:

to add more details to my comment above,

below video at 2:00, how to create high pass filter in davinci

in the video its used for dehalo, but you can use the same process for sharpening, both are going to be needed later,
to sharpen, move the slider up instead of down, blend mode change to linear light / vivid light,
this sharpening method beats every sharpening tool thats available in davinci.

this is my spatial nodes in davinci,
abc

i first sharpen the video until objects gain a new level of clarity at 100% view size,
then i use the dehalo node to tame the white halo / shininess / oversharpening created around hair from the process before,
then finally denoise with fast (this method produces a lot of noise so better and enhanced wont be strong enough)

below video starts at 5:50, is a nice trick to get perfect denoise and dehalo result
the video is for denoise but the same trick applies to dehalo too, so you will always only dehalo just the sharp bit

then i run the output in topaz to AI dehalo a bit more.

it may or may not give better result than doing it all in topaz but its another method to try.

2 Likes

There are many resizers and it’s hard to tell if any are better than another.

DPID claims to preserve detail when downsizing. There’s some proof in their paper. The documentation on the lambda parameters is poor. I keep experimenting with DPID and various lambdas but have yet to see any difference.

After reading their paper again, it seems that lambdas less than 1.0 may sharpen more while downscaling. It could be my downscale factors are not large enough for DPID to show any visible difference. The largest downscale I’ve done is 8K to 1080. Their paper has greater downscales.

The DPID paper:
https://dl.acm.org/doi/pdf/10.1145/2980179.2980239

The plugin:

Lots of discussions on this a few years ago:

Prior discussions:

The last link has a test with AreaResize which I need to research.

And there’s this, which appears to be the same idea as DPID:
https://johanneskopf.de/publications/downscaling/

Check out its pseudocode link. I’m thinking that algorithm or DPID (maybe with way lower or higher lambas) may give us results (additional detail) at the scale factors we typically use with video.

The details are all way over my head, but I do understand the basic concept: upscaling higher than our final resolution generates additional detail, and that detail should not be tossed when downscaling.

An AI model for downscaling is a good idea and may be superior to any algorithmic method.

2 Likes

Very interesting and professional post
Thank you, going to investigate it !