Seperate models for denoising, sharpening, upscaling, removing compression artifacts

I believe having models that only do one of these things at a time would allow for more control when processing a video. Especially a model that only denoises the video and another model that only sharpens it would be great. Perhaps even one sharpen model to remove motion blur and another model to fix out of focus blurriness.

To me it sounds like you’re requesting that they make the Proteus model.
How does it not fulfill your request?

The biggest use in VEAI is upscaling, but not like any other method of upscaling—as in: it does more than just make resolutions bigger. It applies denoising, sharpening and removes compression artifacts depending on the model.
Why?

Here’s a thought experiment.
You take a 1080p movie and convert it to 720p with high lossy compression.
Use VEAI to clean it up but not upscale it. It will look better, for that resolution.
Use any other non-AI based program to upscale it.
Compare it to the original, it will inherently be more blurry and less detailed than the original. This is the limitation of other methods of upscaling.
Use VEAI to upscale it, instead.
It will still have differences compared to the original, but will be more detailed than the non-AI based version.

(In case you don’t know. That’s how you train an AI. Convert a high quality source to low quality. Mark what one is correct. Repeat that thousands of times with various videos and compression types.)

Even if I set all parameters of Proteus to zero and resolution to 100% the output is still visibly changed. If I want to just remove a slight amount of noise Proteus is not the best tool to do that.

And i know how AI works, that is exactly the reason why I believe a model that does only one specific task can outperform a model that does multiple things at the same time.

Fair enough. I was mislead by your desire to “fix out of focus blurriness”. Even in the original source, it will still have the out of focus parts blurry. There’s no way to train for that… Unless you mean current models are adding detail where it should be blurry.

I agree that more models with granularity like Proteus would be nice. I have found movies that I just can’t get looking right in all aspects. The sad truth of that is that I would probably need to take the time to find the best model for each scene.

I meant if the object that should be in focus is slightly out of focus, the AI should be able to fix that, not the background. This is often the case when working with older video that was not shot by professionals.

Or have three models at the same time, one for denoise, one for decompress and one for upscale.

Would be interesting.

1 Like