The idea is to create a new mode/model that will upscale and increase picture quality using a higher quality picture (one or maybe more) of the same subject (human figure or face, an object or a scene) as a reference or guidance that is effectively telling the model how the high quality picture should look like.
How it could work:
Guided upscaling with reference
The software could use a higher-quality reference image of the same subject or scene to learn the specific features, textures, and details that the user wants in the final upscaled output. This way, the AI wouldn’t be just predicting or trying to create details based only on its trained dataset but also matching them against the provided high-quality reference.
Enhancing with feature transfer
By comparing the high-quality reference with the lower-quality version, the model could recognize common important details or patterns. It could then use this information to more accurately fill in the missing or blurred details in the upscaled version.
Such model would be a great help and a timesaver, especially when:
-
upscaling low-resolution variants of a photo using the final edited or raw version as the reference
-
enhancing and/or upscaling scenes rendered in low resolution or with low number of samples (to shorten the rendering time) using high resolution/posprocessed render as a reference
Examples of Related Technologies:
Deep Image Prior
https://github.com/DmitryUlyanov/deep-image-prior
https://arxiv.org/abs/1711.10925
Perceptual Losses
https://arxiv.org/abs/1603.08155
P.S. Thanks for all the great work you’re doing!
Regards,
John
