Nvidia new AI upscaling tool "RTX Video Super Resolution" 🥳

The new tool uses AI to improve the quality of video streaming by removing blocky compression artifacts and then upscaling video resolution, 1080p → 4K upscaling.

The new feature will work within the Chrome and Edge browsers, and requires an Nvidia RTX 30 / 40-series GPU.

Article from PC World:

2 Likes

It’s a very basic statement for no details at all. No usage guidelines, what type of video will it upscale, how does it deal with faces, hair, animals, low light etc., etc.

Have you tried it?

The most amazing thing is that this upscaling is applied in real time to the video… obviously you have to see the final quality, but it is sad then to compare with TVAI which is very slow to upscale to 4K instead!

1 Like

Its made for in Browser upscaling.

:rofl: you could try to upload your compressed video to YT then you could upscale the compressed stream and record it.

I say it is not Magic and TVAI is still superior.

OK, but it’s still a 4K upscaling of a video stream in REAL TIME!!! (repeat with me… REEEAALL TIIMEEE)… for me it’s still something stunning if the quality is really improved, TVAI-style.

Then of course, you have to try it by hand and make the comparison in the final quality, but there is no doubt that Topaz definitely needs to work on the performance of its upscaling, as the RTX 3000 series GPUs and Intel i7 CPUs from the 10 series onwards are not exploited properly, I see it with my own eyes with my configuration (i7 10700 and RTX 3070)!

2 Likes

To my eyes its a INT8 denoise and sharpening filter.

That means that the GPU will not taxed much and is very fast.

The Quality of the source Video in the comparison is already very good, you dont see that much of compression.



Here is a video about the AI upscaling in Photoshop by RTX.

We know that it is years behind that one of GP.

1 Like

I’m going to go in the opposite direction and suggest that Topaz improve upscale quality even further at the expense of speed, but increases to the speed and efficiency on the underlying hardware are always welcome.

Currently NVIDIA’s RTX Video Super Resolution feature is browser-only (and just two browsers at that), but I think NVIDIA will expand it to media players in the future. Once that happens, its convenience will put it at a great advantage over other solutions. Even if Video AI’s speeds match or exceed real-time, one still has to spend time creating the upscaled video before watching it.

I don’t think RTX Video Super Resolution will be as good as Topaz across a variety of videos (and similar skepticism is shared by folks on Reddit, especially due to people’s experiences with misleading marketing from NVIDIA and other tech companies) overall. In particular, I wouldn’t be surprised if NVIDIA’s feature focused mainly on gaming and streaming videos at first rather than old home movies and other kinds of videos.

In my opinion, I think Topaz should make model quality its single top priority, aside from bug fixes, in the near future, especially given the lack of updates to upscaling models over the past year. If I’m going to spend several hours (common) or multiple days (rare but it happens) upscaling a video, I want it to be of the best possible quality. I think a future version of Video AI could thrive in two different directions, examples of which are described below.

  1. Strongly emphasize video quality improvements. Enhancing a video may take a lot of time even on powerful hardware, but you’re going to get considerably better enhancements than NVIDIA’s or other solutions. Moreover, the large number of updated models and other features such as increasing the FPS means that Video AI is a great solution for almost all of your videos.
  2. Add “media player” capabilities to Video AI, including speed controls and subtitle options. On suitable hardware (and NVIDIA’s feature also requires powerful GPUs), Video AI can upscale a video fast enough so that you can play the upscaled version with sound in real-time within Video AI. Now here’s where this feature can be integrated into Video AI’s current approach. Presumably if you’re watching a real-time upscaled video in Video AI, then there’s a reasonable chance that you’d also like to have an actual file of that upscaled video for one reason or another (e.g. other devices, video editing, you don’t want to tax the GPU every time you watch the video). So Video AI could export the upscaling in the background, and on even more powerful hardware it could use a different model that is slower but higher quality. The idea is that after you finish watching the video, the upscaled version is waiting for you as a video file.

These two options are not mutually exclusive, but do require work, and if I had to pick one I’d choose 1.

2 Likes

I very much agree with what you say, but highlighting this new technology from NVIDIA serves more than anything else to make it clear that the margins for improvement in the speed of export and upscaling for Video AI are very much there, given that programmes such as this one being developed by Nvidia or even the latest generation TVs are able to do an excellent job even reading the video in real time and at the same time do an excellent upscaling.

Obviously Topaz must continue to focus on the quality of the AI models, but at the same time obviously don’t forget to make the most of the PC hardware with the new NVIDIA cards that have all the technology to do great things (from the RTX 3000 series onwards).

I have to say though that with the latest BETA here on the Forum, the use of my 3070 has improved a lot, with much better export times!

1 Like

Yes. So called Super Resolution in Photoshop seems like its just interpolating pixels a bit better than some traditional methods. It does not differentiate between noise and texture or sharp and non sharp elements in the picture, its just treating everything the same it seems. Topaz is similarly fast now in GigaPixel and much better results.

Actually, some video players like Media Player Classic use GPU supported shredders to sharpen, de-noise, upscale video in real time as well. Maybe nVidia does it a bit better, I’m not sure I would have to compare, but simple and still quite good up scaling in the video player in real time has been around for years now. You can check it out, it’s free. Try Media Player Classic and their shadders. I often use it on by default for lower res videos. “Chroma upsampling”, “Sharpen Complex 2” etc. It’s not AI but it’s pretty darn good for older tech available on slower machines and for free. You can stack shadders or build custom ones for your needs.

1 Like

They are just simple upsampling using interpolation not AI. :sweat_smile:

Modern TV have built in AI processor to upsample low resolution video to 4k in real time. They can remove compression artifact while preseving film grain integrity. They also use deep learning to detect faces & objects, and able to seperate the foreground from the background then applies different tone mapping to produce more real life image.

Current gen RTX40 GPU are so much more powerful than those AI-processor find in TV. So, they should be able to do the same AI upscaling in real time as well.

Here is an example from last generation LG TV by JTC at 11min 13sec. Of course the result may not be as good as VEAI, but it is pretty impressive consider it is process in real time.

I’ve stated that in my reply, genius. My point was that AI from nVidia did not impress me much over what I was able to get with older non AI tech. But since AI is written on everything to the point of causing inflation in the marketing departments… you should not jump on the bandwagon just because it say AI. Nor do I want taxing AI stuff for watching a video with only marginally better image. I want to run low resources app on an old hardware and get decent results. I also don’t use Topaz Video AI for every video, because some stuff just is not worth the hard drive space, processing time and hardware needed to upscale it. But if you get so excited about anything with AI marketing label, by all means. Knock yourself out.

What’s the price, over 2000 Euros? That is a lot of money for upscaled video. Mabye you have extra change laying around somewhere in your pockets, I don’t. For 2000 Euros I get a proper 4K camera or workstation that actually can make money, not just spend it, thank you very much.

1 Like