Is Possible Virtual reality video improvement with Video Enhance Ai?

Hi, i am working with old vr 180º 3d and i want to know if video enhance ai could handle that.
An example, 3200x1600 15mbs to 3840x1920 25mbs, and improve definition.

I tried your trial but it created a lot of artifacts.

Thanks in advance.

P.D. Sorry my english, i´m not native.

Very ulikely. Maybe you get luck, if you downscale it first. The improofments at 1080p and higher are often negligible. With that said, the software is young. It will improof over time.

But it is not trained for VR (Highly distorted input material). Maybe a solution could be, to remove the distortion and try again and add it back. Or scale it down first. Scaling it down often increases the final details - of course some details get lost in that process.

In 5 years, this VR Video will likely be extremely easy to upscale to insane quality. But for now…

What Model did you actually use? And how much compression does the input video have? Very insame compression can lead to strange artifacts, that cannot be removed (except by hand which is no solution for video)

Hi, they are familiar videos that family lend to me. They want to improve resolution, i dont know exact model of camera and i am not very trained on video editing.

I look for an automated software that could do it on a simply way.

Then i wait more time to better software, Thank you.

Well, you still can try to scale it down to 50% and than upscale it to 200% or 400%. This can actually sometimes really improof quality. Do not ask me why, I guess the AI is not trained on already high detail input. I mean, it is really hard to get even higher resolution videos to train the AI. This might be the reason.

If Topaz will continue to improve, it will likely be great in just a year from now. It is usable for many things and it actually creates far less artifacts than Gigapixel AI (I used it to upscale Video frame by frame, but Gigapixel is not trained to render frame by frame - it does not know, that frames belong to each other. It is just meant to batch process different images like out of the cam or from the web.).

Good luck with the 50% scale down and than scale up attempt. Might work for you. :slight_smile:

Thanks, i will try it with a trimmed one.

Thanks a lot again.

Another question, whick ai do you use, gaia or the other one?

Thanks in advance.

I only use Gaia HQ. I got the best result (by far!) with that for any input I tried. I tried Live action stuff and even 2D animation - both looked better in my opinion. Less artifacts, more details I would say.

Thanks, if you watch video in 2d it works flawless but when you put your headset (vr) artifacts begins to appear.

I think that devs need to improve their ai software for vr video, no 180/360 flat ones. A lot of people with a cheap vr camera buy their product because a good vr camera costs a lot of money.

1 Like

Hi Gideon

I am very interested in this topic also and try to enhance 360 image and videos with topaz…

You cleverly suggest

How would I do that? Any software to suggest? Or photoshop script maybe?

Thanks :pray:

As far as I can tell, you can use ffmpeg to remove lens distortions. So you could exploit that. Some video tools also have some lens distortion effects.

Vegas Pro can do some lens distortion fixes. This should work. The extreme distortion could however be a problem, since the effect might be limited to a certain ammount of distortion. I do not remember the effect name, since it is long time ago. Just search for lens distortion fix vegas pro or similar. Other tools most likely have the same effects/plugins.

These effects are used to fix the lens barrel effect or how one would call it.

I am happy with the results of enhancement from 5.7k prores vr file to 8K.
The video were taken from Insta360 One R 5.7K 30 frame, stitching in Insta Studio becomes Prores422

Using Imac I can only choose Artemis HQv8, with custom setting 7680x3840 for fcpx setting, scale up 133 percent. The result was good, I like it, the sharpness is slightly better, since some said you need 16K to have a good VR resolution and that was what I’m looking for … At least 8K will be better in my calculation.

But you have to put the equirectangular back to the file since fcp does not automatically recognise the file as 360, and you have to stitch back with scaling x factor by 100.25 percent

Well that was my experience with Video Enhance AI, hope we can make it to 16K ! :sweat_smile:
I don’t have a professional 360 camera with 6 or 8 lenses, maybe the results would be better.

A note : Youtube will only play 8K file on Google Chrome, others will stuck up to 4K

I have noticed, that details need to be sharp already, or they get detected to much as blur, which gets just scaled.

If the input video quality is flawless, the upscaling works quite nice.

I have to correct you on that. Youtube plays fine in Firefox with 8k60 with my GTX 1060. Hardware-decoding is at 85%, so it is a stable play. Hard to buffer honestly. I have 50mbit/s, but sometimes youtube only provides like 20mbit/s or less depending on the usage (tested on other sites, the 50mbit/s are stable). So 8k on youtube is not really working well at all.

But if you get it to buffer, it plays well - at least if you have hardware decoding. I do not know, what the requirements are, to get the decoding. I have windows 10 and a 1060, which works. No idea, if integrated solutions work with that at all.

Ah you are right, my mistake, won’t work 8K youtube in Safari it seems then … and yes it’s true it will be hard to be able to see 8K on Youtube unless you have a good hardware and connection. I am using Imac with Radeon 580, good speed internet and can watch stable youtube 8K.
I haven’t touch Gaia caused I am using AMD GPU, and it took forever to render.