Convert 2D video to 3D stereoscopic using AI

Would love to take some 2D footage and have it converted to 3D. Ai would be ideally suited for this, learning to create depth maps, based on existing 3d movies.

7 Likes

Cool idea if it works better than current ways of doing it.
The amount of people that care though—is really small. There is no way enough money would be made back after spending any money on the development of such a model.
If some rich individual wants to fund it, it might happen…

2 Likes

why do you give them free business tips? Companies pay lots of money for that.

If what I think is common knowledge, can actually be sold to businesses… I should rethink my life. (But I won’t.)

Anyway, I want Topaz to succeed. I don’t suspect them of actually being able to give this thread suggestion much heed. To make a model for it, you would need thousands of sources that were shot with stereoscopic cameras. Not much of that exists. The majority of existing movies that playback in 3D were just filmed with normal cameras and later had 3Dness added. It would be easier to make a model that mimics that process, but most people agree that that can turn out very artificial looking.

I also want to inform that most people find modern 3D movie technology to be unimpressive. Though it can be cool, I really wish they would have spent all that money on developing holograms.

Hi - ex-stereographer here.
I worked on many films during the 3D craze 10 years augo. Couldn’t agree more with you, ForSerious. But I also really agree with nalamanos-170537: a software like Topaz could do wonders in automatic 2D-to-3d conversion. Today automoatic systems work poorly and 2D-to-3D conversions are done by hand, most of the time by huge teams in India or China. Price tags are pretty big (much better than 10 years ago, though).
Topaz could probably find a marketr by converting Hllywood flicks automatically. But yeah - the team should proabbly focus on something else.
If the idea tickles you mind and need some advice/an eye with expertise in the field, feel free to contact me, I’ll be happy to help. More info on our old website: https://www.parallellcinema.com/docs.html

Alexandre Saudinos
NEOSET

2 Likes

I can easily pay twice the price of the software to have this feature decently added… :wink:

2 Likes

I’ll give this a big thumbs up of an idea.
I found this article by a guy that coded a way to do exactly this (How to Convert a 2d Movie to 3d. A tensorflow implementation of DenseNet… | by Alexander Kharin | The Startup | Medium) and you can run it on a Google Collab machine. It’s not amazing but it is a good enough 3d effect that I have spent hours of Google’s machines on a few videos.
There exists small implementations of independent developers but all eventually fall flat, probably for the lack of budget and market.
Would love Topaz to give it a shot with a 2D to 3D SBS model

2 Likes

Everything starts with an idea. Jobs got idea of iphone, others OLED, others a software like topaz. Yes, it is valuable. Don’t give them advice for free. They sell this software for A LOT of money.

I told them (if management even bothered to read it) to NOT make this model because I think it’s a waste of money and could potentially be the end of them.

Now if they were to make a model that reads a movie and creates 3D objects and landscapes all textured to be opened and rendered in a 3D program like Blender… there is loads of value in that.

I cannot take credit for that suggestion, since nalamanos-170537 already suggested they do something with 2D to 3D. (They just blunderingly used the word stereoscopic in the title. Misleading any readers to think about the unimpressive, now gone from theaters, only good when done on computer generated animations, still owned by 70 people today: 3D TV screens paired with 3D glasses.)

1 Like

While I don’t expect this to happen, I would LOVE if it did. I’d happily pay up for another app if it doesn’t fit within the existing Video AI.

It all boils down to cost effective and ROI / P&L. I don’t see this being cost effective from business prospective for Topaz. the demand is just to low verses the effort needed for this to become applausable footage , not to mention huge amount of man days / weeks / months of on going support, as this is an extremally complex AI. All this costs money , and a lot!!!
If this would ever come out as a 2D->3D AI, it should be an independent Product or the core Video AI product would cost double the current price if not even more.

Hey Everyone, i would have loved Topaz to do an AI-based 3D conversion, as they certainly know how to process frames and their AI Models not only enhance within a single frame but use the information of adjacent frames to extract structure information/moving objects etc…
HOWEVER… take a look at https://www.owl3d.com/
One or two years ago these guys came up with some really rudimentary AI-stereo converter, which evolved a lot during the last months. It seems to be based on models for monocular depth estimation (like Midas / Leres) - which flicker a lot when processed frame by frame - an add a post process for temporal consitency to that so that eventually you get a smooth and really good depthmap and left/rigth views. It wokrs perfect for movies wit a lot of close up shots and a lot of movement (like any action movie) but struggles with more detailed scenes. And of course, a depthmap cannot represent transparencys or blur - its just like projectiong the image on a deformed rubber surface - but good enough to get a decent 3D effect. I dont want to advertise for OWL3D, but as Topaz guys, i would definitely say “Challenge accepted” as there is a lot potential to extract structural information/image composition form a motion picture and get much better sharper depthmap and object separation than just “temporal smoothing of a series of monucular depth estimates”.

1 Like