Wild idea to improve faces

Hi,

I don’t know how feasible this is but I have been playing around with deep-faking tools and they work quite well.

Would it be possible to use a similar approach to enhance faces in a video.

So a video may include a lot of frames and scenes of a person from different angles and distances.

The idea would be to have a model that runs a training program on the video, recognises the people in it and gathers detail on the faces before running the upscaling model

this way the model could apply detail from close-up shots onto scenes where the person is further away.

Not sure how possible this is, but I think it’s a cool concept.

What you can do is load several individual frames into Photo AI, restore the faces and export the frames again and re-encode it to video and then process it with Video AI. It’s a bit tedious, but at least you can do it that way.

1 Like

I think everyone wants AI enhancement to eventually get to this level. I’m not sure if Deep-fake models can be ran on consumer hardware just yet.

Indeed I have seen other vendors do it and You end up Do you have a few faces or multiple faces in this video . Then You have to think about this then after 5 days end up with a result where the faces seem to be coming out of a game made in 2003 .

The only thing you can say is that Topaz might license some of this from LG that does also AI and has apparently some magic in its software in the latest software and does generate natural faces.

That depends on what you mean by consumer hardware, there is a free and open source python project hosted on GitHub GitHub - iperov/DeepFaceLab: DeepFaceLab is the leading software for creating deepfakes..

Sure it takes time to learn how to use it and the model training takes a long time, but the results are quite impressive with the right content. Sure a real beast of a PC will do the job faster than a weaker PC.

I think it would probably be easier to enhance a face that is further away with details from a close up shot of the same face, rather than replacing a face with someone else’s face. And you would most likely get way better results as well, because the dimensions, skin tone and what surrounds the face should still match.

Well that’s pretty cool. So maybe we’re closer than I thought.