Would be amazing if we can have a model that can take a video and initiate lip / face movement to match an input audio in a realistic way as if it was recorded like that. Just an idea that I know I will be using all the time.
I believe that this could possibly work well for dubbing video into other languages as well. However, I believe that it would be out of the TopazLabs scope.
What a great idea. Would love to be able to see a singer singing something different to what was already recorded.
May already be patented by this company. If not patented, they’re ahead: “Los Angeles-based movie-making AI firm Flawless, and its TrueSync technology that analyzes the entirety of an actor’s performance, utilizes deep learning to create a volumetric 3D representation of their face, and then adapts their mouth movements in the footage in post-production.” Source: First AI 'visually dubbed' feature film hits US theaters this May
