UPDATE 2 - VEAI Model
Blending my idea with Kyle’s & Khan’s together in a more simplified way, here it is below:
I would want VEAI to be able to do this:
- Automatically detect the “background” or manually select it
- Automatically detect “people” as separate individuals or manually select it
- Automatically detect “animals” (is optional) or manually select it
- Ability to “select the AI model” of your choice for each detection or selection.
- You can choose an AI model you have trained for that specific detection or selection
- Users can make a custom AI model that stacks off of VEAI models that have already been created and can be used as a base to make custom trained models or start from scratch.
- Name our custom trained models
- Recognition of individuals based on custom trained AI model datasets and uses those custom trained AI model datasets.
- Progress bar of custom trained model, to let us know if the video footage or pictures we used have exceeded their use case or not or the model still needs x amount of time to train off them.
- Ability to share AI models with the community that we built with the topazlabs software, this would decrease the amount of work on Topazlabs and the passion of the community will help bring in unique AI models that work best for certain things. It would be cool if we could even collaborate on these AI models and make them better and Topaz labs can work with the community as well to more fine tune things since they have the programming background.
In practice, say you have high quality photos of your favorite sports team, person, or scenery, but the video footage is lacking or missing details. You can easily select an AI model you have trained for the detection or selection.
Example 1: A person’s face or quality is distorted, you have a trained model for that person’s face or quality to be restored with the correct detail. So say normally if you were to upscale, the smile will be upscaled with the distortion. Instead of that, the correct detail is added as well as being upscaled.
Example 2: The background is missing detail, say leaves don’t appear distinct and blurred, well you can use an AI model you trained for that specific scenery to restore the detail.
Kyle’s idea bout a broad scope AI you can train in general still has some benefits even without the detection and selection thing I mentioned, as he currently uses ESRGAN to do this now for sports videos. It would be nice to have VEAI a paid app simplify this for it’s userbase.
-----MY ORIGINAL IDEA------
Feature Request: AI that recognizes distinct people and or background, allows high quality input data of those individual people or setting, in order to “correctly add detail” to the subject/enviroment that is in low quality (While only storing this data on an individuals computer not uploading it to the cloud).
For example, some faces in low quality videos will be upscaled and look more like rough painting of the person, the eyes don’t look the same, nose, hands, arms, even clothes at times, sometimes the skin gets too smooth and looks unreal.
Solution in practice: The AI model recognizes a person being distinct of other persons. You are prompted to add data for that AI model to work with. So say you have a low quality video of a person but high quality pictures of that person from the same event. You can use the images to upscale with the “correct details” of that person or also if you have generally high quality images of the person you can allow the AI to train into learning that individual and then properly add the correct detail to the video. So lets say person is far away, the image is blurry there face is distorted. Instead upscaling the distortion, correctly add the detail of the face with this model by allowing the user to add input data to help the AI correctly add detail for that specific person in the video. It would be cool is if you could do this for multiple people at a time. But even one person at a time would help a lot with making this product extremely helpful and valuable to restoring videos with low details.
One last note, imagine having a video where the smile of a person is distorted, but you have the correct smile of that person in a high quality picture. The AI model then can give the correct smile rather than exaggerating the distortion. It would be great to be able to add details into a video that arent there based on user input data the AI can work with.