I was thinking of something along the lines of a TopazLabs Benchmark Toolkit since the programs support both AMD and Nvidia GPU’s. Yet AI is quite a unique workload and TopazLabs has realworld usage products.
The program would simply consist of a selection of selectable AI models and a variety of sample images and video clips ranging from different resolutions.
There wouldn’t need to be any output or preview window since we’re strictly looking at performance here. However this doesn’t mean we don’t need settings.
So for things like do we only want to look at 2x scaling, 4x scaling, both, or maybe higher?
What AI models do we want it to be using?
Are we only looking at image processing or video processing? Both?
Do you want the program to record and output data? Maybe you’re just trial running the program to see if your PC can do such a workload if it’s not going to be free.
And ideally there would be a cloud consolidation of data for anyone to access and compare.
One potential issue would be the updating of AI models. But it could potentially be solved by keeping multiple versions available to use. Of course AI models are downloaded per request so a user wouldn’t have a bunch they don’t plan on using.
I’ve been very keen on the performance difference between cards. But trying to get a wide variety of community members with different hardware utilizing a standardized set of media across a range of different AI applications that would have to be individually obtained is a very difficult thing to do. Let alone that users may have no plans to ever download and use up the trial period for a product they don’t plan or currently don’t plan on using.
