Is there a way to use newer models on older versions of the video AI?

I greatly prefer the GUI of v3. Is there a way to run the newer models on older versions of Topaz Video AI? Or, conversely, can the current UI of the program be changed to one of the other layouts? This would be a useful feature to include in Preferences.

I’ve tried with the CLI. All the model options are hard-coded. Old versions did not predict the future and know about versions that didn’t exist at the time of release. Part of the work done in new versions is adding the ability to generate commands for the new AI models.

They would have to redesign TVAI to be more internet reliant. Change it so that it can update it’s list of models, and how to generate commands for them, independently from the version. …And that goes against their current business model. To some degree, new models are supposed to be something that should drive sales. Get old users with expired licenses to come back and pay again for the new models.

1 Like

Well this isn’t 100% accurate. In the GUI, it’s true, they’ve hard-coded which models are available. However for the CLI, it relies on the model-metadata files and the parameters you specify for ffmpeg.

I just did a test on 3.5.4 trying to use Protheus v4. I just dropped the “prob-4.json” file from TVAI v4 into the “models” directory and it encoded just fine using Protheus v4.

ffmpeg -v debug -f lavfi -i testsrc=duration=12:size=320x180:rate=15 -pix_fmt yuv420p -vf "tvai_up=model=prob-4:scale=2:w=320:h=180:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=8:blend=0.2:device=-2:vram=1:instances=1" -f null -

...
CRITICAL:  Cannot open zip file at: c:\temp\models\prob-v4-fgnet-fp16-256x352-2x-ov.tz
INFO: Proxy Settings: 0 hostname:  port: 0 username:  JSON: {"enabled":false}
INFO:  RESULT for https://veai-models.topazlabs.com/prob-v4-fgnet-fp16-256x352-2x-ov.tz is 200
INFO:  Compressed Buffer for prob-v4-fgnet-fp16-256x352-2x-ov.tz verified
INFO:  putting fetched file into: c:\temp\models
INFO:  Created temporary file at "prob-v4-fgnet-fp16-256x352-2x-ov.tz" with 32586551 bytes
INFO:  Loading filename prob-v4-fgnet-fp16-256x352-2x-ov.tz fgnet-fp16-[H]x[W]-[S]x-ov.tz ["out"]
INFO:  Total Devices: 1
INFO:  DEVICE DETAILS: 0 INSTANCES 2
INFO:  PARAM DEVICE: 0
INFO: ModelBackend loading from file
INFO: CACHING MAY NOT WORK"c:\\temp\\models\\prob-v4-fgnet-fp16-256x352-2x-ov.tz"
INFO:  Reading model file duration: 498.147 ms
INFO:  Network Name: torch_jit with 412 layers
INFO:  Inputs: 5
INFO:  Input name: hr_prev
INFO:  Input name: lr_curr
INFO:  Input name: lr_prev
INFO:  Input name: params
INFO:  Input name: scene_change_threshold
INFO:  Outputs: 1
INFO:  Output name: out  == out
INFO:  Loading time for model file: c:\temp\models/prob-v4-fgnet-fp16-256x352-2x-ov.tz is 521.631 ms
INFO:  Shared PTR: 2
INFO:  TOTAL INSTANCES: 2
INFO:  Init backend runner 2
INFO:  Auth Check Watermark: 1112023-12-012012-10-10
INFO: Applying watermark
INFO:  Video Processor setup successfully for model prob-4
...

That said, and before a Topaz product PM goes ballistic and tells the team on next morning stand-up “Alarm, Alarm, Drop everything! Patch this out!” Thus disrupting any actually useful work the team is engaged in, I’ll clarify.

  1. This should be seen as an unsupported hack that just leverages knowledge about the product’s architecture. It may or may not work in the future. The Protheus v4 example happens to use a model that does not introduce any new hyper-parameters (same interface). To the TVAI engine (that runs in ffmpeg) all enhancement models look pretty much the same, which is why Topaz can release new models without having to change code (a good thing for them and us!).
  2. Cumbersome workflow. You’d have to use the (3.5) GUI using a different model creating your clip’s configuration, and then bring up the ffmpeg settings for the export, swap the model manually and run the export via the CLI.
  3. You would not be able to preview your settings using the new model in the GUI, which is sort of the whole point of the GUI to begin with.
  4. There is no guarantee this will continue to work in the future. Especially after Topaz has read this comment ( :open_mouth: :wink: )
3 Likes

Awesome. From the CLI point of view, there should be zero difference in the result. So yeah, that’s a lot of work to avoid using the new UI.

Agreed. However like OP, I’d really want the old 3.5 UI back. I find the v4 UI incredibly confusing and annoying. I curse at it all the time. The only actual improvement they’ve made is caching the previews so that the actual swapping between them is quicker. However that improvement is completely offset and nullified by the additional steps required to actually find and choose the different previews (context-menu hunting).

What I hate about the new UI are:

  1. It automatically zooms in the timeline to wherever a preview was last made, disorienting me and having me waste time trying to zoom out again all the time, as I’m going through a clip trying to preview a given setting at different locations of the clip.
  2. Requiring additional interactions to reach the preview-latest-preview-with-external-player (buried under a context menu puzzle, after you’ve gone through the puzzle of finding the latest preview in the preview-list to begin with!)
  3. The entire preview list redesign where previews are nested under a given clip in some yet-to-be understood manner, requiring me to spend time over and over to find the latest preview. I just want the linear preview list back, where I can quickly compare previews from different clips, without having to play this silly and time-wasting game of whack-a-mole.
  4. The removal of the double-click-preview to play the preview in an external player, which is required since I use video-compare for the actual comparison because it is able to align the frames between the source and preview clip, something TVAI (still!) is unable to.
2 Likes