Things needed in Video AI

Things missing from this program which are common in video editing programs, which would improve the workflow.

  1. Time code read out.
    It would be nice to see an actual timecode clock so I can scrub to a specific location.
  2. Markers.
    Having the ability to put little markers on the timeline helps identify areas that need processing at a later time, like if my video was a montage of various videos combined into one file. Markers would help me locate each clip within the timeline, so I can identify what’s what and export them individually.
  3. File location
    A simple right click on the exported clip or other, to help me identify where the file is being rendered to, if I have multiple hard drives installed. It also helps to mover a file after rendering if need be.
  4. Audio channels
    Simple audio left/right channels with a meter so I know I’m not clipping or peaking, the output file.
  5. Audio level controls
    Simple audio level controls to adjust output exported file.
  6. Porting
    The ability to port to other devices like BlackMagic, Red, etc
 for real time previews, scrubbing, etc

  7. Color correction wheel and fixes
    Having the ability to adjust hue and saturation, tint, vibrancy, color correction is a must.
  8. Video corrections
    Having the ability for the AI to correct old analog video problems like, rainbow stripes, interlacing problems, smoothing, dullness, distortions, color bleeding, over saturation, and such, but also for film.
  9. Multi processor and render farm
    The ability to set a certain number of processors and GPU for rendering, but also a render farm involving server rendering.

TVAI is not a video editor. It’s got some nice-to-haves that are generally found in video editors, but those are bonus features. TVAI is a tool for enhancing videos. It is best used in combination with a real video editor to enhance short video clips. It can be used to enhance full videos, but the longer the video and the more diverse the content in the video, the more undesired artifacts are introduced.

Number 1. :ballot_box_with_check: There is a time code, if you have it set to time instead of frame number in preferences. In addition, both the in and out markers have their own time codes that are always shown once set.
Number 2. I hope they are moving the UI in that direction.
Number 3. :ballot_box_with_check: The last line in Show FFmpeg Command is the output path. Once the Export is done, you can click ‘Show in Explorer’.
Numbers 4 and 5. :ballot_box_with_check: It’s not an audio enhancer. I always tell people to grab all audio, subtitles and chapters from the source and merge them back into the TVAI output. If Topaz makes an audio enhancer, it had better be a different product—or be able to output to DTS and Dolby Digital.
Number 6. I know nothing about Porting. What I do know is: ‘Real time previews’ in TVAI are very situational. Most hardware and use cases turn out much slower than real time.
Number 7. Not a video editor
 but there are ongoing issues with colorspaces. Something better should be done about it.
Number 8. Various AI models fix various video problems you mention to various degrees of success. We always enjoy more models to try in the TVAI toolbox. You want more? Make idea threads detailing the models and argue the case and need for them. Check if someone has made the idea topic already first, of course.
Number 9. Awesome idea, but, TVAI is built on needing to know temporal information. At best it would need to automatically split videos into scenes and send them off to be processed as a whole video, then join them back later. If they’re going to make that, they had better make it so each scene can have its own AI model and parameters set.

2 Likes

Re #9, they mentioned last year in the roadmap section that they were already investigating a cloud backed option. That would mean a render farm.

Personally I’d prefer if they’d just released a runner agent and make the GUI able to talk to it. I don’t want yet another vendor to assess compliance with. Some of our material has heaps of confidentiality requirements and it took ages to get even AWS to pass. So we’d want to use the vendors we already have vetted. Just give us an agent we can deploy to a dedicated VPC or network at whatever provider we’re already using, and I’d be happy. And No requirement for egress to Topaz servers. Again, due to security and confidentiality. A Topaz managed solution is a no-go for us. A backend (agent and orchestrator) we can deploy ourselves, I couldn’t be happier about.

I’d prefer not to see Topaz waste developer hours adding functions I can get from freeware video editors.

OTOH, I’ve gotten about a dozen offers from cloud GPU providers offering access to processors that cost more than my car for less than $1 an hour. If I knew for a fact TVAI knew what to do with an H100 or an MI300 I’d be up for giving that a shot.

Agreed. But they’ve stated “cloud backend” as part of their top priorities, so my comment was that if they are dead-set on sinking cost into such an initiative, then it should result in something that’s of highest value to customers / money invested. I’m perfectly fine not having any cloud integration since I’ve got an in-house solution for that already, and would prefer they instead drop that from the roadmap and focus on improving the core models instead. Though my opinion is just one data point and they’d need to know what the rest of you think as well to make a good decision on that one roadmap item.

Cloud GPUs are much simpler to do than a cloud backend or render farm. You load the app on a virtual machine on your computer and configure the VM to use the cloud provider’s processor as its GPU. The only thing the app needs is models optimized for whatever processors the providers are offering.