i am referring to artemis V8 low quality and high quality, that i still use in some cases. in many cases you get artefacts with these models, but in some other cases (when there is no fine patterns, and not used for upscale) they do a better job for me.
1 Like
Hello, can you give us some insight as to how Video AI utilizes the M1/M2 CPU for processing? Or does it use the GPUs to assist? And what about the neural engines? As I’m told the neural engines can provide better/higher image quality and faster processing. Is there any plans to leverage them in the future?
Currently, I have the M1 with 8 high performance cores. However, according to the V AI settings, I can only employ 4 processes. I know more cores may not necessarily improve performance, but any hopes of using additional machines like a “render” farm?
Thanks.
Great to see that we can use previous versions of models for comparisons! I started on 2.5/2.6 so hopefully I can use it to see how things have come along
Any chance for SVT-AV1 encoding in addition to VP9? VP9 Good/Best’s preset encoding performance / thread splitting isn’t good (just generally even with lots of tweaks in ffmpeg outside of VEAI, I see 4 cores max for a 1080p video) and image quality wise it’s worse than AV1/H265 hw imo. Loves to smooth details out, AV1 does that but at least it’s not as bad, and a futureproof format
Video AI does make use of the Neural Engine on the M-series processors for most tasks. The primary processing is still done on the GPU cores, but the Neural Engine is also assisting.
The macOS Activity Monitor does not display these cores in any of the graphs available for CPU and GPU usage, but using a tool like asitop, you can monitor the power and utilization of these cores.
We’re currently working on some major improvements to the export options available in Video AI, and while there is no current plan to add support for SVT-AV1, we do support AV1 hardware encoding on the NVIDIA RTX 4000 series and Intel Arc GPUs.
Unfortunately no mention or future planes for color management. Topaz Video AI tends to change original color appearance in its output. For professional use this is important and having color consistency with proper color managed workflow is needed. When can we except development in that area?
4 Likes
Great point! This is an on going area of improvement. We are very close to resolving this 
4 Likes
- New enhancement and upscaling model with better face recovery (beta)
PLS PLS PLS release beta version of betawant it so much <3
its amazing feature, even if it works with error, i want try to use it in beta <3
4 Likes
You can try it in the beta version already. → Proteus v4 (Iris)
3 Likes
Good to hear.
Someone also made that point.
Any chance of getting a Video AI OpenFX plugin for Resolve?
Plugin integration is an area we are currently looking into
where i can find Proteus v4? downloaded beta TopazVideoAIALPHA-3.3.0.0.a.dmg but there is no Proteus v4
Just select Proteus and click “show export command”, you should see it use prob-4.
For futher discussion related to beta program, I think we better discuss within the beta forum.
2 Likes
Yes, face recovery will take this to the next level. It would be nice to have some options for face recovery strength, like in the photo app.
5 Likes
I would love to see a dynamic Deinterlace / IVTC fiter for mixed content on the roadmap.
The reason that this should be in Video AI is that tvai_up should not waste time upscaling repeat frames caused by pullup/detelecine. In order to achieve the best results, the upscaler should always be fed with the highest temporal and vertical resolution that is available - whether the scene was interlaced or telecined. At present, you have to decide whether to pre-IVTC or deinterlace - and that can vary by scene in mixed-mode content. Applying both is destructive or applying the wrong combination results in repeat frames being sent to the upscaler.
At present, Video AI uses the excellent bwdif in the workflow for deinterlace from fields to frames. But TVAI does not perform pullup or pullup detection. So you either have to pre-process ITVC or feed duplicate frames to the upscaler.
The current FFmpeg native filters do one thing only, either deinterlace or detelecine. There’s a need on the market for an AI driven auto-fieldmatch/deinterlace filter that deinterlaces where necessary and detelecines both fixed and broken cadence, outputting VFR (which can subsequently be interpolated to CFR where necessary). The goal of the filter would be to display progressive fields at the PTS of which they occurred in the original content, without duplicates and without discarding a field - even if that results in VFR.
Furthermore, the frame tags that indicate interlaced or telecine (TFF, BFF, RFF) have often been destroyed upstream. FFmpeg’s idet,metadata=mode=print does an ok-ish job of attempting to identify whether a frame is interlaced or part of a pattern. The fieldmatch filter in debug mode has a different set of logic. bwdif and nnedi try to work in an adversarial way to determine whether the result is better than the original. But these are all separate processes in a filter chain. They fight and work in opposite directions. They all have their own internal logic. decimate and mpdecimate have their own internal logic too. Throw in some fps and dejudder logic into the filter chain and FFmpeg’s output is far from ground truth.
FFmpeg’s process of chaining idet,fieldmatch,(mp)decimate,dejudder,bwdif,fps is clunky and there’s a high probability of destroying valuable frames and fields, or destroy resolution that would benefit tvai_up. The FFmpeg fiter chain must be customized for each asset - this should be unnecessary in the world of multi-frame, stateful AI.
Cinnafilm Tachyon Tachyon - GPU Based Image Processing & Frame Rate Conversion does AI field/frame restoration as a NLE or cloud service, but there is not a standalone tool out there, like VideoAI. The TVAI team could use their already proven AI smarts to perform automated frame and field restoration all in a single filter, which will give the upscaler the best possible chance of making the most of the content.
Of course, this is more a criticism of the currently available FFmpeg filters being a chain of individual processes - TVAI are uniquely placed to do this better in a custom filter.
A use-case of TVAI is the upscaling of SD, DVDs & ATSC1.0 to progressive HD - and the optimal pre-handling of mixed-mode telecine and interlace is critical before wasting processing on the upscale.
Please do pass on this suggestion to the Product Managers and ask them to take a look at Cinnafilm Tachyon’s capabilities which could be added as a proprietary workflow for the correct (or at least the best reasonable) temporal restoration. Thanks!
4 Likes
Exactly! There is a serious need for this in the market. I work with different mixed sources of HD1080i50, HD1080p25, PAL and NTSC with broken cadence. Often a video is built from different sources. HD1080p25 can be a bad upscale of original PAL-content with 50 interlaced frames. PAL can be a bad conversion from original NTSC-content with awful interlace artifacts in each frame. I spend days of manual work to bring NTSC back to original shot 24 fps by checking scene-by-scene where duplicated frames are. I really need an autodetect funktion in VEAI, where I can put in my various videos and output 4Kp50 or HD1080p50 where VEAI automatically detect the source on a frame-by-frame basis and use the relevant models for each frame. In the perfect case, VEAI should also detect bad upscales or conversations and find back to the original format, from which the correct model should be used.
2 Likes
THIS! VAI often changes black to “dark grey” or changes the intensity of red.
1 Like
This sounds fantastic! Here’s to hoping for a Resolve-compatible OFX plugin!
1 Like
