Topaz Video AI Beta 3.1.3.1.b

gregory.maddra · February 6, 2023, 10:51pm

Hello Everyone!

We have another beta for y’all. Please let us know if processing gets stuck and send us the logs.

Download:

Windows
Mac (DMG) | Mac (PKG)

Changelog:

Apollo model now produces sharper results
Updated duplicate frame identification (DFI) to detect small changes
DFI now checks with original frame not the most recent
DFI now uses a threshold that can be adjusted in the FFmpeg command with rft option (not available in GUI)
Maximum duplicate frames handled now depends on the model maxFrames option(default 16), number of duplicates handled is maxFrames/(slowmo x fps change rate)
Fixes stabilization second pass taking too long to start on M1/M2 Macs

Thanks for testing

Please upload problem videos and logs here: Submit files

20rushtonj · February 6, 2023, 11:58pm

According to the command line, Apollo has support for removing duplicate frames, but the setting isn’t exposed in the GUI. Is this intentional?

Can we adjust the maxFrames value? If so, how?

I just want to be sure, a higher value means the duplicate frames detection system will mark larger changes as “duplicates”? And a smaller value means only small changes will be marked as duplicates?

And this is just personal opinion, I believe the Ignore Duplicate Frames option should be renamed to Replace Duplicate Frames. It explains it better (at least to me).

Thank you all for the work you do. I’ll start properly testing this new beta soon and provide proper feedback on it later.

suraj · February 7, 2023, 1:23am

Thank you @20rushtonj

According to the command line, Apollo has support for removing duplicate frames, but the setting isn’t exposed in the GUI. Is this intentional?

Apollo has always replaced all frames and so doesn’t require duplicate frame replacement, the rdt parameter is ignored for Apollo. This is similar to many parameters being ignored in the upscale filter when using models other than proteus.

Can we adjust the maxFrames value? If so, how?

You will have to add a field maxFrames in the model json file.
“maxFrames”: 32,

I just want to be sure, a higher value means the duplicate frames detection system will only detect larger changes? And a smaller value means only small changes will be detected.

Yes, higher the value means more frames will be detected to be duplicate.

And this is just personal opinion, I believe the Ignore Duplicate Frames option should be renamed to Replace Duplicate Frames. It explains it better (at least to me).

Agreed, will change it

20rushtonj · February 7, 2023, 2:06am

Just confirming, does it go at the end of the .json file. Like this for C:\ProgramData\Topaz Labs LLC\Topaz Video AI BETA\models\apa-1.json (Apollo Soft)?

    "minAppVersion": "2.3.0",
    "minCMLMacVersion": 10.16,
    "modelType": 2,
    "postflight": 2,
    "preflight": 1,
    "processAll": false,
    "shortName": "apa",
    "maxFrames": 32,
    "version": "1b2"
}

dunmer2007 · February 7, 2023, 2:12am

Thanks 20rushtonj for asking the important questions while the excellent dev team clarifies the answers that we need going forward!

suraj · February 7, 2023, 2:30am

Yes, that should work.

dunmer2007 · February 7, 2023, 2:44am

YES! The issue of the frozen export, while using apollo soft, is no more!

Thanks dev team!

20rushtonj · February 7, 2023, 3:06am

Sorry for being a nuisance, but is there a plan to make the maxFrames value configurable through the command line and GUI?

If there isn’t, could you add “maxFrames”: DEFAULT_VALUE, to the .json files in a future update?

suraj · February 7, 2023, 3:12am

The maxFrames option is model specific and will be always in the json file. For the release we will add it to json files.

TomaszW · February 7, 2023, 6:49am

Personally I will not download anymore because there is still no news on the side of the models, no updates of the models, and still no manager of the models in sight. So for my part I pass because I see that you don’t even respect your roadmap. I will download the version of TVEAI when there are the updates of the models and its manager with the old models like Artemis 7, 8 etc…

dunmer2007 · February 7, 2023, 8:59am

I ran the apollo models along with the Ignore Duplicate Frames checked, and all went well, however, as far as both Chronos models were concerned, both failed at 99% when IDF was checked, but ran flawlessly when unchecked.

I just sent my logs and settings info, if you need the videos/Video info, let me know.

joachim.ehrhardt · February 7, 2023, 9:04am

How to detect if a model had been updated/improved, and not only been “renamed”.

meimeiriver · February 7, 2023, 10:21am

‘We’ (as a community) have given them a pretty hard time over dismal performance. They have taken drastic steps to improve on that. Usually these things go in waves: cute new things are getting introduced, followed by an optimization pass, followed by new innovations again. It doesn’t look like Topaz has the staff to do both equally well at the same time.

Personally, the highest priority, for me, lies with ‘not distorting small faces;’ aka, the introduction of some sanity checks on first recognizing, then repairing small faces. Hopefully a Proteus 4 will do that. I also wish the devs would spend less time on motion blur detection/removal and stuff, which are, for all practical purposes and intent, utterly unusable due to taking a near full century to complete on current hardware. Recognizing faces was on the roadmap, at some point, then quickly disappeared from it again.

Anyway, I will continue to test, as overall progress is being made. And the latest performance pass was really welcomed. With Lamba purportedly having reached sentience, I am still holding out hope for a little more ‘i’ being injected into the whole process, though.

pavel-5256 · February 7, 2023, 1:04pm

I have a low light recording done with gopro which was wrongly set to 60fps instead of 30fps. Gopro in such case creates duplicate frames for each original frame. What settings is recommended to use this duplicate information to create 60fps footage, where the duplicated frame is created from previous and following frame? What model and/or settings are the best? Removing duplicated frames and having 30fps is an option but I am trying to get more out of it with AI options.

mikmod1 · February 7, 2023, 2:49pm

With Chronos Fast and Chronos I’m still getting a stall, just at the end:

It’s that video with irregular 57 fps, and obvious couple-frames freezes. I uploaded it to the DropBox. Funny, since the stall occurs when a running player enters the frame in the background (there’s no scene change). The image change is, however, substantial and that might “trip” Chronos Fast with Ignore Duplicate Frames the wrong way.

However, with Apollo Soft and Sharp and Ignore Duplicate Frames ON, this video and D&D cartoon preview went well, and frames weren’t cut out this time when scene changed in the cartoon.

I have a feeling both Chronos models are forced to stall, when they “meet” the certain amount of frames needed to be removed in this beta (I tested it with over a 1s of freeze on almost-60 fps video, I set up "maxFrames": 128 to be sure I “hit” it, though 64 and 48 values were also enough to make it stall; default value ~~[I don’t know it]~~ 32 (?) leaves this “over a second freeze” untouched and the stall happens at the end of the preview, at 99%, always).

Edit: Increasing “maxFrames” value to over 32 in Apollo Soft models also might stall the processing, when this beta of VEAI “meets” a video freeze long enough to incur it. Otherwise, the stalling might not be caught with Apollo models.

This time, however, ffmpeg stops doing anything, while stalling in preview (for Chronos, Chronos Fast, Apollo Soft models).

Using the "maxFrames" with 16 caused AI Error on my side.

dunmer2007 · February 7, 2023, 6:43pm

With Chronos Fast and Chronos I’m still getting a stall, just at the end:

Uncheck the Duplicate frame setting, and it should work!

david.123 · February 7, 2023, 6:45pm

Don’t worry - nobody is forgetting you want face detection - since you talk about it nearly every time you post

If you think motion related models are slow, wait until you see how long it takes to detect and improve faces in video. Based on how much extra processing Photo AI needs for this, get ready for 4 - 10x slowdowns - and then nobody will want to use Video AI (except you because as long as it fixes faces in a 240p source video that never had face details to begin with you are prepared to wait 3 months to process a 10 minute video ).

My opinion is that Topaz is currently focusing on the correct areas for the models - features that improve overall quality - not gimmicky features like rebuilding faces that don’t exist.

meimeiriver · February 7, 2023, 7:09pm

FTR, my sources aren’t 240p, of course, but simply 1080p blu-rays with faces in the background: unavoidable in almost any movie.

Also, looks like I haven’t bnrought up the issue enough, as I have stated earlier, that face recognition can’t occur locally (on our own computers), but would have need to be done on the large systems they train their models on.

Moreover, recognizing faces, while too slow for our home computers, does not nearly take the ages you think it does. See: How Does AI Detect Objects? Quite doable with today’s hardware and knowledge.

yazi.saradest · February 7, 2023, 7:12pm

Hi - I’d recommend you use Chronos or Chronos Fast with “Ignore/Replace duplicate frames”. I’d also keep the Slow Motion set to None.