Video AI 7.0 - NEW Starlight Mini (Local) AI Model

Since I have not even had an acknowledgement of either of these 2 points in weeks I have posted them as formal bug reports:

Edit - VLC playback issue resolved - problem was on my end - hardware decoding was disabled on VLC.

Thanks

1 Like

It’s just checking that you have more than the minimum of 8GB.

Was this post about Starlight Mini specifically? Because mine was about overall Topaz performance and my comments were about Proteus in that particular situation. Having performed more testing, I can pretty much confirm (at least on my end) that this version doesn’t seem to be utilizing the 5090 properly at all. I continue to see that the GPU memory never exceeds 7.5/31.5 GB and there’s very intermittent GPU utilization - most of the time hovering in the sub-50% range according to Task Manager. While non-Starlight Mini model performance is still respectable, it obviously makes you wonder what proper utilization of the 5000-series’ render pipeline could yield. Hopefully they can get this figured out sooner rather than later.

Yes, I was talking about starlight specifically. I also hope that the 5090 can be utilized to its max speed potential.

Hi @Moebius . On my 5090 if i set anything above 70% it takes forever to load.
At 70%, for example a clip takes a few minutes for the model to load and say 30min to render. If it is above that say 80, 90, 100% which I have tried I have waited well past 40 minutes for the model to load, and gave up before it did.

At 70% memory setting the gpu runs as expected but only uses half of the 5090 memory. 74% and above it loads the full memory but fails to start or get past the model loading stage.
Even if you reduce to like 30% it still uses half the memory of the gpu (as per 50% setting)
It really is just not scaling.
I just leave mine now at 70%

3 Likes

you can try disable shared memory as i wrote above, at the moment I don’t know it is working or not, I have to wait until my two computers does finish my Starlight jobs

I thought about testing that. I think what might be happening is the app is trying to use more memory than the video card has, so it is trying to use system ram, which causes the speed to dramatically lower.

I don’t really want to mess with turning off the shared memory stuff, so I’m just hoping a future version fixes the issue so that the program uses ONLY the video ram.

This is definitely what’s happening, when the resolution is high enough to exceed video memory. If I disable shared memory for CUDA I can turn the memory usage up to 98 (probably 100 but I set 98) and the video I was processing went from <0.1 to 0.3 fps and I can see the GPU is running at full power now. For smaller resolution videos like 480p → 960p for me it was running at full power, but anything higher res like 720p → 1440p was slowing to a crawl and GPU started working very intermittently, drawing 150w vs 500w.

Interesting…I thought disabling the shared memory would cause the program to crash. Maybe I will test disabling it after all…

Is there a rough estimation when AMD users will be able to use Starlight Mini? Sitting on a 7900XTX unable to go local.

1 Like

What a phenomenal writeup and analysis! :slight_smile: Thank you so much for putting this together!

1 Like

This is a new model with new architecture, and we’re often on the cutting edge. It’s no small feat to get these types of models running locally on any architecture outside of a training or a Linux-based inferencing environment. That’s what we do here at Topaz, but it takes time! :slight_smile:

2 Likes

This is real upgrade. congratulations Topaz team. Starlight mini is very well on video quality. I upscaled 720p to 1440p. and than I enhanced and upscaled it to 2160p with the Iris. result is very well. I think if I could 1080p to 2160p upscale the result would be much more better. but I need at least 10x faster video card than my 4080 super. may be nvidia 7000 serie in the future will easly handle the Starlight.

Secondly, I noticed that Starlight mini doesn’t utilies CPU like Rhea does. If It can do it may be we get a bit more speed.
Anyway thanks for your hard work. I hope you can find a way to speed it up a little without any compromise on quality.

1 Like

After doing several Starlight Cloud and Starlight Mini comparison renders, overall I think Starlight Mini looks more natural. Starlight Cloud sometimes overcompensates on my clips and washes out subtle textures which Starlight Mini seems to do a better job preserving. That said, Starlight Mini does struggle with interlaced footage when there is fast motion or a dissolve. For example a person walking outside a window visible through the open blinds has an outline of interlacing around him as he walks. Overall though, the results are amazing. I have hours of local TV and video projects I shot back in the 90s and 2000s, mostly on U-Matic and BetaCam SP, and I am very pleased with the results I am getting. I just wish I could render more than about 2 minutes on an overnight session. But good things come to those who wait.

5 Likes

Hello.

Starlight Mini does struggle with interlaced footage when there is fast motion or a dissolve.

Yeah. Not just in these frames, though… an Ai model/architecture is only as efficient as its dataset (HQ or LQ data, diversity, ect.) that it’s trained on. Not the so-called “limitations of today’s hardware” as some folks (uninformed) believes in this thread.

On my Astral 5090 LC OC it uses the whole VRAM and at 100% power.

Whatever I try it still uses all my vram and even shared gpu memorty (3080 10 gb). I tried 70% and even 50% still using 9,7/10gb with 0.1 fps.

crash could happen but just TVAI and you can set it back. Disabled shared Ram takes only effect on from you joined exe. I think it is phyton.exe maybe also phytonw.exe under ProgramData\Topaz Labs LLC\Topaz Video AI\models\Lib\venv\scripts\nt\

does somebody know if 1080 1x is faster than 540 2x?

Are you using the NVIDIA app by chance? If so, are you using the Studio drivers or the Gaming drivers? I’m just trying to narrow things down at this point.