Video AI 7.0 - NEW Starlight Mini (Local) AI Model

I can render a 24GB 1920x1080 Flux Image Model and load a 90 billion parameter LLM on a 6GB GPU, pretty sure I can load a model requiring 8GB vram…

What’s the story with 5000-series optimization on this version? I bought a 5090 earlier this week and got it installed to replace my 4090 and paired it with a 9950X3D and it kind of seems like Topaz’s utilization is all over the place. For one, the dedicated GPU memory is showing as 7.5/31.5 GB and the utilization goes from mid 25s-35s and then will occasionally (and briefly) spike to 100% and then repeat the same thing. My GPU temps are also quite low at 37-41 degrees Celsius (my GPU is watercooled FWIW), but that also seems too low especially with what I saw with the 4090, which was also watercooled. Anyone have similar issues?

If I set everything to 100% max vram on a 5090 it takes FOREVER to load the model so I don’t even mess with that. I have my max memory set to 70% right now and it’s much faster, and results still look good. Maybe they would look better with 100% memory but I’m not going to wait 10 minutes + before the video starts processing.

I believe the utilization is close to 100% but I’ll have to check. It’s still rather slow. They said they are working on speed optimizations, especially for the 5000 series. I usually do a 2x upscale. The speeds I get are anywhere from .2 to .9 fps.

Someone said that the screen darkening is happening on “modified” versions of the program. That’s not the case. I’m running the latest version with no modifications at all and I posted two examples above that show screen darkening. It seems like it’s more pronounced in some videos rather than others. I posted screens of the insomniac show. That definitely shows a darkening and color shift. I also posted a screenshot of an old cartoon and you can see that it darkened slightly and colors shifted slightly, but not as much as the insomniac show screens. It’s odd because the darkening/color shifting appears to change from video to video.

Very simply, someone should upscale a video with a bright white background with some type of object in the center, then see if the white stays white.

2 Likes

I will take a look at 100% memory usage issue. Also, pay attention to the color tone during model training. Thx for ur professional suggestion. :grinning_face:

1 Like

Re-posting this as both are still an issue in v7:

1 Like

Wmv format to mp4 dont works with starlight.

1 Like

Just checking in – I’m running a starlight-mini upscale on RTX 4090 (24GB RAM) and it is showing 100% GPU utilization. It’s doing 320x200 to 1280x960 (minimum) and getting about 0.2/frames sec. Is that normal? Or am I seeing some kind of misconfiguration? I realize starlight-mini is slower, but my rig does other models at 30-60fps. Is it really 150-300 times slower?

For me, using up 100% memory makes starlight run way slower, so I’ve been running things at 70%. You could try that to see if it works better for you.

It’s been said that using more memory means there will be better temporal consistency.

.wmv support would also be nice. The whole video will process, but it will fail to export. mp4 seems to work fine.

pends on how much Vram gets used, when you lower it and doing 960p it should be above 0.5fps with your card, but it could also be when source resolution is very low, you have 320x200, it is slower than doing 480p to 960p

I also have RTX 4090 and I’m running at 90% memory and mine runs at 0.7 a sec. I think the model is just intensive and not very fast but the results are the best I have seen in any AI model I used. I have an old TV show that has tons of noise that no denoiser could fix without removing most details and Starlight pretty much remastered the whole footage. Yes it is very slow but worth it. Let’s hope future updates will give it a boost.

4 Likes

Absolutely. This is without question THE BEST upscaler out there for video. The results are phenomenal. There are some little oddities here and there, but I have full confidence that these will be addressed. The videos I’m upscaling are all for fun.

If I was doing this professionally, I wouldn’t care how long it takes. I would want the best quality possible.

4 Likes

100% with you. And yes as with every model, little things can be ironed out with more training especially bright/low light footage as those can be tricky especially if there is noise and the model will not distinguish it from the actual footage and remove the whole thing.

2 Likes

Sensational for SD sources, 1080p to 4K results were less convincing to me. But there are also unsolved problems, for example Starlight still produces alien fonts and its not completly free from face disfigurement

2 Likes

Ok, thanks for the pointers. I tried lowering the memory usage to 90% and it improved to 0.8 sec / frame, which is a lot better. I also noticed that change caused actual graphical memory usage to drop from 24GB, which is all of it to about 11GB, which is a bit less than half. So it seems like there’s some memory threshold at which starlight-mini tries to do something more intensive? I compared the results visually and they’re not identical but I’d say the quality seemed about the same.

3 Likes

I should of course report that the results are amazing. By far the best I’ve seen on very poor quality video. I’m running some nearly unwatchable video through it and getting very decent looking results. Starlight-mini is amazing, slow as it is. Hope they find some major performance improvements, but it’s a godsend in any case.

4 Likes

Only for LQ inputs. Using it on MQ inputs it’s not as good as Proteus/Iris etc…in fact it makes things worse.

3 Likes

Have you tried 1080p to 1080p for videos with a lot of artifacting or videos that are interlaced? Those seemed ok from my testing.

I did try going higher than 1080p and I agree, it didn’t look great. This model is definitely best for upscaling lower resolution footage, which is the most difficult to upscale.

2 Likes

Hi everyone, I just have a few technical questions.

I have a 3080, 32GB of RAM, and an Intel Core i9-12900K. I ran a test with a 1 second video (30fps) to see how long it would take, and Starlight Mini stays on “loading model” for a few hours and then takes around an hour and a half (or a bit more) to render.

As I said, the clip was one second long at 30 frames per second… so 30 frames total. Vertical format, 1080x1920 input and 2160x3840 output.

Is there a way to speed up the process (bring it under 4/5 hours per second), or is it already running at its maximum speed?

Because normally I work with clips that are already 4K native, and this way I basically can’t use my computer for an entire week.

That’s all I wanted to ask, if it’s possible to change any settings to make it faster, and if maybe Starlight in general will become faster in the future. Thanks for everything, the program is amazing

My opinion FHD to 4k isn’t worth with Starlight mini, the results often gets to blury for me and it is unbearably slow under 0.1 fps. I focus again on SD sources.

4 Likes

You can try reinstalling it. I experienced the same issue where Starlight Mini suddenly took a long time to load and then froze. After reinstalling, it went back to normal.