Low light video and lighting flicker?

Hey there, I’ve enjoyed your product.

I have 2 things I would really love to see added to your product. It might already be there or be on the way. I would just like to let you guys know what I’m looking for.

I do a lot of low light video work. Often times I will use the natural lighting with the A7s. I did submit some video files showing some stuff I was working with that didn’t turn out that great but I have no idea if anyone worked on a model for it. There was an issue where if there was any flash of light in the dark scene, the area in the highlights would momentarily retain some pixel noise before dissipating. So it was almost like a delay to denoise the first 3 or 5 frames of the image that transitioned from dark to light.

Basically I need low light noise removal and sharpening. And as many of you probably know, a lot of the noise in low light video often times is kinda a purple/pink/magenta that appears in the pixels. That’s been my experience with the a7s. The a7s iii got rid of a lot of that hue in the corners or sides of the image.
This video illustrates what I’m talking about. And feel free to rewind it to see the types of noise. But i’m specifically looking at the color in the corners and hue of the noise. Because I was able to get rid of a lot the noise, but you still see the magenta blotches. Sometimes green blotches. basically it just looks like it blurred, what I would call, is more like color noise.

But ideally you guys at topaz labs might be able to come up with a model for correcting this often very predictable issue. It’s very repeatable. And I think most of the a7s cameras have it in the same spot, with the only difference being the degree to which it shows up.

But even other cameras from other brands, we see that same type of noise/color. I think it has to do with the sensitivity of the sensor for each color/spectrum of light. Specially Cmos sensors pic up the magenta spectrum more than anything else or something like that.

So having a model to get rid of this hue or color would be a godsend because obviously you could just upgrade your camera, but from what I gather sony pretty much improved the color with onboard software at the expense of overall detail and image brightness. Even though the a7sii is nosier it is also brighter than the a7siii so that’s kinda the trade off.

So maybe Ai on a actual computer could do it better? And even when using small sensors like gopros or cell phones at night, you see that purple magenta hue in the noise as well(sometimes green). A lot of times I’ll just turn down saturation and almost all night footage from any cmos camera will look better.
So adding this specific feature would actually shore up a lot of different cameras short comings, because unless you have an a7s you usually can’t film at night or in low light. Like if you try using go pros or any small sensor at night you’re usually going to have a bad time in post getting rid of the noise.

The other thing I would like to see is a deflicker option for lights. there are some plugin tools out there for this already, but ideally topaz ai video would be a much of a one stop shop for any kind of correction before editing. The type of flicker I was getting was more a rolling bar for sped up time lapse. I was filming a car being wrapped and the led lights that had in the shop, although not noticeable with normal footage played at normal frame rate, as soon as you sped it up and made a time-lapse there was this very noticeable flicker and dark horizontal bar rolling down the screen. So yeah it would be interesting to see if your AI models could get rid of this.

I can submit footage for both of these ideas if you would like to work on it? I just want to get the most out of the product so I’m not running it through a bunch of different plugins at different times. And from what I’ve seen your models are way more effective than a lot of the older mainstays of video editing. So it only makes sense that these two areas would be focused on as well since no other programs have really perfected a solution yet.

I support the request for deflickering. In many led-illuminated environments the flickering can be seen in normal footage that was shot with short shutter times vs. the frame rate, and it cannot always be seen in the camera display during shooting. I have used plug-ins from external suppliers to remove it in post (with varying success, by the way) and would be interested what Video AI could do.

Hello and thanks so much for this detailed feedback!

We are actually working on a “cinema denoise” model that is in training right now, and we’d love to work with this footage as a test case.

You can securely submit your file(s) to our Dropbox using the link below. Please be sure to send me a note to let me know you sent something.

@wingzero @h.petersen

Submit to Dropbox

Hi Tony,

In this moment I am not at home, but next week I will look for a sample in my archive.

With kind regards,

Henk Petersen

image001.jpg

image002.jpg

Good request. I support this as well and its good to hear Topaz has denoising model in development. In the meantime Neat Video + Davinci Resolve for example, is a powerful combo to fix a lot of this, and than upscale if you so choose. Also I like to use Digital Anarchy Flicker Free plug in. That should be enough to take care of these issues. But if Topaz can do it all in one tool, automatically and reliably. I’m all for it. For sure.

Is there a way I could share my thoughts via a live stream or a desktop recording with your developers or researchers? I have plenty of footage I could gather and send but I would like to give examples of where the training could focus on.

1 Like

Is it also intended that the video or marked objects in the video can be brightened or darkened?
:bulb: :film_projector:

If the iso and color noise could eliminated completely, you could have a feature that turns up the gamma and brightness theoretically. The only reason you don’t normally do that is because the colors will wash out and the image will flatten.

But if the ai model could correct/compensate for that you might be able to pull it off.

Hi Tony,

I have found a couple of exemplary files, some are really horror files. They are all in intermediate format, meaning that the file sizes are too large for Dropbox (several Gbytes each). Can we find another solution to let you have them?

Kind regards - Henk

image001.jpg

image003.jpg

Thanks for finding those! Our Dropbox should be able to handle large uploads, but if you run into errors when uploading message me directly and we can set something up through a direct sync.

Tony,

I have uploaded 6 files to your Dropbox.

I wish you success with your efforts!

kind regards,

Henk Petersen

image001.jpg

image003.jpg

Would like to add that the frequency of the rolling bands is probably dependent on the mains frequency. We have 50 Hz, so in the US it may be different.

kind regards,

Henk

image001.jpg

image003.jpg

Tony,

I have uploaded 2 more files, recently shot in the cathedral of Murcia, Spain. The pattern is similar to the previous videos.

Kind regards,

Henk

image001.jpg

image003.jpg

Best thing is to suppress flickering already at the recording.

You can change the fps to the frequency of the flicker (e.g. 25/50 fps for Europe or 30/60 fps for the US) and this can be quit a game changer .

I’ll try that. Another good way to avoid flicker is to avoid short shutter times. This, however, may necessitate the use of a ND filter.

kind regards Henk

image001.jpg

image003.jpg

Unfortunately not every camera allows for exact settings and I think some of these led bulbs are actually flickering at a non standard rate. And this flicker was only really noticeable after speeding up the footage.

It would be also good for this model to remove not only flickering from frame to frame but also normalize lightness of frame parts (so called local flicker, when different parts from frame flicker in different way). It is a common problem on old footage.

You might want to give Zero-DCE a try; I’ve recently contributed with a script that takes video as input and I’ve been using it on SD footage only, not sure how the model and my script will behave with higher resolutions- if you give it a try, please let me know how it works for you :slight_smile:

A video DeFlicker AI model is a great idea!
I found something on GitHub (All-In-One-Deflicker) that might be good for that if it works.
It’s based on deep-learning and can fix several types of video flickering.
Here is the paper about what it does and how it works.
I haven’t gotten it to work on the computers I’ve tried it on - it’s quite hard to install and get running and needs special requirements to work at all.

Hello, has there been any progress on this effort for deflickering?

1 Like