I would like to suggest adding support for reading professional RAW formats from ARRI, RED, Sony, CANON, DJI DNG, BMD cameras, and the correct interpretation of logarithmic color spaces such as LogC-AWG, S-Log, D-Log, ACES, etc.
Very often i have to work with such formats with noise reduction in such a video.
Therefore, the second part of the proposal is to add a special model for high-quality noise reduction and fine-tuning without increasing the video resolution, since very often I shoot professional cameras already in sufficient resolution - 4, 5, 6, 8K.
agree. Apple Prores RAW, CinemaDNG RAW.
1 Like
you can press blue button “Vote” above my avatar
Just thinking about how AI models are trained… There is a conundrum here.
If you’re getting raw footage from professional cameras, and they still have so much noise that you feel the need to reduce it, how can noiseless footage be gotten to create noise reducing models?
If the answer is use some algorithm or well known progrom (synthetic means), then you should just use that.
I roughly understand how model training works - you need to give 2 images with and without noise.
Theoretically, NeatVideo works according to the same logic - I show the program a fragment of a frame with a uniform one-color texture, where there is noise, and then the program removes this noise throughout the video.
Can try to request fragments from camera manufacturers, for example, Colorchecker at different ISOs and thereby build a denoising model for different cameras. And camera selection can be made automatic based on meta from RAW.
That would require licensing the RAW codec from each individual camera manufacturer which might be cost prohibitive for Topaz unless they increase the price for VEAI which is what other software vendors have done to offer specific RAW codec processing.
When working with digital cinema camera RAW codecs I prefer to work in Davinci Resolve which has the ability to isolate or qualify specific areas of the image which Noise Reduction can be applied to without affecting other areas.
My workflow so far has been to use VEAI to the deinterlace, uprez, mild denoise then import the file into Davinci Resolve for additional clean up and color grading.
I’m not sure, but it seems to me that camera manufacturers will not require any royalties to use their RAW - it’s in their interest
@Iilya camera manufacturers have a concern to protect their intellectual property hence forth the need to license the use of their RAW product.
Other professional software I currently use which can process certain digital cinema camera RAW formats do indeed pay for a license to incorporate it into their software or hardware.
In addition Professional digital cinema RAW formats such as Arriraw, Sony X-OCN, Redcode RAW, Blackmagic RAW, Canon RAW etc require both a high performance GPU and CPU workstation for the debayer and decoding process.
On an underpowered workstation in order to increase processing performance speed the debayer quality can be set to either 1/2, 1/4 or lower quality at the expense of overall image quality. Lowering the debayer quality can result in additional noise and artifacts as well as degrading the image quality as a trade off for faster processing speed. This is counterintuitive if one is looking to start off with the highest quality RAW debayer then apply the VEAI processing.
Since VEAI already requires a high performance GPU the additional processing stress to debayer digital cinema RAW formats would be very tasking and counter productive for some users except for those who have high performance workstations with the necessary CPU and GPU(s).
For professional users who work with digital cinema RAW formats on a daily basis a faster and more streamlined post production set of hardware and software products already exist such as Davinci Resolve, Baselight, Assimilate Scratch etc which offer advanced denoise and color grading in a more robust and intuitive workflow.
Overtime as CPU and GPU workstation become more powerful and less expensive solutions such as VEAI may become an option once the RAW licensing is dealt with.
My hope is VEAI may one day be licensed as a plugin for Davinci Resolve and other professional grading applications.
you are confusing processing programs and noise reduction.
I myself work in BaseLight, Resolve and Colorfront, As well as in MTI Cortex, in fact, mega computers are not needed to work with RAW, I work comfortably with RAW 4-6-8k on
i9-10940x 96Gb RAM 3070+3080
On another station, I have enough Xeon Silver 4314 128Gb RAM T1000 + A4000.
All of this hardware is comparable to a mid-range gaming PC.
All these programs have very mediocre noise reduction, one of the best is NeatVideo but it also cannot work with logarithmic color spaces.
I think it would be great if VEAI first learned how to correctly interpret logarithmic colors at least Arri LogC AWG. And only then think about RAW support. Moreover, my colorist said that there is no point in crushing noise in RAW, as whether there will be noise or not will be clear only after the pregrade. And there, as a rule, already LogC or Slog as an intermediate or in general ACES AP0
No I am not confusing processing programs with noise reduction.
Agreed in the world of PC lower cost solutions are available for working with digital cinema camera RAW. However one must confirm if the application in which the RAW file is viewed on a particular workstation is actually debayering the image at the highest quality level without the aid of an “Behind the scenes” automatic override of debayer quality level which is done to increase the playback performance of the workstation due to a limitation of the CPU and GPU. In Resolve the user has the ability to disable automatic playback performance mode.
I work with the highest end Mac Studio Ultra which has no problem debayering at the highest quality 6K Arriraw, 6K Sony X-OCN, 4K-8K Redcode RAW, 4K to 12K Blackmagic RAW etc with automatic playback performance mode set to disable.
The ability to selectively qualify or apply a power window for noise reduction is a tool I use frequently within Resolve which VEAI does not offer. In this case I have found the noise reduction in Resolve to outperform the results of VEAI.
I agree VEAI should first incorporate the proper OETF processing which would be useful.
Regardless the RAW licensing I mentioned remains an issue for any software developer to contend with.
you are talking about debaying and the noise of the matrix itself, but I mean that if you took a shot and it turned out to be very dark, there is no way to reshoot, you need to work with what you have. When aligning this frame with neighboring ones (pregrade), strong noise appears on it, Resolve and other programs cannot cope with it. The best result is, as I said, NeatVideo for Resolve, but the problem is that NeatVideo also cannot work with logarithmic color. NV does not work badly with Rec709, but this means you have to convert video from Log-C to Rec709 and then back, and this is a significant loss in quality.
Yes my prior comments regarding this topic have been in reference to the RAW debayer and decoding processing.
I tested out this tool, it is pretty cool but it’s not useful enough to me until it can import native Red Raw files (.r3d). I didn’t see .r3d even searchable across posts which (for me) was not a good sign in terms of the development process. Up-scaling is sort of pointless when you are shooting everything at 6k or 8k RED RAW and down-scaling to 4k 422 output. I would imagine that starting with the RAW formats would yield better results. Especially if you have gyro data for stabilization. Noise isn’t really an issue for me (unless it’s to much iso …but then just turn it down because its RAW) but stabilization is.
Test footage: RED Helium S35 8K camera, Contax - Zeiss 35mm 1.4, shot at 6k W/S , 47.98/ fps , at EDC '23 Las Vegas , processed on m2 max 64gb (I should have bought the 96gb)
