Video Enhance AI thoughts and requests

First of all, thank you Topaz Labs for this revolutionary product and having listening to the requests of having it not only a cloud pay for use product but also a local desktop program.
I tried the Beta app since november and was already conviced so as soon as the final version was released I immediatly purchased it before even installing it and launch any trial :grin:.

First feedbacks : Like I was for the beta I was surprised of the output video codec. It’s the 15+ years old mpeg-4 part 2 codec aka h263 and without any choice of the output bitrate. Is there a specific reason for that choice ?
So for now I only use still images outputs for the best quality output for a use with lossless codecs or pro codecs like Cineform, ProRes on post processing programs using images sequences input. Yes files are incredibly huge but that’s why 6-8-10+ TB hard disk drives are made for after all. And you will have a high quality master file for your post processing, editing etc.
My first request would be the choice of the output codec and settings. The app relies on ffmpeg after all so this should be easilly done at least using a command line like the beta version had.
Also converting SD footage to HD is tricky because they don’t use the same color space standards.
SD uses Rec601 specifications and HD Rec709, so by using the direct video mp4 output you will end up with an HD video encoded with rec 601 spec and it’s wrong because players and even edit softwares will decode it with the rec709 matrix and the colors will not be correct. The output must be flagged as bt 709 using internal YUV 601 to YUV 709 pixel format conversion before encoding.

Other requests :

  • maybe a jump to a specific frame of the video would be useful and more precise and faster than the slider
  • a command line option like in the beta
  • more than 30 frames preview when we try to figure wich model is the most suitable to use
1 Like

If I may, I like to request to add support for the Radeon RX 570 Series GPU.
With CPU, it takes a long time to initialize.

6 Likes

Yeah, I second that…
The program does a amazing upscale job, but the lack of optimization/choice of a codec isn’t still implemented… I hope in the upcoming versions we can choose/optimize our output file with decent codecs like ProRes, and/or even using QP encoding method with x264… ffmpeg supports a lot of video codecs, would be interesting seeing this kind of improvement :slight_smile:

1 Like

Everyone remember to click on the vote box on the top left of the thread. That is an indication of how many agree with this request for enhancement.

1 Like

Hi there
A couple of questions comments and issues
First of all, I’ve been trying to upscale SD video for several months now and I’ve tried every application and plug-in available
Your application blows everything else out of the water and the quality is impeccable and nothing even comes close.
That being said I understand that you have just released this application and I have noticed some issues.
I know it’s says it does not take interlaced video so I tried De-interlacing my videos and the application will not import them. The application just locks up and freezes.
When I bring in a source file that has not been De-interlaced the file loads off and is able to be processed by your application.
One major feature that I Believe it is necessary in your application… when you choose crop to fix screen and the red box shows the outline, you should be able to have a choice to position the red frame so you don’t cut off anything that’s important in the video.
When I import SD video 720×480, I’ve then choose crop to fix screen when I’m out putting at 1080 x 1920, a red frame comes on and shows the outline of the new video but I am not able to manipulate it to the position I want.
All applications that I have used in the past for this procedure has an X and Y access Point adjustment where you can relocate the cropped image.
I believe this is a necessary feature that needs to be implemented.
Also, I know your application disregards the audio portion of the video…
Maybe you can add a feature that will export the audio as a separate file before the upscaling process begins so they can be married back together at a later point after the processing and new file is completed.?

Also I primarily work on a Mac and I know you are working on a Macintosh version, any idea of a release date?
I work with a black magic GPU pro external video card and I’m hoping your Mac version will recognize this card and utilize it.

If you would like me to send the files that I have been trying to import so you could see what the issue is I would be happy to.
Just let me know how I can get them to you.
Thank you for your time and I’m looking forward to your response
Regards
Mark

3 Likes

This is from the scratch really a new way/idea in order to improve bad quality videos.
I tested so far a lot of small video scenes. Mostly video footage with resolution 640x480 and below.
Only with the modus “Upsampling(HQ-CG)” I get remarkable improvements. The other both modus are worsen the picture quality and therefore are not useful.
I tested all scenes several times with different resolutions/sizers/percentages.
Even I get remarkable improvements with modus “Upsampling (HQ-CG)” faces and hairs get destroyed!
So the software needs improvements at least on face recognition.
Today Video Enhance AI therefore is not usable for me. I wait, how improvements wil take place.
199€ are way too much for a not finished develped software.

3 Likes

HQ model as it says requires quality input with few compression artifacts and noise otherwise these defects can get amplified. HQ-CG does a good job of removing these defects but at the coast of a strong denoise so we loose the fine details. If we could get a control of the denoise amount we could get stunning outputs in some videos. The LQ model seems useless even on the worst inputs I could find from VHS or ultra low res compressed videos. It does nothing really noticeable at the best and often it gets worst.
As for the face detection I agree, there should be a face refinement like in Gigapixel AI and with a customizable threshold slider 16/32/64/128 pixels. It’s better to have a face that looks human even if the subject will necessarily look different as in reality than the subject looking like an alien or monster.

3 Likes

Please make it available on iMAC platform.

I made further tests last days so far again with bad quality video footage.
Resolution original footage 640x480 and below.
Sometimes the results are very impressive. I never thought before, that it would be possible to deblocking, denoise and sharpening such videos in such an impressive quality. But the results are very various from video to video. I guess, Topaz has to fill the AI much more with reference footage.
And again: faces and hairs get often destroyed.
ONLY the modus “Upsampling (HQ-CG)" brings sometimes such impressive results.
The modus "Upsamping (HQ) brings more sharpness but destroys faces and hairs much more.
The modus "Upsampling (LQ) works for me on no footage. In each rendered video it adds spots which were not part of original footage.

My wishes so far on improvments:

  • add better face and structure regocnition
  • implement slide controls for sharpness and noise reduction to be more flexible in order get best results
  • implement audio
  • improve the modus "Upsampling (LQ); e.g. also here treshold sliders for manual adjustments
4 Likes

2 posts were split to a new topic: Video Enhance AI - Current Issues

I’ll chime in on the HQ-CG mode seeming to produce better results in many cases. It sure handles lines and edges very well, anything where aliasing would be an issue. Seems it handles such potentially jagged edges (step-stair appearance on diagonal lines) very well.
But then it seems to handle textures less well in some situations, meaning it smooths out fine details somewhat between such lines and edges, making some surfaces look plastic. The HQ mode seems to keep textures better, but seems to have aliasing-issues with lines and edges.

Suggestion:
Some sort of combination of the two modes might do wonders. Maybe a slider to add in textures and fine details from the HQ mode into an otherwise clean/smooth HQ-CG upscaled video would solve this. Would make it possible to have nice upscaled edges without aliasing issues, and keep more detail where they might otherwise get smoothed out.
A slider for sharpening might also be good.

Regarding the sharpening; as it is now I get pretty good results by upscaling to about 1.5X the target resolution, and then apply some sharpening and scale down to the target resolution in Premiere/AE or something.

1 Like

I further tested several footage.
Often the deblocking and denoising are very impressive.
Also smoothen and straighten of edges are nearly perfect.

In often cases the denoising and smoothen of the footage is too much and the video looks too unnatural on the end, cause fine structures are gone.
Here it needs some adjustment sliders in order to increase/decrease

  • denosing
  • deblocking
  • sharpening

In other cases of e.g. grass the tool adds unnatural structures into the grass.

Is a next update so far with first improvements foreseeable?

4 posts were merged into an existing topic: Video Enhance AI - Current Issues

I have tried it with various inputs and so far I am very impressed with the results. One area where it doesn’t really work all that well is digitalized analog video. I have quite a few videos from analog sources which show all the signs of analog footage (wobbly image, scanlines, combing). If the product had a way of upscaling these just as convincing as it can do upscaling of videos with lots of compression artifacts that would be a real game changer for people doing restoration of old footage.

1 Like

please add amd gpu support! :heart:

5 Likes

I made the same experience. I tried with different settings some old Hi8 video clips, which I digitalzied years ago.
The improvements are on a very small level. Details are not really improved. Also edges are not stabilized after rendering.
Same with old VHS footage.
Obvouisly at least DV footage is required as minimum requirement in order to get improvements.

5 posts were split to a new topic: Video Enhance AI Install & Update

Also agree that sliders for various DeNoise settings would be useful.

Please add industry standard output codes like ProRes and ProRes HQ. The h263.Mp4 format is not useful at all for pro applications.

Keep up the great work!

1 Like

I received this morning a Topaz Newsletter, that a new version for Video Enhance AI is available.
Is this true?
If yes: which features was changed and which features was added?

Sorry, typo…I meant codecs…ProRes please!