Video Enhance AI v.1.2.3 - Gaia-HQ model improvement (hotfix)

Download: Windows, Mac. Released May 26, 2020.

Major features

  • Gaia-HQ model improvement. Tweak Gaia-HQ model to make it match the quality in version 1.1.1.
5 Likes

On upscaling DVD’s, I am not able to match the quality of Artemis-MQ from version 1.2.0 on any version after that.

Is there a chance you post screenshots from each version you tried, to see examples. I’m curious what is the difference. Thanks.

Good quality PAL DVD.
…
…
…
…

Bad quality PAL DVD. 28 Days Later was shot in non interlaced “frame” mode on Canon XL1 cameras, which is only 360 pixels vertical. It was resized and over sharpened for PAL DVD. Video Enhance AI breaks down at being able to enhance such low resolution content, or overcome any filtering (sharpen, etc.) that may have taken place on the source video. Hopefully that may change in the future.

Another issue is that with version 1.2.1 and above, Video Enhance AI makes the result somewhat brighter. I’m not sure if that’s a bug, a deliberate pre-set adjustment for gamma, contrast, or brightness, or if it’s trying to run some kind of “auto adjust” algorithm. Anyways, if this is intended, it should be an option the user can turn on or off. (“Auto adjust video levels” or whatever.)

Maybe already heavily processed 28 Days Later is not the best benchmark, because it will already have halos on the edges and unnatural detail, but I do notice the slight brightness increase in newer version of Video Enhance AI. I noticed slight averaging on videos I tried with the skin tones. If the skin tone is little bit more magenta or yellow than what is considered average, Video Enhance AI will change it to more average skin tone. Sometimes its a change only one can see if you watch before and after but it is a change from the original. I assume that its basing this correction on the data it has been fed with. Its not a bad adjustment, we all like good skin tones, but sometimes these changes are something one would wish to control. Thanks for the comparison shots.

The Gaia-HQ mode is perfect now :slight_smile:

1 Like

The Mac disk image is apparently not properly signed.

Catalina says the disk image can’t be scanned for malware and thus refuses to open the installer.

Please re-upload the correct version, thanks! :slight_smile:

Much better, thank you!
The is still (much) work to be done on face recognition - especially the weird results on eyes - and on textures. Sometimes texture and textile enhancement works, sometimes it doesn’t. At least we got rid most of that weird “noise texture” on scenes with stars in the sky.
The overall look is a bit “plastic” but it’s ok. The main issue (as I already said) is the eye recognition, especially on scenes where a face is intentionally blurred, or scenes with many people on the background.
Keep up the good work!

5 Likes

My thoughts exactely. The eyes are often the most important focal point and yet they seem to have the least eye pleasing results when upscaling. Hopefully the machine can be trained for correcting that in the future release.

Am I right in assuming that once I have my (1154) tiff files, I must use an external editor (Photoshop, VirtualDub etc) to stitch them into full-motion video?

I mean sequences of files are not stitchable in Video Enhance at all?!

Yes in version 1.2.3. However, there is a new beta being tested now that puts the images together into a video at the end of the process.

1 Like

indeed!! we definitely need a better encoder!! and also a lossless video output option!

2 Likes

just use ffmpeg to encode it into video with the codec of your choice

2 Likes

saving tif images to keep the image quality and then encode them via mpeg 4 after will drive the attempt to keep the quality ad absurdum. :thinking:

I think my Save out by default name is broken, does anyone have the same issue with this version?

Anything but H264, or H263 we need something better. h265 and if that is a problem because of license maybe something like the new standard being proposed Av1.

" AOMedia Video 1 ( AV1 ) is an open, royalty-free video coding format designed for video transmissions over the Internet. It was developed as a successor to VP9 by the Alliance for Open Media (AOMedia),[2] a consortium founded in 2015 that includes semiconductor firms, video on demand providers, video content producers, software development companies and web browser vendors. The AV1 bitstream specification includes a reference video codec.[3] In 2018 Facebook testing that approximates real world conditions, AV1 achieved 34%, 46.2% and 50.3% higher data compression than libvpx-vp9, x264 high profile, and x264 main profile respectively.

Industry observers found one of the most compelling features of AV1 to be its royalty-free licensing model, unlike existing technologies such as HEVC that require royalty payments and prevent widespread adoption in open-source projects.[4][5][2]

Cisco’s Thor development was started in response to HEVC being perceived as unviable for use in open source and freely distributed products,[6] but the work of Google and Mozilla (previously under Xiph) on royalty-free video predates this event and is not attributed to HEVC’s licensing woes,[7] although also seen as a problem by Mozilla"

From wiki.

I don’t like Prores because its eating up disk space like crazy, but I would like to see that as an option next to something that is good quality and good compression like Av1 or H265. And propres for people with extra disk space or important commercial projects. For everything else it would be overkill. We really need something with good compression and good quality. Av1 or H265

1 Like

I do that, but it is a reencode of the h263 that VEAI has written. Avoiding multiple encodings is the goal. Since they use ffmpeg internally it does not seem unlikely that a menu option for output encoding could be provided. Ideally at least h265, with a further option for quality. for h265 a crf of 28 is the default, higher for smaller file, lower for higher quality. From here that appears (as a coder) to be an easy option to add.

Back when 1.2.1 was released, bowen.wang wrote: “GPU cmd line option in” - I hoped that it might mean there was something we could do, but despite a few asking that statement was never explained or documented as far as I know.

1 Like

I’ve tested Video Enhace AI recently and found that it is unable to handle VFR (variable frame rate) Video. This is very common with mobile phone video and some older recordings.

Has this been fixed already? :slight_smile:

1 Like

It can handle variable bitrate, but it make some issues, like removing some frame at the end of the video.
to avoid this issue, best si to re encode the video in constant bitrate until it’s fixed.
(no idea when or if it will).