Video Enhance AI thoughts and requests

Unfortunately, there is compression artifacts like blocks & banding, etc. For example, just a brief look at one of my MP4 outputs.

https://i.ibb.co/HBnkfD6/In-The-Closet-AI-x2-00x-mp4-snapshot-04-15-2020-03-23-16-15-37.jpg

I can assure you that the source is very good and doesn’t contain such problem at all. No blocks, no color banding, little noise.

There are other examples as well, if you look closely. Yes, it’s at a very high bitrate, but the quality isn’t exactly what you’d expect in such bitrate. My own encodes usually don’t have visible blocks & color banding, and can be achieved in a much lower bitrate. I’m confident to say, that with the proper codec and encode settings, this can be improved by a lot.

1 Like

any gadget/tv from 2014 to now should be compatible with h265

does this happens on every setting (LQ, HQ and CG)? And what are the specs on the original video (res, codec, bitrate)? In my experience VEAI usually lessens the amount of blocks. I always use HQ because it seems to me to be to have better ‘errors to image quality enhancement’ ratio, but it may differs depending on settings and original video quality

Have you confirmed that the problem is not present if you output png (lossless) then encode the frames?

I would like to have to option to resize the original video and let the program sample the smaller video I want because I notice that for some lowtier videos lowering it makes everything greater!
Anyways its like having a whole CGI team at your home this so amazing I cant believe this software is real ! Holy shit my brain explodes by this software!

No, I haven’t tested the same source again, and I assume it’s the compression’s fault. If it’s not, then it would be a bigger problem.

In my latest PNG output test, I found that the output has small blocks near a few edges where there weren’t any blocks (very smooth). This is very upsetting as well.

I want to echo the call for better encoding options. The H263 codec is giving me playback issues on my Vero4K running OSMC (e.g. Kodi.) It plays properly on VLC player on my Windows PC but the Vero4K has crazy bands that make it unwatchable.

The Vero forums recommended I encode my video with H264, but I currently don’t have that option with Video Enhance AI.

3 Likes

I’m guessing they went with h.263 because they think it might be “more compatible” with older players. h.264 is pretty much the defacto standard today. I have equipment from 2010 that handles h.264 just fine. They really should change it, or give users an “advanced” box with some options for video encoding.

2 Likes

Reencode with AVC/h264 or HEVC/h265 with handbrake. I sugest going for a ‘Trial and error’ with a 10 second clip first with different bitrates, but as for myself I got good results with h265@6,5mbps or h264@8mbps for 1440p VEAI video files. Deppending on image quality and resolution you may want to use higher or lower bitrates

I finished a lot of small old Vhs/S-Vhs footage. The results from clip to clip is very various.
I testet each kind of pre-rendert down- or upscaling in order to deal with trial on error.

On the end It depends on the elements/content on each clip:

  • footage which were made e.g. in/from towns/city’s with mostly buildings in the picture brings the best
    results; sometimes near perfect.
  • footage in a mix of buidlings and green (Trees, grass) shows first big problems: to all green
    plants VEAI adds smaller or bigger structures like a ball-net >>> not usable!
  • footage with only landscape content shows real big problemes with new added smaller and bigger
    structures >>> absolutely not usable on the end
    In order to get rid of the new added structures, I pre-rendert the landscape footage with the plugin Neat Video. This tool reduce picture video noise very impressive. I very carefully reduced the noise.
    Afterwards I used VEAI: yes, the added structures on landscape content is nearlly gone, but the picture is now “washed” and too smooth without sharpness. Also here I tested all resolution sizes without any improvements.

So VEAI needs definitly more and better AI templates, which can recognize in the footage scenes with e.g. more buildings and/or more landscape and green plants.

Have you tried the 1.2.0 update?

OK, it tested the new 1.2.0 version.
The rendering process with my GTX is faster.

Again: I tested my common old VHS/S-VHS video clips with different pre-rendered resolutions:
640x480, 720x576, 720p and 1080p.
I see on each new named modus - I tested each new (named) modus in VEAI - no improvements on landscape footage. I use since many weeks in testing landscape footage several clips von Monument Valley. You have there a good mix from big stones, table stones and green (plants). Even on the table stones and much more on green plants VEAI adds the always named small point structures like a net.

The 3 variations of Artemis modus are not usable for low resolution, noisy and blocking footage.
The improvement results are always noisy, blocking and has no clear edges like e.g. the CG modus.

I tested alternative other landscape footage like Yosemite, Sequioa and so on. Same as with above named footage: The only modus, which brings a real improvement is the CG modus. But here again the added point net structures on e.g. greens/green plants/fields are not helpful.

I get a small improvement - also with the versions before -, if I very slightly denoise my footage with Neat Video and us it afterwards in VEAI. But also here on some clips I get the point net structures added.

It seems the common thing I hear around different areas of the net about this program is “LQ (now renamed Artemis) is no good.” So it sounds like the 1.2.0 update didn’t change things much. When I tested it I could see what people meant in that it seemed to try to create detail that didn’t exist in the source, making it look more like artifacts. Hopefully this could maybe get better in the future. But I still think we’re a ways off from feeding in crappy EP mode VHS and getting footage that looks like it was shot in modern HD. But with the way computers and technology is progressing, I really do think we will get to that point some time in the future.

Thanks, that’s what I’ve started doing. The other issue I run into is that the mp4 produced by Video Enhance AI has the audio horribly out of sync. I’m not sure but I think it may be because my source is 24fps and the program seems to default to 29.97 fps, and I don’t see an option to adjust the fps.

That the idea/concept of VEAI from the scratch by Topaz could work pretty good is already known.
For example: I have old bad footage of Las Vegas from 1992 via S-VHS. It’s very noisy, has no clear edges/lines and has common blockings. Overall: the picture is really “dirty”. Here VEAI works really “magic”. The results are really really impressive. Nobody including myself could believe the comparison before and after. No remarkable noise, clear edges and lines, deblocked and sharper picture. Great Great “magic” :slight_smile:
Also impressive are results of the night/dark footage of Las Vegas. Really impressive!
Same mostly with footage e.g. from San Francisco (also from 1992). BUT: footage taken e.g. in parks of San Francisco shows the problem: greens/plants/grass/trees became the named additional structures.
Perhaps it need much much more AI learnings, which depends on landscapes, especially on handling of plants/trees/grass. Perhaps it’s on the end not possible…I don’t know. But I hope very strong, that Topaz will work hard on it, cause using VEAI only for “town/city” footage can’t be the business case.
From my side I always stress adjustment sliders. Perhaps it need on the end only one to adjust the added fishnet/pointnet structure.

Have you checked fps before and after you render with VEAI? I’ve only had the out-of-sync issue when I render a video that was previously rendered in VEAI. In other words, I take the original footage, run it through VEAI at 100%, then run it once more and it will have the out-of-sync issue.

Using Mediainfo, I see the fps before and after are the same. They’re 23.976. Both the original footage and the VEAI rendered clip so I doubt that’s the problem. I have not tried it in version 1.2.0. I will wait for the new release next week and some release notes.

Have you checked fps before and after you render with VEAI?

I just ran this test with a short clip. The source was 24 fps and after encoding it with VEAI the frame rate is 29.97. I’m using 1.1.0.

EDIT: I may have found the issue. MediaInfo is telling me the source file is 24 fps (which makes sense since this is a DVD rip of a TV show) and VEAI is detecting it as 29.97 when I load it. So maybe VEAI is just reading the fps incorrectly?

Oh wow, maybe that is the problem. I can’t test right now because it is currently processing a file, and still has a few hours left. But I’d be interested to see if VEAI reads the files that it produces incorrectly. Even then, mediainfo is still telling that the final file is correct fps. Will update once I test.

Sounds like your source might have some kind of Telecine or 3:2 pulldown going on.

How to check this: https://tommycatkins.com/2020/Neuron2_Video_Frame_Structure.htm

If you have such source material, it should be deinterlaced first, then decimated (remove the duplicate frames inserted for interlacing, restoring the original 24 fps).

Try these Hybrid settings if your source is 30fps telecined and you want to get it to 24 fps:

Further testing at home with DVD:

Gaia HQ seems to do the least. Looks like a regular scale with a little detail enhancement. Tool tip says this is really intended for HD sources. (So I’m guessing 1080p to 4K basically.)

Gaia HQ-CG is a mode a lot of people seem to like. I have found that currently, in some sitations, it adds “pattern dither” types of artifacts that are clearly visible even when viewing at a distance on a TV screen. It looks similar to say dithering an image to a 64 or 32 color gif with pattern dithering. I think it’s coming from the training data trying to re-create detail. This effect doesn’t appear in other modes.

The “Artemis” mode is greatly improved. THESE are the settings to use for scaling standard video (480p) to HD (1080p).

Artemis-HQ looks good but sometimes leaves details more blurred than Artemis-MQ. It can also “enhance” certain slightly noisy elements. I think it really needs to be used on only VERY clean sources.

Artemis-MQ seems to be the main setting to use for upscaling DVD and other average to good quality HD sources. It produces the most sharpness in fine details while denoising and removing some other slight artifacts. The file size is only slightly smaller than Artemis-HQ which to me says some denoising is taking place. Film grain is still mostly visible.

Artemis-LQ when used on the same sources begins to start resembling Gaia HQ. Detail is once again lost and many scenes take on a “painter” type of effect. This I believe is due to heavier denoising and deblocking. This mode is inteded for low bitrate sources with visible blocks and artifiacts so it’s mainly trying to remove those. Don’t expect it to save very poor quality VHS or 1990’s “postage stamp” sized video encodes from RealPlayer.

On several discs, including good quality 1970’s films with visible film grain (which you want to keep), and recent 2000’s movies with no visible grain, Artemis-MQ seems to be what wins every time.