For now, I agree this would be more for the tinkering type, like myself, but I wouldn’t describe it as a disaster, but more of an evolutionary thing. I think the potential is there to upscale old video to native 4K, but not for many years to come. Look at computing technology in the 1950’s and compare it to what we have today. This applies to all emerging technologies, not just AI and video upscaling.
It looks nice and has a lot of new features previously promised.
But performance is a total failure. In my case, version 2.4.0 is the fastest, 2.6.4 is about 10-15% slower, and 3.0… well.
3.0 wants more CPU resources to perform the same or worse than 2.6.4.
This premiere took place definitely prematurely.
(5800X, RTX3090)
3 Likes
I’ve been testing the new version and I can say I like it so far, because it has more codecs, more ai models to choose from and they actually all look different compared to each other.
BUT!
I found out that ARTEMIS works worse. I tested a .mov I upscaled in 2.6.4 with artemis high quality and it looks much better. More shadows in the noisy areas preserved while 3.0.0 just blurs these areas and causes detail loss. Unfortunate…
VEAI supports only 2x or 4x scaling, other scales are scaled by lanczos.
In the veai_up filter, there is a scale option and w,h option. If w,h is set when scale=0 (or not set), the scaling factor is automatically calculated internally, and “scale=2” is applied if w or h is larger than 1.2x, and “scale=4” if w or h is larger than 2.4x. scale=4" is applied internally.
For example, in the case of 720x480(wide) → 1280x720 proteus, the show export command will result.
-filter_complex veai_up=model=prob-3:scale=0:w=1280:h=720:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=0.9:instances=1,scale=w=1280:h=720:flags=lanczos:threads=0:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1:color=black
This is internally equal to setting scale=2.
-filter_complex veai_up=model=prob-3:scale=2:preblur=0:noise=0:details=0:halo=0:blur=0:compression=0:estimate=20:device=0:vram=0.9:instances=1,scale=w=1280:h=720:flags=lanczos:threads=0:force_original_aspect_ratio=decrease,pad=1280:720:-1:-1:color=black
So the flow is
720x480 → veai 2x scale(1440x960) → lanczos resize(1280x720) → padding black
The scaling factor calculation also takes the aspect ratio into factor, so there are cases where the image is scaled only by lanczos instead of the expected scale setting, which I think is causing confusion.
In the alpha version, there were no w and h options, and scaling factor calculations were done on the GUI side, so it was obvious at a glance how many scales had been set by looking at the options in the log.
2 Likes
@ida.topazlabs @nipun.nath @gregory.maddra
Until this release, I have limited myself to using the VEAI 3.0.0x betas. IMO, they were vere weak in the areas of cleanup - denoise decompression and deinterlace, (especially, deinterlace.)
As you have been doing regardless of source, my working output from VEAI is always ProRes 422 HQ.
In the new officially released version of VEAI 3.0, they have very much improved the output of noise cleanup and artifact removal, but there is still a curious “sawtooth” pattern on the right edge of the image. (I’m deinterlacing wide-screen video from a 4:3 SD source)
I wonder if this is due to a difference in the actual number of pixels or dots in that odd/even lines deinterlace (TV x2 dehalo) performs. It also makes me wonder whether there are latency issues and whether the odd/even lines in the output are out of scale all through the left to right scan. - There are several places where I can see what may be the earmarks of this problem. In several places it is obvious that horizontal lines often “bleed” into adjacent solid colored areas. It often loses subtle stripes on horizontal fabrics, extends eyebrows and creates more patterns on checkered areas.
They’ve fixed a lot of the noise and artifact issues, but numerous defects above still remain.
As for the Enhancements, especially Proteus, they are capable of doing some very good work, - That is; provided you can feed them clean material. This is especially true if the input’s colors are not overly saturated. Oversaturation can make it difficult to recover detail in small, similarly-colored areas and then some detail disappears.
IMO: VEAI 3.0 is a major advance from previous versions, but a lot of it still needs refinement.
3 Likes
“IMO: VEAI 3.0 is a major advance from previous versions, but a lot of it still needs refinement.”
yes, full d’accord!
It would be helpful to avoid misunderstandings if the weekly releases could follow a given roadmap of prioritized “known issues” .
Actually releases appear total random what gets delivered. Welcome in the VEAI improvement lottery! Evolution by try&error that’s AI.
Digitized SD video does not use square pixels, with NTSC at 720x480 they are 8/9 pixels for 4:3 and 32/27 for 16:9. So 640x480 is correct if the pixels have been converted into square and it’s a 4:3 aspect ratio. (16:9 is usually translated to 853x480 but some software uses 854x480).
Non HD video in PAL does this as well.
2 Likes
Well, that’s life; trial and error. - In the digital world it’s called debugging.
Your name is Imo. IMO stands for “in my opinion”. He’s not quoting you - and you’ve been told that before.
7 Likes
lmao, true 
I know that. The only scaling I use in VEAI is 2X or 4X. Final scaling is done in Avisynth/Hybrid.
I clearly show in the screenshots that it’s a 2X scale, and I’m only mentioning scale=0 / scale=2 because it was suggested as the fix to make 3.0.0 results “exactly like 2.6.4” results, which isn’t the case in my example.
The source in my example is 640x480, not 720x480.
1 Like
Actually I upscale 1280 to 1920 progressive videos downloaded from TV-Mediathek with these settings
Very often (but not always) I get those artifacts
How can that be avoided?
2 Likes
Either clean the noise by running Artemis Medium Quality at 100% scale first, or try turning the reduce noise slider to -50 or more.
I’ve tried the relative to auto and found no values that produced results I could tolerate. I’ve been using manual settings instead.
2 Likes
I find that switching to “manual” with
Revert Compression 50
Dehalo 10
and everything else at 0 works quite well. Adding Sharpen and Recover Detail will always add artefacts like in your example.
1 Like
Thank you, I’ll experiment these settings and report.
1 Like
I hadn’t seen that before so I appreciate it.
However, it’s not clear to me how to apply the “mod”.
You say, look for this line:
"vidArtifactType": ["Compression", "Noise", "Blurry", "None"]
So far so good, I can find it in the json’s.
However, you say this:
That’s where I’m lost. I don’t understand how you would do that. Like this? -
"vidArtifactType": ["Compression", "Noise", "None"]
…or, like this? -
"vidArtifactType": ["Compression", "Noise", "None", "None"]
I’d really like to try this, so please can you clarify this or give an example of how you changed it?
What does Proteus show for this frame when you Estimate the values. That would guide what the sliders need to be set to. Every video will be a bit different unless you know the source material. How bad is the frame with just using Auto?
We just need someone in here named LMAO now.
2 Likes
I downloaded a video from the site you mentioned. The video was 1280x720 @ 3000Kb/s.
This screen shot probably won’t do the images justice but here is what I got.
