Topaz Video AI Beta v3.2.10.b

Hello everyone,

We have another beta available for your testing.

Downloads:

Changelog from 3.2.9:

  • Maps all audio streams with metadata to output
  • Added DPX support
  • Fixes issue with estimate not working for certain image sequences
  • Updates NVENC (NVIDIA encoder) - this requires at least driver 531
  • Beta build for AI Engine 3 (No user facing change)
  • Updates for Iris model from previous beta

Known Issues:

  • Videos with mismatched metadata and streams will display incorrect duration
  • Frame number preview length may shorten on app restart

Enhancement model - Iris

  • Primary aim of the model is to improve human faces
  • This model is also designed to enhance low- to medium-quality video, particularly, with noise and compression artifacts. The model tends to lose details on high-quality videos
  • This model is trained to work on all three categories - progressive, interlaced, interlaced progressive - but the main target is progressive videos
  • Since the model has significant differences from the Proteus model in terms of aim, scope, and performance, we have decided to give it a completely new name
  • Please test the model and give us your opinion. Thank you

Please upload problem videos and logs here: Submit Files .

9 Likes

The image quality is quite good and dependend on the source material maybe the best so far. Currently testing a 1280 * 544 video upscaled to 1920 * 816 with Iris V1. :slight_smile:

1 Like

Getting what is probably corrupted output files from this build. Using VLC, video stays still and audio continues to play. Happens with every source clip I’ve tried so far (using Apollo and Iris, 2x upscale).

Logs and sample clip uploaded. Also uploaded bad output file.

Edit: tried more than one output format and made no difference.

Edit #2: the output does play properly on windows media player but does not on VLC. Something has to be off though, VLC usually plays everything

Tested the source video where the artifact occur.

The black spots was not improved.
The afterimages was not improved.

The AI engine seems to have been changed, but the lack of improvement suggests that the model may be the cause.

2 Likes

What this AI Engine 3 ? :thinking:

5 Likes

Disadvantages of DPX files.

  • DPX sequences consist of many frames, often at high resolution. To export and edit a DPX sequence, you need a computer with a high processing speed.
    how about this
  • You must pay a license fee to edit in DPX on post-production and film editing programs.

It would not surprise me if this is the cloud engine pixop.com uses
great for short videos a costly exercise for long videos.

In any case ChatGPT had no clue so I might be forgiven that it is not very well understood.

The file size was the same for the one processed yesterday with v3.2.9b and the one processed with v3.2.10b.
Is there anything different in the “Beta build for AI Engine 3”?

AIEngine changes are not user facing so if you see no difference then it is working as expected. Please let us know if you have any processing issues.

3 Likes

I’m still in awe, since “Proteus v4” times, how (now) Iris v1 model handles interlaced input:

Linus looks a tiny bit better in his first Linus Tech Tips video, too. The prompt text - not so much, though. :slight_smile:


Keyboard, despite the video being very LQ, still gets a beating in terms of details. :slight_smile:

I like how Iris v1 handles some external shots in Stargate SG-1 episodes [S06E11 - Prometheus]. :slight_smile:


They’re a menace, since the series is a progressive video, excluding some outdoor city/buildings shots in some episodes.

2 Likes

Comparing the before and after image in the preview is still not easy because those frames are never the same. :eyes:

1 Like

Yes, this does appear to be model specific and we are working to determine if it is fixable with the current Proteus model or if a new model needs to be created.

2 Likes

Yes, this happens most often with high res files and can be quite annoying. The preview experience is a major project and we have run into a few roadblocks along the way.

I can not confirm this. For me it happens on any, yes on every video file which I process!

Face improvement is a very good thing, but, if it’s going to lose details in other parts of the image, then it’s really not worth using IMHO. If there could be a model that improves faces AND enhances detail… I think that’s what everyone really wants, isn’t it?

7 Likes

I’ve been thinking about how to best improve the iris model.

Iris does 2D faces amazingly well as if follows the image (Faces) on the screen and produces a result from exactly what it sees, so if an eye looks misaligned due to compression artefacts it will match the eye’s position no matter how bad it messes up the face.

I’m guessing that it has to do with topaz using a similar model from their image rescaler, which is an excellent way to apply it here for video, except video features 3D faces in 2D, what we need is iris trained with simple geometry as well as 2D facial reproduction, say an egg-shaped oval for the head, that way it can predict the face’s features position such as eyes, nose, mouth on a 3D model.

Visualise a projector projecting a face on a 3D model of an egg, and you’ll see what I mean.

1 Like

THIS^ Also, noting that grain will not work as it does not generate folds and seams in clothing and the like.

Am I the only one here getting output from this beta that is unplayable on VLC? Can someone please test this on their system?

Currently, uninstalling the previous and installing this beta now, I’ll give it a go.

EDIT: It takes a bit to uninstall as I’m using windows uninstall troubleshooting prog, as I cannot uninstall this program due to that accursed permanent deleting shortcuts bug!

1 Like