Today we’re releasing an updated alpha of our next Enhancement model, Rhea.
Updated training checkpoint for Rhea.
Two additional tile sizes added to increase performance.
Note: This build is configured for TensorRT versions of Rhea, but we are still in the process of converting and publishing model files. While this is in progress, NVIDIA users may see a brief “Downloading model…” message at the start of previews and exports while the app checks for optimized model versions.
Rhea represents a combination of Proteus and Iris. The model is intended to be more accurate in preserving fine details than Iris, while also handling text in a less destructive way.
There are a few key limitations for this alpha that testers should know about:
The current version of the model is not fully optimized and speeds of this alpha are not representative of speeds users can expect when Rhea is released.
We’re very excited to have testers working with Rhea. Thanks as always for your feedback and testing.
New version is still too broken to use. 1st render (720x480 input, 5sec @ 4x output size) took about a minute. Great. Results very slightly plastic looking but much better than I’ve seen out of TVAI before.
Next I moved the output size down to 2x (hoping for a speedup) and selected another 5sec render. Estimate now says 7min!
Showing Before/After side by side views, first it showed me the before in both windows.
Then after messing around, the after in both windows. After more playing I finally got it
to show before and after as originally intended for the 5sec.
Looping is still not working - it played right past the rendered part.
I’m confused as I thought this was a test for new models.
The previous release of software more or less worked with looping, side by side viewing, etc; not this version.
Are we officially moving toward a state where TVAI will be unusable with any GPU that is not an Nvidia with tensor cores? Just thought I’d ask before buying my next workstation.
The foreground / Commander Data looks excellent but the model is trying too hard and sharpening faces (and Yar’s hand) that should remain blurred in the background - not be forced into focus.
Rhea needs to be trained to respect depth of field more.
Rhea - faces that should not be in focus are circled:
Guys, i have very good source data for model training. Download 50 cent’s - In da club music video (vob), then download prores version of 4k upgraded version of this music video (there is no ai upscale, this is pure from 35mm film scan). This is perfect data to train your models! If you want i can provide you those files. Just where to write?
There seems to be a point of depth of field or original image resolution at which point the model shifts to smoothing. The shot below is from 720 x 480 S-VHS circa 1989. Note the brick pattern changes from the bottom of the shot to the top. The bricks go from decent looking recovery to plastic as do other details at the upper left part of the frame.
BelowGreatly improved results from a crappy RCA consumer VHS recording compared to other models. There is a threshold behavior that Rhea exhibits between really great recovery and almost good enough recovery. The model simultaneously produces some really impressive results and then some oddities. The oddities being this threshold between areas of great recovery of clarity and detail, and then we have the plastic, craquel and what looks like depth of field focus changes. The distinction of these behaviors represented simultaneously in the same image is unnatural. I truly hope additional training can fix these issues.
Look at the tinsel in the grab below and you will see a blurriness that doesn’t get compensated for quite like other areas of this shot do, and it really stands out to me.
Clip showing depth of field issue as noted earlier. Look when scene changes at 0:04 and the background characters look like bobble-heads due to the unnecessary enhancing of their faces (watch in full screen mode).
Probably expecting too much for any model to be able to distinguish between foreground and background. It’d need some way to tag each for different levels of enhancement.
I think Photo AI has something like that (Selecting Subject), but no idea whether that could be added to TVAI.
Yup I know it won’t be easy to solve, but if I don’t point it out it definitely won’t get looked into. Model needs to check ‘is there something in sharp focus in scene ie. a main subject? if yes, go easy on enhancing everything else’.
