Although the current results of interpolation and upscaling of 2D/Anime content with TVAI are not bad at all, they are not as good as the results of other technologies that are free of charge.
In interpolation we have one called GMFSS, it produces the best quality interpolation I have ever seen, in fight scenes every interpolated frame looks native because it has no artefacts whatsoever. Although being optical flow, repetitive patterns and static elements such as opening credits are its biggest weakness. It also brings a level of smoothness to character movements that I have never seen in any interpolation software before.
On the other hand, in upscaling, we have several technologies that are better than the current TVAI upscaling:
- ShuffleCUGAN
- RealESRGAN (Anime6B model)
- RealCUGAN (for me this is the best one, since it restores artifacts and doesn’t screw too much the small details)
- SwinIR (good quality, but consumes excessive VRAM and is excessively slow, me with a 4090 I’ve even to enable 512x512 tiling to be able to upscale a 1080p video to 4k because it exceeded the 24GB)
Here’s a 10mins clip of a anime film, it has everything to show, backgrounds movements, explosion effects and characters movement. You’ll see that is perfect, interpolated with GMFSS (“Union - Fortuna” model) and upscaled with RealCUGAN: Bubble_LosslessCut_GMFSS-2x_RealCUGAN-2x.mkv - Google Drive You must download the video to be able to watch it.
TVAI needs to improve its processing of 2D/A content as the results for interpolations and upscaling are not that great, and adding these kind of technologies would make it better than it already is, at least for IRL/CGI.
