Using VEAI 2.4.0 / Artemis HQ v11 / Chronos v2 all via ProRes 422 HQ, I am working on an animation (Appleseed 2004) and want to know if there is some guidance on workflow. Through trial and error I know I need to color grade before upscaling, but would frame rate conversion come before or after upscaling? Developers would clearly know certain assumptions that would make one scenario more preferable. Current frame rate is 23.98fps and the closest doubling is 50fps and I have stripped audio out before all processing and will add it back in at the end. Upscaling is 1080p to 4k.
But I’ve noticed some wonkiness in the interpolated frames (like a spinning staff not being straight as it twirls), or in more complex scenes involving natural terrain (grass, dirt, snow).
Upscaling with Artemis first and then interpolating with Chronos might yield better results; perhaps having more information (higher resolution) for the AI algorithm to read from would eliminate a lot of the erroneous interpolations.
Running some tests I have noticed that if you run Chronos after upscaling the processing time becomes prohibitive, so it was better to do it before upscaling. I could not confirm quality difference doing it before or after, but I did notice some artefacts with Chronos V2 processing. They include:
Heavy blurring when moving from one scene to another; and
Blurred interpolation between frames.
The last one is probably a design choice, but the first should be avoidable.
Cheers S
Yes, it’s for that reason I personally run Chronos before at the lower resolution. I’ve run the Chronos V2 model on a 3.5 min 1080p MV and that alone had taken about 4 hrs.
You can cut down the artifacts that occurs in Chronos by running a cleaning pass beforehand; less false details to interpolate from.
Actually, denoising and deblocking is built right into the Artemis AI model (and many of the other models as well) and it will do so at any percentage, not only at 100%. The setting ‘100% denoise/deblock’ is so named simply to clarify that it does have a function, even though the output will be at the same resolution. Likewise, the 200% setting could say ‘200% denoise/deblock/upscale’, but it’s not necessary.
You can test this yourself by feeding Artemis a noisy/blocky video and seeing how it becomes much cleaner at 200% upscale (or any other upscale resolution).
Yes, if I was only upscaling resolution I’d only run just a single Artemis pass. But for upscaling resolution and increasing framerate, I’ll run a 100% Denoise/Deblock pass first to reduce the amount of artifacts in the frames the Chronos AI model will interpolate frames from. Afterwards I’ll use Artemis to upscale resolution. It saves time increasing framerate from 720p source instead of 1080p or 4K source, even with the initial 100% Denoise/Deblock pass.
For my current project the time would’ve looked like this (per episode; time tested on same file):
Artemis upscale 720p to 1080p (5.75 hrs)
Chronos increase framerate to 59.94 fps (11.75 hrs)
17.5 total hours
- vs -
100% Denoise/Deblock (2.75 hrs)
Chronos increase framerate to 59.94 fps (5 hrs)
Artemis upscale 720p to 1080p (5.5 hrs)
13.25 total hours
It does when the source files are low-to-mid quality. I did the bulk of my preliminary testing with the Dead Fantasy series and increasing the framerate to 120 fps. Without the cleaning pass, the Chronos AI would interpolate frames with weird, stretched, rotated or pulsating textures. Though this example is a bit extreme since Dead Fantasy’s fight scenes are so fast paced.
It matters less for high-quality source files and can probably be skipped.
My current project has mid-quality source files (720p at 3800 kbps); for most scenes it wouldn’t make a difference. But the low-light or night scenes tend to have a lot of blocky compression artifacts and those were being pushed through and made more noticeable by the Chronos model.
I have VOB files that are 480i - 29.97 frame rate.
the settings I use are upscale to HD and 2x deinterlace (59.98)
I have also done the upscale to HD keeping original frame rate at 29.97
Using chrono fast and chrono with Dione TV along with Dione TV halo.
Not seeing much difference between the mix of these settings.
Am I gaining anything by 2x deinterlace (59.98) or should I keep original frame rate?
