Yes! This is exactly what my use case would be which is the reduction of aliasing/moire/chromatic abberation in super slowmotion video and other cameras that employ some sort of lineskipping in their recording modes (ie. Freefly Wave in 2K mode, Sony RX10 IV, Sony RX100 VII, Sony FS700, FS5 HFR modes). I believe taking that Starlight file and doing a 2X Proteus pass with some dehaloing and adding film grain would enhance it even further and make it look more natural. The other models (even Artemis AA) donāt seem to do well in terms of reducing this kind of aliasing/moire in severe line skipped video but Starlight is def. doing it much better! Artemis LQ and Artemis Strong Halo comes close, but it removes aliasing/moire in some areas while not doing it in other areas of the image making it look uneven. Looking forward to see how Starlight progresses and hoping for a local model or new model that focuses more on improving aliasing and moire reduction.
My current upload seems to be stuck on āinitializing resources.ā Cancel or just let it run?
Should be good to just let it run, weāve seen a spike in requests over the last 30mins so additional machines are spinning up now
Iām not really a fan of cloud processing and paying for an intermediate coin currency that lowers the inhibition threshold to pay too much for a service. We went through this already a decade ago with Xbox and Nintendo points.
I rather invest in and own good hardware and have local processing.
Back to topic: The model works like magic for early CGI work: Topaz Compare
I look forward to get a smaller variant of it for local use.
Awesome sample!
I wonder if itās possible to apply for Early Access without owning a X account.
Directly linking to that shady platform from inside the application is already weird.
Had to manually enable the domain in my filter settings to āLearn Moreā.
You are already in the beta group, you just need to download the installer or use the web app linked in the post
So this is a clip I recorded with one of my super slowmotion cameras. The resolution was 640 X 360 and at this resolution, I can shoot at 5899 fps which when played back on a 24 fps timeline, you get around 245 times slowmotion! I missed focused on the bee very badly and I have to say Starlight did a decent job with it despite me missing focus and the very low input resolution. Iām sure with a few tweaks in post I could get the file to look a little better, but I think Starlight is headed in the right direction (at least for this scenario) as the older models I have tried with this clip havenāt come anywhere near this close! I tried Artemis LQ, Strong Halo and Rhea and the bee is still blurry. Even if I would not use this shot professionally, for experimentation with slowmotion wildlife, the Starlight file is def. the one I would choose for viewing.
I just used 300 points to do a cloud processing of the whole early CGI video:
Original video: https://www.youtube.com/watch?v=JOVmmml7PCs
Enhancement I did with regular TVAI: https://www.youtube.com/watch?v=ry2xKqlqmT8
Project Starlight: https://www.youtube.com/watch?v=WH4XjO-jqBg
Did project Starlight use the original video or the one you enhanced with TVAI. Did you use Gaia in TVAI?
Project Starlight used the original video as an input.
The regular TVAI is chr2_nyx3_gcg5. A perfect solution would have been using Gaia, Proteus and Rhea for different scenes of the video. But I didnāt feel like editing it together afterwards.
One more example using CGI / Gaming Footage:
Original: https://youtu.be/Pgg3VG7Hug8
Project Starlight: https://youtu.be/xe3Kh9PmMXk
Still feels like magic what kind of fidelity it can deliver using heavily compressed source material.
<Imagining additional 80GB H100s jumping into the fay>
One example of Sony RX10 III 960fps slow motion. Temporal coherence is very good, and very important for this sorrt of footage.
Nice to see another Sony RX10 user on here! I did more tests as well and itās pretty consistent when it comes to reducing the aliasing/moire/noise artifacts in HFR mode, but it does tend to oversharpen shots too much where itās akin to an unsharp mask or convolution sharpen plugin cranked way up. If they can find a way to make the restoration detail more subtle (ie. Artemis or Proteus) while keeping the aliasing/moire reduction properties of Starlight, it will look even better.
Below is an example of what Iām talking about where it does decently reducing aliasing/moire but the restoration of detail is too aggressive and the banding in background is very posterized looking. Footage is from my Sony RX10 IV in 480 fps mode. I believe with more training Starlight will eventually get to a point where the detail restoration isnāt too bold or oversharpened. They can try to add some dehaloing elements from the Proteus slider to add more naturalness to it. The dehalo slider really helps aleviate that ācrunchyā compressed look, and I hope we can get an improved dehalo element in all the models.
I agree with you, and itās especially a shame about the poor quality of some backgrounds. If they can improve these aspects, we can breathe new life into these magnificent cameras that Sony has unfortunately forgotten to update.
Yes, I hear you! Iāve been waiting nearly 9 years for Sony to improve the high frame rate recording modes! And yet Sony started to remove the HFR feature out completely at the higher settings with their Sony ZV1 Mark II and also their successor Sony HXR NX 800 camcorder so itās sad they didnāt care to improve the resolution or improve the fidelity in the 480 fps and 960 fps mode in order to give Topaz VEAI a higher input! But I believe Topaz, like you said will help breathe new life into our Sony cams that do these special slowmotion recording modes and make them much higher fidelity. Just needs tweaking which I believe they will be able to figure out.
Itās true, in VHS quality terms, that video would be really good. Though, the blurriness and fringing indicates āanalogue eraā of video processing/recording.
Iām aware that VHS tape usually has way more āvisual damageā, and Starlight has to rely on multiple frames of āthatā to recreate a single frameā¦