LTX has a lora where you can extend a video in one or more directions by choosing the amount of black space to fill in with generative ai that is temporally consistent and visually consistent with the rest of the video. It’s not 100% perfect, but it works fairly well.
Top:
Bottom:
Left:
Right:
I can enter any number into these fields above and it will extend the video by that many pixels on that specific side.
For example, if I have a video of someone walking towards the camera from the waist up in an office, and I also have video of the full body walking towards the camera in a different part of hte vidoe, I could choose to extend the bottom portion of the video by as many pixels as I want so that the model will fill in that space with video that matches the rest of the video content.
The limitation with the ltx lora is that you can only run it up to about 20 seconds and you aren’t able to incorporate the entire context of the video. It would be awesome if there was a Topaz model that would allow for this type of technology to be applied to longform video.
Another example would be if I had a show in 4:3 and I wanted to convert the entire thing to 16:9 while being as temporarlly consistent as possible, and ideally the model would be able to look at all parts of the video to decide what details should be used to fill in the sides I choose to fill in. With the ltx model, you need to use a screenshot, but this wouldn’t work for a longform video because there could be many different scenes with different angles. The model would need to understand which screenshots to use to pull the data from.
