AI Engine Failure After 15 hours :( Design Weakness

After running for 15 hours and with less than an hour to go, I happily go to check and find AI engine failure. I assume that this is nothing to worry about as it will carry on where it left. This does not appear to be the case and I stand to be corrected.

If it indeed is not the case I would be curious to hear what good reason would exist for such as a monumentally crazy setup? It seems inconceivable that this would be true, particularly given the evident instability? Having to start again, will now split 55-minute video manually in 10 pieces an do one by one. I am in pain, and an engineer is to blame :frowning:

Someone please tell me the video is sitting somewhere and can be recovered

I processed a whole movie. Fortunately it took only maybe two hours. Everything was fine and the work finished. Once the moment when I realized that the application completed the job I switched to the application and selected File → Exit. Just that moment my monitor turned black and the GPU fans ramped up to full. I had to hard shut down the PC via the power button. :eyes:

1 Like

Which graphics cards work best with Topaz? It seems like the NV36ads A10 v5 (36 vcpus, 440 GiB memory, 24GB vram A10) doesn’t fully utilize its cores or GPU, making a two-hour movie upscale seem impossible. The CLI instructions do hint at potential optimizations, but clearly one can get more speed without it.

The fear of a long process failing at the last minute always keeps me on my toes. There might be a good reason for the current setup, but intuitively, it seems reasonable to break down the process into segments. This need not necessarily be visible to the user, unless there’s an error.

this is why I prefer to use 8-bit tiff rendering :slight_smile: at least everything that has been processed is not lost

I feel quite foolish now. Anything else is asking for trouble.

it’s not really stupidity, rendering in image sequence has the advantage of not compressing anything, and of being processed without being lost in the event of a crash but it wastes a little time, when you have to open an image sequence.

Our team is looking into options for segmenting output and being able to recover from an application crash. For now, image sequence exports can be used to preserve partial exports.

We know it’s not the ideal solution, and we plan to improve this experience for standard video outputs.

1 Like

Even the saving to images option did not work recently because the frame numbers of the saved images did not match the frame numbers in the application so it was impossible to continue the next frame from what was saved last. Not sure if this was fixed recently. :eyes: