(Storage) Space adventures

…it’s an adventure in space.

Over the past year, I’ve been trying out new and different ways of managing storage drives for processing videos in TVAI.
The requirement: Be able to write 16bit 1920 by 1080 tiff files at full speed. It gets pretty demanding once you try to interpolate a three and a half hour long movie to near sixty frames per second. Oh and I’m not going to pay 900 bucks for an 8GB SSD that’s rated for this kind of writing.

First setup: 4, 1TB WD velociraptors configured as a striped drive array.
The reading and writing to the same ‘drive’ slowed things down.
So next I added 2, 2 TB cheap SSDs in a striped array. I would read from one array and write to the other. Those SSDs maxed out at writing 20MBps once the cache was used up. This was the slowest configuration by far.

Then I asked the internet for SSD recommendations. They mostly scoffed at my question and didn’t give helpful insights. But in the midst of the noise, someone suggested the Samsung SM863A. An enterprise SATA SSD that’s old enough to not be useful in modern servers, but new enough to not be used to near-death—and they tend to be sold for about 100 each, used.

I traded out the cheap SSDs for those Samsung ones. That was pretty good. I was getting way better speeds and it didn’t look like I was maxing out the HDDs too often. I made the mistake of turning off the Windows write caching on all the HDDs—then never remembered to turn it back on.
After a long time of running it like that, I realized that TVAI was only processing at 14fps.
Enabling the write caching put it back up to 30fps.

For my last adventure, I got two more of those SSDs (PM863A variation this time. [I think they run the same.]) I increased the size of the SSD array to 3, 2TB SSDs. And I tried to figure out how to add the other SSD to the HDD array. It didn’t really work. So I set my system environment variables to point to that last SSD for all temporary files.
TVAI is now processing at 40fps.

Summary: 4, 1TB fast HDDs reading from or writing to 3, 2TB SSDs and Windows set to use a fast SSD for its temp files, equals top speeds for FHD tiff files.

1 Like