For videos, you want more space. This SSD is fine for the operating system. I recommend getting more storage for TVAI, since it currently does not have any output formats that are good for final viewing. Because of that, you ideally will be outputting to something lossless.
I process videos into images and then back to videos, so maybe I think about storage more than others. A 2 hour video takes up about 1TB of space in PNG format.
An idea I had not long ago, is get like 2 to 4 cheap small SSDs and create a stripped volume (RAID 0) with them. The more drives you put in, the more speed it gets. Works the same way with HDDs. If you can find 3 WD Velociraptors for cheap, they’d get you crazy good speeds. The speed I look at is, after the cache has been filled, how fast can a drive copy really large files onto itself. My 2TB Seagates can do it at about 25MBPS. My 4TB game console USB drives can do about 50. An 8TB WD Red drive does 150, and surprisingly, a 512GB WD Velociraptor can do 100! (They only come in 1TB max though.) Most SSDs are not made to last as long under that kind of load, and they preform even worse than HDDs once the cache is filled, but putting them in this configuration would solve all of that.
Anyway, make sure your motherboard has enough SATA ports.
@ForSerious - I get 300 to 350MB/s sustained data transfer speeds on SSD (internal) to SSD (USB 3) for large video files of 200+ GB each, basic SSDs like Crucial CT1000/MX500. Having the extra drive is a godsend. Thought it was worth a mention anyway.
Not the same case, but at least it shows you’ve got some good drives that don’t wimp-out fast. Pretty sure all SSDs are made with an SLC cache and than whatever the main storage medium is. The issue with processing in TVAI is that, unless you set it up to use all different drives, by default it might write a lot to C drive (That’s the default for Windows TMP and TEMP folders), and then it has to read a lot while writing a lot to the same drive.
So in my case, I’m taking a DVD movie that’s maybe 6GB → upscale to PNG, now 1TB → Interpolate with Apollo, now 2.5TB → convert to H.265, now 4GB.
At that part where it’s doing Apollo interpolation, that’s the most intense reading from, and writing to, the same drive.
Some SSDs can keep up with the SLC cache, but some cannot. My use case is self-inflectedly extreme. My hope is that if I can get my case to run smooth with no slowdowns, everyone else should have even less slowdowns.
Yeah, I tried a variety of virtual machines via Paperspace and found that better GPUs weren’t making much difference past the mid-range (I think around RTX4000), probably because I couldn’t choose the other specs.
Even my older gaming laptop (i7-9750H, 2.6 GHz, 6 cores, 12 logical processors, 16 GB RAM, RTX 2080 8GB GDDR6) seemed at times slower than my Surface Pro 8 (i7-1185G7, 3 GHz, 4 cores, 8 logical processors, 16 GB RAM, Iris Xe G7). The 2080 appears to score 5-6 times higher than the Iris on benchmark tests. So I’m currently planning to prioritize the CPU in my next purchase.
I was trying out the benchmark feature and got unrealistically good speeds, which I believe you discussed on the benchmark result threads. Every machine I’ve used for TVAI has always started out with a relatively high number of FPS, which quickly declines to a fraction of the initial speed throughout the first few minutes of upscaling. On the other hand, previews and very short clips can run pretty fast.
Based on your explanation it sounds like this is when the CPU cache is filling up. Does your choice of exporting to PNGs provide the side benefit of maintaining initial speed, because the individual outputs are so small? That sounds even more appealing…although I’m not sure how slow and novice-unfriendly the process is to convert PNGs to H.265. (Thanks once again for your expertise!)
I used batch files when I started out.
Here’s a sample bat file: "[Full path to non-TVAI ffmpeg.exe]" -hide_banner -stats_period 2.0 -nostdin -framerate 23.976 -y -i "[full path to input folder]\%%6d.png" -c:v libx265 -crf 22 -pix_fmt' yuv420p -preset slow -x265-params aq-mode=3 -sws_flags spline+accurate_rnd+full_chroma_int -vf "colorspace=bt709:iall=bt601-6-525:fast=1" -color_range 1 -colorspace 1 -color_primaries 1 -color_trc 1 "[Full output path].mkv"
Change all the paths. Double check the -framerate and -crf values. I just put the most common values I use.
The colorspace stuff… This works for DVD to FHD. Might not work for other resolutions.
I don’t think it’s that. CPU cache is for repetitive tasks and is usually in the megabytes range of size. I do think there is something that fills up. My first guess would be SSD cache, but several users have shown that their SSDs can handle the intensive writing that TVAI can do. So that might not be it.
I have noticed that dark and empty scenes process faster. Like end credits.
Well I feel like an idiot. Now that I have a higher-end system and am taking this hobby seriously, I finally tried using ProRes HQ for a big file instead of VP9 Best. With ProRes, the speeds do not slow down at all. I’m getting a sustained 40+ fps for Proteus 480->1080, as opposed to a sustained 2.5-3.5 fps using VP9 Best. And compressing with Hybrid really doesn’t take long – though it’s slower at around 20 fps. Anyway, I’m very glad I followed your advice to compress later!
Drive throughput isn’t important as TVAI reads the source file at a pretty low rate (usually well below 50MB/s) so even a single hard drive can keep up. It writes in larger chunks, but as long as the drive has a normal amount of cache, that’s not an issue either.
Personally, I’ve never seen the drive be a bottleneck. Maybe some REALLY old SATA 2 drives with little or no cache would be a problem. I can’t imagine any SSD being a bottleneck. Even if you were reading a 60Mbits/second source that’s still less than 8MB/sec. and well under the average speed of SSD drives.
I bought two cheap 2TB SSDs and configured them into a stripped 4TB drive. While processing in FHD PNG files, after about two months I started to see speed decreases because everything had to wait for the SSDs. I do multiple passes in TVAI.
Because of that, I replaced them with 4 used 1TB WD Velociraptors.
SSDs do lose speed after much writing. PNG is a little extreme, but I recommend that everyone should process to some lossless format. The size differences are not going to be much compared to PNG. (Okay uncompressed tiff would be more extreme.) My little anecdote doesn’t even include UHD size files.
So my conclusion is this: If you can find SSDs that are made for heavy writing go for it, but it will eventually slow things down. (Could be months or years depending on so many things. They last longer and write faster depending on what cells they’re made with. From longest lasting to shortest: SLC, MLC, TLC [3D NAND], QLC and I’ve heard they’re making a quintuple layer cell now.)
HDDs are slower, but putting several in a stripped configuration will get the speed needed. They will have the same speed until they die.
You did TRIM the SSD regularly, right? Without regular TRIMing, writing to the SSD slows down because the memory controller has to look for no longer occupied cells. When TRIMed, all unoccupied cells are marked as available, saving the controller a lot of work at write time.
I use a Crucial 500GB SATA3 drive as my write drive for TVAI. I have 4 PCIe4 NVME drives available (2 of which are Samsung 990 Pros) but since I write to the drive and then move the files to my NAS I don’t want to wear out the more expensive NVME drives. When I started I using this SSD I had previously written about 1 TB to the drive. I have now written and erased over 41 TB and have used 4% of the drive’s lifetime. It still can write at over 300 MB/s - unless there was a video with over a 2Gbit/sec bit rate it would still be able to keep up.
To be fair, TRIM is usually enabled automatically in Windows 10. I won’t even check (let alone do it manually) unless my Crucial SSDs drop below ~340MB/s on VERY large file copies (my main use case). That’s happened only once in the last 5 years, I googled how to re-enable it, and now I just let W10 do its thing.
Would that do anything? (And I agree that DaveL is correct about Windows 10 and 11 being built with optimizing for SSDs in mind.) I’m writing ~2 TBs to the drive over the space of ~2 hours, then deleting it once it all gets converted back into movies. Then I do it all again for the next batch of movies. Not the typical use of an SSD. I doubt there’s a way to tell the SSD to not bother emptying the SLC or DRAM cache into the NAND because it’s going to be deleted in a few hours.
Anyway, the point is some SSDs just can’t handle the load as well or as long as others. So we cannot say that any SSD will increase speeds in TVAI.
If a drive that has been properly maintained slows down so much that TVAI can’t read or write as fast as it wants to, then there is something seriously wrong with the drive. TVAI simply doesn’t read or write that fast. Even a TVAI process that reads a 30mbit/second original at 3X real-time will struggle to reach 100Mbits/sec.
While Trim is SUPPOSED to be automatic (issued after a delete), I have seen many Windows installations (including my own Win 11 install) where write to SSDs slows dramatically and is restored after an explicit TRIM. Automatic trim in Windows has been flaky since before Windows 11 was released.
Finally, it is a fact that you can’t write to static memory as fast as you can read from it. Most drive specs only measure write processes for volumes less than the drive cache. If you over-run the cache, write speeds drop dramatically - often as much as 70% slower, including the latest and fastest NVME drives.
Sure…but not what we are discussing - i.e. the virtues of SSDs versus HDs. The suggestion was made that striping SSDs was required to support TVAI. My point is that any SSD should be able to keep up with a “standard” TVAI use case without issue.
