Hardware Encoding on M1 Macs

ffmpeg supports hardware encoding on M1 Macs using VideoToolbox. This can encode a video in minutes that will take hours to encode using GPU. It would be nice to have an Option to use hardware encode for H.264 and H.265 on Mac, similar to what e.g. HandBrake has.

Hi Georg, I have a Mac Studio and can confirm TVAI does use hardware encoding (via VideoToolbox) for 264, 265 and ProRes. If you copy and paste the ffmpeg CLI created by TVAI you will see something like “hevc_videotoolbox” in the command.

Thanks.

Andy

Thanks, good to know.

I’m not sure what VideoToolBox is. I just got a MacBook Pro with M1 Pro Chip and 16GB of memory. I just processed a 50 min video with just basic upscale to HD from 720p. I ran it barebones with nothing else running. I don’t think TVAI is giving me the results I expected. First, the video is a huge 20GB. Second, while the quality is good it took 2 1/2 days to process. It was absurd. Now, I have a 2 year old Windows Computer running Windows 10 and I just put the same movie in with the same settlings. It has been running for under an hour and it is already at 42% completion. Now it does have the Nvidia RTZ 3070 in it but its processor is a few years old. I think it’s an i9 10,700 as I recall. It has 16 GB RAM. I would not have expected such a massive processing difference. I am not even done typing this note and the Windows machine is already up to 45%. I’ll have to check if the results are the same in the video but I’m severely disappointed to have what I consider to be a more power Apple Computer only to find it doesn’t perform even 1/200th as fast as the Windows computer with this program. What could I be doing wrong on the Apple M1 Pro?

Okay, figured out what VideoToolBox is. How do I get TVAI to use VideoToolBox? I suspect that the problem may be that it is not using it.

Andy, Thanks for the post. I commented in this thread. I think my MacBook Pro w M1 Pro chip is bypassing VideoToolBox. To you know how I would verify if it is using it. I don’t think it should take more than 2 days to process 50 mins of video to go from 720p to 1080p. How long are your processing times?

Chris

Hi Chris. You can check if hardware encoding is being done using Apple’s Activity Monitor. While TVAI is processing a video, check if there is a process called VTEncoderXPCService listed in Activity Monitor. My Mac Studio is only the base spec (M1 Max) and as an example I can get around 10 fps upscaling SD (720x576 25 fps) to HD (1080) using Artemis while processing a single video. I can get faster than real-time overall if I process 6 in parallel. Unfortunately I can’t think off hand why yours is running so slowly. If you could provide more detail such as which AI model you are using etc. that might give a clue and / or others might have ideas.

Thanks.

Andy

Thanks Andy. I am using Artemis and upscaling virtually the identical dimensions to yours from 720 to 1080p. Original video is 24.97 fps. I left it as a High Quality input video in the settings but for this test I moved it down to Medium Quality. I checked and I have 2 VTEncoderXPCService Running and 2-3 VTDecoderXPCService Running in Activity Monitor. The number of Decoders appears to be alternating from 2 to 3. With nothing else running it starts out at 4.5 FPS and within 1 1/2 mins it goes down to 1.2 FPS and then at 5 mins it goes down to 0.4 FPS and then it takes forever. No clue why it’s doing this but I can’t spend 2 1/2 days on a 50 min clip.

If you or anyone else has more input I’d really appreciate it.

Chris

Hi Chris, the initial 4.5 fps sounds reasonable but the slowing down appears very much like thermal throttling that might happen on a fanless MacBook Air. But as yours is a MacBook Pro, I assume your fans would provide enough cooling. Are your fans running at maximum when things start to slow down? Otherwise, perhaps your power / battery settings are causing the CPU / GPU to throttle significantly after reaching a certain temperature?

Thanks.

Andy

No clue Andy. One thing I can tell you is that even after processing for hours I don’t hear any fans running at all. My MacBook Pro 14" fans haven’t turned on at all when I run TVAI. This just baffles me. My computer should be more than enough to handle this but takes forever. Makes no sense to me. I even tried another test running a background app to free up memory when it gets low it doesn’t help. Then I tried “sudo purge” in Terminal to manually reclaim RAM once every hour. Neither thing helped. I’m stumped. Computer is only 2 1/2 months old.

Chris