Processing on M1/M2 Macs is always Neural Engine(NE) first, in almost all cases the NE has similar performance to the GPU at a fraction of power, so it is favored. On the M1/M2 Macs since everything is on a single chip, selecting CPU or M1/M2 will have the same internal selections.
- This seems like a bug. It would be great if you can provide more information about this problem. Can you send me logs with memory set to 100% and whatever lower number that is giving you better performance.
- Memory allocations do not determine the device. But lowering the memory generates smaller pieces leading to more devices being used.
- Running in low power mode disables the use of GPU, so this observation is correct.
All the layers cannot be run on the GPU and there are certain inter frame dependencies that restrict full utilization of hardware with lower resolution inputs. If you want higher GPU usage use a higher resolution input or run multiple processes.
