Make FP16 / FP32 choosable by user

Some cards (GCN 4.0) are the same speed with FP16 and FP32 models. But since the FP16 models require less VRAM, in some cases it would be beneficial to force the FP16 variant manually.

I have seen some examples in the past where the FP32 Model was chosen, maybe this is obsolete, haven´t checked, but in case its stillt there - my request it is :slight_smile:

Since I was just about to mess around with some older hardware I needed to check, I fired up the latest TVAI beta and ised it as a stability test…

GCN 3 and 4 cards are still very common, even with TVAI users, in the benchmark section many RX 4xx and RX 5xx pop up …

Unless I missed it in the newses releases:

Making FP16/32 chosable would enable a lot of older cards to be put to work - the compute power from GCN 3.0 to GCN 4 and Pascal as well is the same on FP16 and FP32 shaders … Running the FP16 models would only use half the memory, thus speeding things up and also enabling larger scale factors and resolutions…

I manualy tweaked my local copy, this works wonderfully and is very easy - but one has to take every single .json file , change it… with every update again…

so… a swithc for fp16 model variants for the older GEN GPU users… please :slight_smile:

1 Like