As you have a separate tab shown to edit the faces, if there was a radio button offering to only use AI for each gender it would stop grandma getting a more manly moustache than mine. Instead of only showing a Yellow or Gray rectangle around each face, you could show Blue and pink if the gender is set manually. I’ve also had it add very lushes eyelashes and plump lips to some of the men in the upscaled photos.
Are these digital photos or scanned prints you are talking about?
Just so you know, your request is a good one because the current models have not been trained on anything except digital images.