This may be kind of reaching ,but have DenoiseAI Load Mask AA Trimap for areas to ignore ,to process and analyze .
Green Process,Red Skip,Blue consider.
One thing it could do is contrast where there’s a border of different selections.
Like Blue meets Green.