This has never happened to me either on my RTX 3080 Ti.
Really looks like a driver glitch. Overclocking? Video card that start artifacting when overclocked usually show long, drawn-out lines. Not saying a single quare is impossible, but never seen that with OC-ing.
Question is, of course, does square show up in output too?