I guess this was the case (installing side-by-side with 2.2 and not touching icons). Completely unexpected for an upgrade.
So, after uninstalling 2.2 and 2.3 and installing 2.3 afresh it works. Except icons were created only for Administrator… Reported a ticket for this.