Wow, that would be irritating, ESPECIALLY intermittency!
The problem could be Mac-specific or GPU-specific - the selection doesn't shift even a pixel for me. I can't recall a time it has ever done that. I have an ATI GPU, though on a PC, not a Mac.
Additionally, we have not heard really any complaints about this here on the forum, though not everyone would perceive a single pixel shift.
It might be good if you'd take some screen grabs or a video, to show the shift graphically when you let go of the mouse button, just to illustrate it thoroughly. Something along these lines...
-Noel