Whast exactly is the trick to move the rotation arrow from the centre of a selection to somewhere else?

And also how do I avoid this? I find myself occasionally moving the arrow, when I want to move the object and move the object when I want to select the arrow?

If I may suggest a change, then it would be to introduce a long press (until the colour of the arrow changes to purple (or the selection color for that matter)), which then has the rotation arrow selected and ready to move around.