It's not your "privilege" to exceed the speed limit either, or to park in an inappropriate place. No-one is denying your stance, merely defending the alternative stance that is that is not unreasonable to re-site a very low-risk item, notwithstanding legal "ownership".
OK, it's "wrong" to interfere, but life must go on...
I had no "right" to attend my neighbour's property last week when they were on holiday, to make a gate secure...but I committed no offence that would have resulted in a conviction. I would call on negotiorum gestio in my defence, and the Court would dismiss it de minimis anyway...as I said, it's the "degree", and that is a judgement for the individual. Trust me, unless you cause trouble or damage by moving a BT socket, you are unlikely to face prosecution. On the other hand, if you move the gas meter...well, that's a whole different scenario, in my opinion.
Now, I see my glass is empty...so if you will forgive me, gentlemen and ladies, I will pour my nightcap and retire for the night, wishing you all a good rest and a happy awakening.