If you are neutral it has little effect. But if you are allied and do not have permission it is likely to instigate a war.

Of course the Spanish were always walking all ove my lands and the second time I did it they declared war