I admit it did come in handy later in the game when I finally accepted a Crusade mission to Antioch. I landed my army on a Sicilian held island to avoid some enemy ships which had a significant manpower advantage over my own fleet. Because of the Military Access agreement, my relations with Sicily did not fall because I was in their territory.
Not just that, but it also hits rep which makes all diplomatic negotiations harder and when trusted alliances get enabled, (as several mods and probably the 1.2 path do), your reputation matters when determining if an alliance is trusted. Thus walking all over other peoples lands when not at war means it's nearly impossibbile to keep alliances.