God, of course I forgot the most important thing : block spies/merchants/assassins/priests ! When you have the luxury of having natural borders (either bridges or moutains), sit peasants there, one unit is enough.

Agents cannot go through military units (don't build forts though, spies can infiltrate them and pass through. Or you can build forts and park spies of your own in them of course, but that's more expensive), and if the peasants turn brigand... well, they're peasants aren't they ?