Well if you want to neutralise them, capture 2-3 cities of them, smash everything to pieces and force diplomacy them back? (Unless your not a FD man).

Otherwise build forts at strategic locations, ie. before and after bridges. Whether you choose to arm the fort with an entire stack or single unit is up to you.

In my Romani file I had become quite wealthy, so I armed forts around the German border with full stacks of levies. Haven't had a random attack since!