I´d rather modify the "interlooper" trait for gouverneurs only, so that the pure entering/staying for a while in a settlement under a client ruler won´t be taken as affront, since the particular FM might be simply a guest/resting etc. Same for "impostor". There are situations in game when one better keeps the generals in a settlement, like greedy enemy diplomats, a superior army outside of the gates, or simply to rest a general who is suffering from a frosty winter.