Funny, I have the quite the opposite reaction because when I ungarrison they stand there until I'm about 10 meters away. Give em cannons and they will assault including the cannon unit (how stupid) but they always attack the gatehouse so its fun to see foot knights defend of an entire army.