I noticed this as well, and had assumed it was the routers fighting back as my cavalry engaged them. I've not observed (too much chaos) this for my units that are nowhere near routing units.