This isn't a bug, when you attack your selected unit(s) tries to get in charging formation, but because the enemy is running away your selected unit(s) has to reposition themselfs all the time.

If you want to kill routing enemies, just make your cavalry run at the direction the enemy is running. Your cavalry will run right through and kill most of them. The enemy won't even change direction or anything.