This often happens to me when a single soldier gets separated from the rest of the unit. I had a lot of trouble chasing a unit of Pantodapoi Phalangitai because somehow one soldier out of 25 remaining had managed to be in the forest around a mile (don't know how that happened) away from the rest of the unit. My cavalry determinedly stayed in the exact midpoint between the large group and the single soldier, and so were too far away to attack them.