When the AI retreats, he doesn't do it smartly, i.e. have a few troops to cover the retreat. All of them just retreat and he doesn' care if you slam a few cavs into his rear at all or only hunt him with 1 or 2 isolated units he could kill to reduce his own losses.