Sometimes they rally easily, other times I can't get them to rally at all even when there are still many next to the unit and they are near my general. I have no idea what controls it, but I've not found the general's rally ability very useful. Some of the units that DO rally are the ones closer to the enemy and they get smashed before I realize they rallied instead of the bigger unit near my general. They seem more inclined to rally with distance from the enemy, than by what my general does.