Rally does capable to get routing units back... just make sure no enemy troops following them...

Bet you try to rally troops fled from heavy combat, when enemy still nearby... their morale is boosted... but they soon dropped again because enemy still killing their friends...