Ctrl-A also works, It selects all units that are in the city you are currently looking at. Then you ctrl-click on one or two units to unselect them to leave them behind and then move the rest out.