I would think morale penalties are most important as it affects all units, while stars just affect units around your general.