Penalty is enough to get anyone running after affected and your full cavalry charge to flanks.
EDIT: tried this with silver star peasants and all random units, none at golden stars thou.