I haven't tried it, but judging by the behavior of the Faction Leader and Faction Heir (big units) and the 'normal' family members (smaller units) the size doesn't seem to have much effect on their behavior.