I haven't been able to avoid it either. And getting rid of the people with those bad vices doesn't help either, as it takes time to find a good replacement, and the new people end up getting them as well. Its a balance thing, and a worthless one imo. But the worst I have seen myself is -20% trade and -20% farming.
Bookmarks