I find that, if I've got a general with -2 Morale vices, he's pretty much good enough to win the battle anyway (having seen some serious action) so it's not an issue.

But I'm a big freak for religious buildings anyway. I usually build them before armouries. So, morale penalties don't usually affect me.