I think so, but I'm not 100 percent sure.

But you can make new buildings, replace the temples completely, which have the same effect, just don't put them under the category of "Temple Of Something" and they won't be mutually exclusive.