Why do you need to destroy barracks/temples? I've never done that before, and I've always been able to build temples or upgrade barracks.