It's basically impossible for an army of the time to march across the desert. Also, the town is used to place marker buildings to track campaign progress. See the FAQ.