Line-of-sight. Basically, the A.I. seems reluctant to target forts, while often making opportunistic attacks on cities. My guess is the A.I. has to see a fort in order to be able to target it. It does not remember the location of forts, unlike cities.