Setting difficulty level aside, I find the timer against some specific things:

(1) Phalanx. Phalanx are too slow, so phalanx users will find time always running out.

(2) Larger unit size. Units took more time to die. A town center mob-up will take a good 5 minutes in huge unit size. In huge unit size also everything takes place slow...

So if you like to play Greeks/Romans who have a lot of slower infantry and enjoy huge unit size, the timer should be always against you.