On hard XL, the AI tends to not overproduce troops and wreck their economy. Even on expert though, you should easily be able to defend against 4-to-1 odds in less you don't have a decent hill to stand on. You just have to kill the general and time your cav charges right.