Yes......and no.
CA, as a business, certainly needs to turn a profit. However, the way it has chosen to do this, IMHO, is to appeal to the casual gamer by adding tons of new features irrespective of whether the AI can handle them or not. This trend started with RTW and has continued to today. Economically, judging from the record sales of ETW, this was a wise decision but....................only because there is no real competition in this genre, allowing CA to pretty much do as it pleases.
I disagree that designing a challenging AI is akin to building the Pyramids. When CA started with STW & MTW, the simplicity of those games made for great gameplay as the AI was capable of handling what needed to be done to actually win the game. With RTW, the decision was made to go for the 'glitz and glitter' and the AI has never kept pace since. Since 'glitz and glitter' sells CD's, that's what CA has concentrated upon. I'm sure there are competent programmers over there, capable of designing a strong AI (it don't gotta be perfect, nothing ever is).
A recent experience of mine hits right to the heart of the matter, I think. I am enrolled in a program to teach me the basics of game design. In my very first course, the professor asked our class this question: What is the most important feature of game design?
Two of us answered for a strong AI and replayability. The rest of the class were for a$$-kicking graphics and complex play. I'm 56, and the other person who answered as I did is in their mid-40's......the rest of the class is late teen's, early 20's.
That about sums it up, AFAIAK.................................

Bookmarks