I haven't played any TW games past Shogun II, but up to that point....Shogun I had the best AI, IMO. On Expert setting, it can, and will, kick your a$$ if you make mistakes
The AI strives to create mismatches: swords over spears, spears over cavalry, etc, so if you don't have balanced armies, the AI adapts and you begin to lose battles.
The primary (though not the only) reason the AI was better in Shogun I/Med I was because the map was 2-D (making pathfinding easier), the economy was far simpler than when the game engine was changed in RTW I, and there were far less units to deal with.
Not saying that later games weren't challenging and fun to play, but the original game engine allowed for a more challenging AI opponent, and if you played enough campaigns, you saw every clan have a chance to become Shogun, or at least emerge to challenge you for it. RTW was far too predictable...always the Romans in the western Med and Europe, and the Egyptians in the Middle and Far East. That speaks to imbalanced unit rosters, a lot of which was fixed in some of the great mods done by players.
Bottom line....want a real challenge? Play either of the two Shogun games.

Bookmarks