Yeah, that is a major flaw in the game's strategic AI. I agree it's quite frustrating when a faction constantly retreats, especially if it's their last province. For that reason, I generally do my best to keep my armies on a rough parity with that of the enemy, especially when I'm attacking.