Certainly assaulting settlements can be hard, particularly if you do frontal assaults.

Strategically I think they are quite good as it forces you to not be lazy and blast the opposition. I always try a multiple sided attack and try and out flank the defenders.

Climbing ladders into a well defended city will almost always fail unless you have used seige equipment or flanking strategy.

Obviously attacking armies will suffer more than defenders so look on them as more challenging strategically.