Soften them up before you enter the settlement with your archers - auxilia are excellent marksmen. Then drive in with your cohorts and cavalry.

I agree it is tempting to deal with them on the field, but believe me, a bit of practice with Seige tactics, and you'll soon be good at both! If they don't have spears or strong infantry, then shock them with Cav once you break down the gate, then bring in the infantry.