I'm not wholly impressed with the 'you need a unit next to a defence for it to work'. It's good in theory, but really how many men do you need to work/protect a tower? If you could split a unit into smaller groups for siege defence, each manning a defensive position, then that would work well. It's silly to think you could have a unit of 120 men at arms only able to use 1 tower, and you have another unit of 6 peasants allowing use of another. In reality, you'd position the men where they could be of most use to get the best from the fortifications you have. It's going to be beneficial to dump all your small weakened units from previous battles into a settlement to provide garrison support just so you get to use as many towers as possible with the minimum of cost. Then again, if RTW is anything to go by, the AI will only attack from 1 or 2 directions so a few towers will be sufficient to man.