Well you know, in reality they might cause the same amount of damage they do in-game, because there wouldn´t just be a single guy in those towers. There mighht be dozens, if not hundreds in the biggest ones, and all those guys firing bows and possibly light artillery and such could cause great damage to the enemy.