I'm not sure that it is town size. It might have some modifier since I don't see that a unit of barbarian peasants has helped me more than lets say a unit of axemen.