I think this depends also on siege engines - a lot of the issues your army will face will be related to the walls and the garrison around it, since it will fire upon you. If you manage to defeat that part then you should be good to go because it will give you a numerical advantage in terms of soldiers overall.

I think also you're better off just having 2 armies instead - one with siege engines, one with your regular army.