Igno I told you IC that Venice is much stronger and that attacking them would prove disastrous for you. Anyway how did you take that African settlement, do you have siege weapons? The Trebuchet probably did not destroy the wall fully. If even a tiny bit of rubble remains, it still counts as a wall. You have to hear the announcer say that you've breach the enemy's walls for the siege weapon to become irrelevant.
That fort is full of quality troops though, you'd have to have unparalleled skill in leading spear militia to victory over them. The GB unit and the Sicillo-Norman knights alone can probably kill 2000 spear militia.. Such is the way of SS.