Hello yoyolll,

The Fort upgrades should be considered temporary at best. You can either a upgrade a Fort's wall defences or you can upgrade directly to a Keep and get all the benefits of a Fort, Motte and Bailey plus the strength of a Keep. The same goes for any later fortifications (Keep, Castle, Citadel, Fortress) and their wall defences/towers.

Personally I never upgrade the wall defences. I almost always upgrade Forts directly to Keeps as soon as possible. The only exceptions are that, in certain provinces, I may build the Fort and Motte or the Keep and Curtain Walls. I never build any of the tower upgrades. The Fort and Motte is cheap and goes up in one year. It's advantage is that it provides a height advantage for your troops in a siege. I would only build this as a temporary solution on the front lines if I think a province is going to be attacked. I build Curtain Walls around Keeps in bleak border provinces that I don't intend to upgrade any further.

Another reason you may want to build the upgrades, as a sort of an exploit, is that if the province is conquered and the Castle is razed down one level by the conquerors, it may be that only an upgrade will be "knocked off" as it were. This is better than having a Keep brought back down to a Fort.