IMO those upgrades are just a waste of time, because of tree reasons:

1- I think you lose the upgrade if you upgrade the fortification

2- The time you waste on building those upgrades could be spent building a new fortification

3- How many times have you seen the AI assaulting your castles?

So are this worth it? Does someone have some kind of use for these?