Well, the swords look better, they get a nicer overall shape and some golden lines.
Hammers also change and look more brutal for two-handed hammers. VHI will get hammers that look like a hand with a sharp bolt in it.
I'm not sure about maces, have hardly seen any without upgrade.
Axes I have never really checked either, my Varangian Guard has nice, big axes but I haven't seen them without upgrade yet.
Lances of my byzantine cavalry(except Latinkon or how the westerners are called in english) didn't seem to change in any way, found that quite weird, I just couldn't find a difference.

And armour upgrades should be quite obvious.

I really love the work they put into all this upgrade stuff, sometimes I build the armourers later than necessary just to see how the armour of the units evolves.