I've been playing for a while now and noticed whenever I build ladders for a siege that they don't appear on the battlefield. They aren't invisible, they are just gone.