Ladders only appear on regular stone walls (Level 3 walls). They do not work on anything higher than that, presumably because the walls are too tall for ladders.