Great wealth was still needed to construct things. First there's the materials, which may well have been the most important thing. Slaves themselves were hardly free, needing to be purchased, fed and equipped with tools. And construction incidentally was one of the few areas where free labour was used far more than slaves, because of its seasonal nature. It was not good value for money to buy a whole bunch of slaves that then only can work in summer. Free poor men were cheaper, you could pay them and then send them on their way to fend for themselves.

And that Cicero quote about the nerves of war being infinite money wasn't there for nothing either.