Walls are repaired when converted. When you capture a castle that you plan to convert to a town, you can leave the walls unrepaired. I am not sure however of other buildings (i.e. ballista makers, and armouries <-- sp?)