Personally, i feel that I am constantly pressured to speed up and capture provinces due to squalor. It seems that any time a wall or castle upgrade is possible, I am forced to upgrade, even if i can't afford it, or suffer repercussions later on.
Sometimes, I don't want every single one of my cities to be super huge, yet have no worthwhile buildings. Squalor has been nothing but a pain ever since it was implemented.