Why does sending your King to take the abandones/unconquered place hurt your king/you?

I would think that the easy victory and newly gained glory would advance your cause.