Not yet, but the longer you wait the more of your lands they take, the less income you get, and the less troops you can recruit to fight back against them. Kind of like a vicious cycle...

You better hope the network of allies you've undoubtedly strung together shows up in time!