I've had that show up even after canceling military access before declaring war, it's rather annoying since it's already so easy for factions to hate you.