You may have lost a border with a faction, which had trade rights with you...