Culuture penalty is only dependent from type of buildings you have (are their yours or of some other culture), so "colonizing" won't help it any way.