Law is a big factor when there isn't a governor there. If you do have a governor with traits that decrease law, you'll get more corruption.

Other than that, you could move the capital closer to the center of your empire as distance from the capital increases corruption.