Removing bribery? Delete all the diplomats!
No, seriously - Maybe boosting public security values of buildings? Not sure wether public security helps prevent bribery.