If you gift a territory to somone, and then take it again, you take a reputation hit, because the game considers it breaking a diplomatic agreement, and therefore dishonest.