Actually it does have an effect on your relationship with your allies. If you check the relationship, it'll say that you're allied with their enemy, which they don't like. It usually doesn't have a great influence on your relationship, but I think it's a bit annoying.