I think it was tasteless thing to say as man with such an influence.
But in general, I think its only racist if the ones being offended took it seriously.