One problem is that technical correctness is just one part of the equation. When talking about humans, the statement "[remember] you are dealing with a human here" is technically always correct, but that doesn't mean it is neither relevant nor meaningful to bring up for a topic.
Another problem is that in actual application, 'privilege' tends to be applied on an individual level. If you take a random individual x from a statistically privileged group X (not defined by its privilege, but by some other characteristic), there is no guarantee that the privilege this group statistically has meaningfully applies to x. Yet this exactly how 'privilege' is very often applied: group X is privileged statistically, therefore x is privileged. This is a fallacy.
Conversely, not every member y of a statistically disprivileged group Y is disprivileged, and quite a few y might even become more privileged than the average x if members of Y are given benefits that are meant to counter their disprivilege.
There are, of course, certain groups that are defined by a privilege, such the rich. But even then, as per my first paragraph, this privilege is not always relevant; and it's not hard to see how it can be used dishonestly in rhetoric.
Bookmarks