Yet this exactly how 'privilege' is very often applied: group X is privileged statistically, therefore x is privileged. This is a fallacy.
No. There is an assumption that you're missing, that you would have to confront in order to challenge the point.

The assumption is that group identity is judged towards an individual by other individuals, not by the individual itself.

If this assumption holds, then it is tautological to say that "group X is privileged statistically, therefore x is privileged".