He's projecting.
IA referred to what he sees at Legs' disdain for the working class, but because in Legs' mind this is a fault evidenced exclusively by the Right (and Tories) IA must be referring to himself, because IA is a Tory. IA has said he is not a Tory but because he's to the Right of Legs politically in Legs' mind he must be a Tory.
Basically, Legs is being closed minded and prejudiced.
Bookmarks