I think part of the problem is that wearing black masks, waving black flags, _and_ breaking Starbucks window is not a representative activity of anarchists or radicals in general, and everyone (at least in these sophisticated environs) knows this. Therefore, people assume (correctly, I think) that you are deliberately using the few to misrepresent the many as a technique of rhetoric rather than as sober factuality, hyperbole we're all supposed to understand. This leaves your interlocutors with nothing to respond to but mythographic content; they can hardly be expected to respond to that sort of thing by "tak[ing] on the burden of defending it as real politics" since they, including Chuck, had not advanced it as the "real politics" in the first place.
The whole business seems to me to be rather evasive, like your non-answer to my complaints about Gore's moral turpitude in supporting the Drug War.