[WS:] It succeeded because the "old left" abandoned its traditional social issues for neoliberalism. One of the latest issues of The Nation has a good analysis of the breakdown of the USSR. One of the themes that emerges from it is that the party nomenklatura embraced neoliberalism to justify "privatization" or public property grab from which they immensely benefited. Similar trends existed in other Soviet bloc countries as well. It seems, though that the second and third tier apparatchiks that formed the post-Soviet "left" parties took the cue from their bosses even if they did not benefit from privatization. It seems that they tried to appeal to the youngish technocrats and professionals who were absolutely infatuated with the free markets. That, however, gave them little political gains but cost them the loss of the working class base support. Consequently, big segments of the working class became attracted to right wing nationalists (or theocrats in the developing countries) who, oddly enough, married the support for the welfare state with anti-immigrant, anti-foreigner bigotry. This, I presume, is why folks like Angelus Novus - or for that matter my sister who is a professor in Poland - are irked each time welfare state is being mentioned. It is now part of the nationalist right agenda.
In this context, it makes sense to revisit the thesis advanced some thirty years ago by two Hungarian sociologists Konrad and Szelenyi, arguing that the "intelligentsia" as they are called in Eastern Europe, or "technostructure" as Galbraith (following Veblen) called them here is a new social class that is separate from both the working class and the traditional bourgeoisie. It appears that the rise of this technocrat class may explain the ideological shift that occurred in the second half of the 20th century from socialism and social democracy to neoliberalism. As a nascent force in the 1920s and 1930s, technocrats had elective affinity with ideological movements that recognized their status as unique contributors to the production process - as neither the traditional proles nor the traditional bourgeoisie, but rather as "captains of industry" or skilled technicians steering the economy toward greater rationality and productivity. For that reason, they embraced Keyenesiasim in the US (as Galbraith argues), social democracy in Western Europe and socialism in Eastern Europe. However, by the 1960s the elective affinity between the technostructure and social democracy/socialism started to disintegrate because of the universal benefits that social democracy or socialism offered. Captains of industry ceased to be captains - they become functionaries, cogs in the bureaucratic machinery, in a word, a little bit better paid proles.
The attractiveness of neoliberalism to the technostructure is on the one hand grounded in this inability of social democracy/socialism to continue granting the social status and recognition that the technocrats craved. On the other hand, neoliberalism promised them a way out - first by the means of public property grab aka privatization for which technocrats were uniquely situated as managers of public property they eyed to grab, and second by the means of legitimating that property grab by the ideology of entrepreneurship and meritocracy. The technocrats did not just steal public property like common thieves or profiteered by sponging off the public treasury like common welfare queens - no, they were the valiant captains of industry again who, like Ayn Rand "heroes", bringing entrepreneurship and creativity to economy and society sagging under the yoke of mediocrity, "government regulations" and "socialism."
If the above is correct, the technocrats (by which I mean engineers, lawyers, doctors, economists, experts, business managers and kindred professionals who manage the economy) - who form a separate social class in every respect - are unlikely to embrace the old ideologies of socialism or social democracy. Neoliberalism elevated them to the position of status and respectability again, and they are unlikely to abandon it, especially when many of them are downwardly mobile as as result of their own neoliberal medicine they prescribed to others. They still want to think of themselves as captains, even though they run the ship they were steering aground. Without their support, the only "organic intellectuals" the "old left" ideologies have are basically academics in marginalized departments of sociology or literary studies, while the laboring classes dumped them for nationalism and theocracy.
Therefore, I do not think that comeback of the "old" left is likely. Neoliberalism of the technocrats and the nationalism or theocracy that currently attracts the working class can only be overcome by a new ideology that accomplishes what the "old" left in the 1920s and 1930s - it it will offer the promise of a better economic organization while elevating the social status of the class that steers that organization. Or, to comment on the trope that "Somebody" on this list once advanced - it is a combination of superior productivity or perhaps management of the economy and an ideological legitimation of both the new economic order and the social class champions it. I used the word "management" because I think that what is urgently needed is not producing more shit but using the existing resources in a less wasteful, and more sustainable and equitable way. In other words, a management technique that optimizes the quality of life instead of cranking out more gadgets and stuff. My bet would be on some form of environmentalism/earth stewardship ideology that combines advanced technologies with the rational management of ecosystems and with the attainment of good life and collective security by all humans and non-humans alike.
Wojtek