[WS:] I do not think that this kind of functionalist/idealist view is limited to the left. Neoliberalism is even a better example. You can, of course, find this in the history of philosophy as well.
As I see it, this kind of analysis is tied to a particular metacognitive style which I call core-centric. The idea is that our conceptualizations are spatial, or at least spatial-like, i.e. analogous to spatial fields. A field is defined by its centre or core and by its boundaries, and some people emphasize the core while ignoring boundaries, while others emphasize boundaries while ignoring the core. I explain this here http://wsokol.blogspot.com/2013/05/why-i-find-many-radical-lefties-mildly.html
Once consequence of the core-centric metagognitive style is a penchant for philosophical monism - everything is one, everytyhing can be derived from a single principle or a set of principles. A good historical example of this style of thinking is Plotinus http://en.wikipedia.org/wiki/Plotinus.
Functionalism is a variant of this kind of thinking. It is a logical conclusion of the view that everything 'emanates" from the one core principle. It follows that everything is a derivative of One and thus must have some functional relationship to it.
I have been mildly fascinated by this kind of thinking because of its aesthetic simplicity but never really found it very useful to explaining how things work in the real life. My preference is a kind of "middle range empiricism" cum institutionalism. From that pov the world is a fairly random combination of a multitude of conflicting interests and institutionalization is a temporary outcome of those conflicts. That is, you have different interest groups trying to get ahead of each other and find a niche for themselves where they can be shielded from competition with other contenders. If they succeed in finding such a niche, institutionalization results - which means an arrangement that serves the interests of a whole network of related groups which can persist for quite a while by drawing support form all those who benefit form it in one form or another, even if it is detrimental to other groups or a country as a whole. Once institutionalized, however, that arrangement can last for a long time, but of course it is not immune from competition and may collapse with a sea change in the political or economic environment.
The war on drugs is a good example. At some historical point it was sparked by economic concerns of the resident population resulting from immigration. The resident population saw it in their interest to limit the opportunities of newcomers, and one way of doing it was through the criminalization of their habits. Thus, for example, opium smoking was a habit of Chinese workers brought to the US to build railroads. This led to the criminalization of opium smoking, but not opium ingesting (i.e. by drinking opium tincture) which in the 19th century was quite popular among white Americans. In the same vein, the criminalization of marijuana and cocaine was a move to limit the opportunities of Hispanic immigrants who habitually used these substances. Since the 1980s, smoking crack cocaine gets harsher sentences than snorting powder cocaine, because the former is preferred by poor Blacks while the latter by white stock market traders.
Whatever the initial rationale of the war on a particular substance was, however, the war on drugs generated benefits and career opportunities for a whole range of groups - from law enforcement, to testing industry, to prison complex, to an army of experts and moralizers, and to the peddlers of alternative mind altering substances (e.g. alcohol or pharmaceuticals) .
The war on drugs might have lost its initial purpose and even become detrimental to the society as a whole - but this broad array of interests that benefit form it in one way or another use their political clout to keep it going. If anything, it is dysfunctional rather than functional for the "system" as a whole, but it does not matter as long as the parasites that benefit from it have enough clout to keep it going. You can think of it as a cancerous growth on a social organism.
>From than pov, the sheer number of drug users in the US that you mention is
a great success, not a failure of the war on drugs. It is a success
because it gives the drug warriors more opportunities to sponge off public
resources for their parasitic pursuits.
If I were to make any predictions about its future, which is always a very risky business, I'd say that its biggest enemy is a war on terror because it directly competes with the war on drugs for the same resources.
Consequently, expanding war on terror will likely result in shifting material, human and organizational resources away from the war on drugs.
And as this continues, the political clout of drug warriors will be declining. We can already see the beginning of it, as laws legalizing marijuana use are being passed in various states. This was unthinkable before the war on terror.
-- Wojtek
"An anarchist is a neoliberal without money."