[lbo-talk] more critiques Re: (Fwd) Nobel Laureates in Poverty Economics: Abhijit Banerjee, Esther Duflo and Michael Kremer

Patrick Bond pbond at mail.ngo.za
Sat Oct 19 08:00:27 PDT 2019

Previous message: [lbo-talk] Economists project Trump will win easily in 2020 — and by a bigger margin
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]
Search LBO-Talk Archives

Limit search to: Subject & Body Subject Author
Sort by: Reverse Sort

(Sanjay Reddy nails this genre, in a review essay of /Poor Economics, /below: "/the style of metropolitan development economics celebrated in this book leads not so much to increasing rigour as to rigor mortis, by severely limiting the questions that can be asked and shoring up a practical philosophy that is quiescent in relation to many important questions that cannot readily be analysed using the authors’ favoured method. These include questions related to the structure and dynamics of markets, governmental institutions, macroeconomic policies, the workings of social classes, castes, and networks, and so forth./")

https://www.opendemocracy.net/en/oureconomy/impoverished-economics-unpacking-economics-nobel-prize/

Impoverished economics? Unpacking the economics Nobel Prize

When the world is facing large systemic crises, why is the economics profession celebrating small technical fixes?

Ingrid Harvold Kvangraven <https://www.opendemocracy.net/en/author/ingrid-harvold-kvangraven/> 18 October 2019

This week it was announced that Abhijit Banerjee, Esther Duflo and Michael Kremer won the Economics Nobel Prize (or more accurately: the ‘Sveriges Riksbank Prize in Economic Sciences in Memory of Alfred Nobel’). The trio of economists were awarded the prize for "their experimental approach to alleviating global poverty”.

On social media and in mainstream newspapers, there was an exceptional level of praise for the laureates, reflecting their existing rockstar status within development economics. The Financial Times <https://www.ft.com/content/ec6ace8c-ee73-11e9-bfa4-b25f11f42901> even claimed that the Nobel “will help restore [the] profession’s relevance”. However, the widespread calls for celebration need to be considered with a cautionary counterweight.

The experimental approach to poverty alleviation relies on so-called Randomized Control Trials (RCTs). Inspired by studies in medicine, the approach targets specific interventions to a randomly selected group (schools, classes, mothers, etc), and then compares how specific outcomes change in the recipient group versus those who did not receive the treatment. As the groups are assumed to be otherwise similar, the difference in outcomes can be causally attributed to the intervention.

While the laureates were first pioneering this work in the 1990s in Kenyan schools <https://www.aeaweb.org/articles?id=10.1257/000282803321946886>, the approach is now widely considered the new “gold standard” in development economics, also sometimes simply called “New Economics” <https://en.wikipedia.org/wiki/More_Than_Good_Intentions>. The approach has become enormously influential among governments, international agencies and NGOs. The body of work pioneered by the laureates, or the /randomistas/ as they are sometimes called, is meant to alleviate poverty through simple interventions such as combating teacher absenteeism <https://www.povertyactionlab.org/evaluation/encouraging-teacher-attendance-through-monitoring-cameras-rural-udaipur-india>, through cash transfers <https://www.povertyactionlab.org/evaluation/cash-transfers-education-morocco>, and through stimulating positive thinking <https://developingeconomics.org/2018/11/09/why-positive-thinking-wont-get-you-out-of-poverty/> among the people living in poverty. Sound good so far?

While the laureates’ approach to poverty research and policy may seem harmless, if not laudable, there are many reasons for concern. Both heterodox and mainstream economists as well as other social scientists have long provided thorough critique of the turn towards RCTs in economics, on philosophical, epistemological, political and methodological grounds. The concerns with the approach can be roughly grouped into questions of focus, theory, and methodology.

*Focus: tackling symptoms and thinking small*

The approach that is being promoted is concerned with poverty, not development, and is thus a part of the larger trend in development economics <http://hajoonchang.net/wp-content/uploads/2012/01/HamletwithoutthePrinceofDenmark-revised.pdf> that is moving away <https://developingeconomics.org/2017/01/29/what-can-we-learn-from-alternative-theories-of-economic-development/> from development as structural transformation to development as poverty alleviation. This movement towards “thinking small <https://www.hup.harvard.edu/catalog.php?isbn=9780674289949>” is a part of a broader trend, which has squeezed out questions related to global economic institutions, trade, agricultural, industrial and fiscal policy, and the role of political dynamics, in favor of the best ways to make smaller technical interventions.

The interventions considered by the Nobel laureates tend to be removed from analyses of power and wider social change. In fact, the Nobel committee <https://www.nobelprize.org/prizes/economic-sciences/2019/press-release> specifically gave it to Banerjee, Duflo and Kremer for addressing “smaller, more manageable questions,” rather than big ideas. While such small interventions might generate positive results at the micro-level, they do little to challenge the systems that produce the problems.

For example, rather than challenging the cuts to the school systems that are forced by austerity, the focus of the randomistas directs our attention to absenteeism of teachers, the effects of school meals and the number of teachers in the classroom on learning. Meanwhile, their lack of challenge to the existing economic order is perhaps also precisely one of the secrets to media and donor appeal, and ultimately also their success <https://onlinelibrary.wiley.com/doi/full/10.1111/dech.12378>.

The lack of engagement with the conditions that create poverty has led many critics to question <https://www.jstor.org/stable/23269972?seq=1#page_scan_tab_contents> to what extent RCTs will actually be able to significantly reduce global poverty. A further consequence of this impoverished economics is that it limits the types of questions we can ask, and it leads us “to imagine too few ways to change the world” <http://www.ras.org.in/randomise_this_on_poor_economics#fn2>.

*Theory: methodological individualism lives on*

In a 2017 speech <https://www.povertyactionlab.org/news/video-esther-duflo-economist-plumber>, Duflo famously likened economists to plumbers. In her view the role of an economist is to solve real world problems in specific situations. This is a dangerous assertion, as it suggests that the “plumbing” the randomistas are doing is purely technical, and not guided by theory or values. However, the randomistas’ approach to economics is not objective, value-neutral, nor pragmatic <https://www.bloomberg.com/opinion/articles/2019-10-14/economics-nobel-for-banerjee-duflo-kremer-shows-key-change>, but rather, rooted in a particular theoretical framework and world view – neoclassical microeconomic theory and methodological individualism.

The experiments’ grounding has implications for how experiments are designed and the underlying assumptions about individual and collective behavior that are made. Perhaps the most obvious example of this is that the laureates often argue <https://www.aeaweb.org/articles?id=10.1257/aer.101.6.2350> that specific aspects of poverty can be solved by correcting cognitive biases. Unsurprisingly, there is much overlap between the work of randomistas and the mainstream behavioral economists, including a focus on nudges that may facilitate better choices on the part of people living in poverty.

Another example is Duflo’s analysis of women empowerment <https://www.aeaweb.org/articles?id=10.1257/jel.50.4.1051>. Naila Kabeer argues <http://feministeconomicsposts.iaffe.org/2013/12/19/esther-duflo-on-womens-empowerment-and-economic-development-a-must-read-for-feminist-economists/> that it employs an understanding of human behavior “uncritically informed by neoclassical microeconomic theory.” Since all behavior can allegedly be explained as manifestations of individual maximizing behavior, alternative explanations are dispensed with. Because of this, Duflo fails to understand a series of other important factors related to women’s empowerment, such as the role of sustained struggle by women’s organizations for rights or the need to address unfair distribution of unpaid work that limits women’s ability to participate in the community.

Note that there is nothing embedded in RCTs that forces randomistas to assume individuals are rational optimizing agents. These assumptions come from the economics tradition. This is therefore not a critique of RCTs /per se/, but of the way RCTs are employed in the laureates’ work and in most of mainstream economics.

*Method: If you didn’t randomize it, is it really knowledge?*

While understanding causal processes is important in development economics, as in other social science disciplines, RCTs do so in a very limited way. The causal model underlying RCTs focuses on causal effects rather than causal mechanisms <https://onlinelibrary.wiley.com/doi/pdf/10.1111/dech.12140>. Not only do RCTs not tell us exactly what mechanisms are involved when something works, they also do not tell us whether the policy in question can be reliably implemented elsewhere. In order to make such a judgement, a broader assessment <https://www.jstor.org/stable/40962385?seq=1#page_scan_tab_contents> of economic and social realities is unavoidable <http://dro.dur.ac.uk/16430/>.

Assuming that interventions are valid across geographies and scale suggests that micro results are independent of their macroeconomic environment <http://www.ras.org.in/randomise_this_on_poor_economics#fn2>. However, while “effects” on individuals and households are not separate from the societies in which they exist, randomistas give little acknowledgement <https://www.tandfonline.com/doi/full/10.1080/03066150.2014.912432> to other ways of knowing about the world that might help us better understand individual motivations and socio-economic situations. As it is difficult to achieve truly random sampling in human communities, it is perhaps not surprising that when RCTs are replicated, they may come to substantially different results than the original <https://www.iree.eu/publications/publications-in-iree/estimating-microcredit-impact-with-low-take-up-contamination-and-inconsistent-data-a-replication-study-of-crepon-devoto-duflo-and-pariente-american-economic-journal-applied-economics-2015/>.

Not only do RCTs rarely have external validity, but the specific circumstances needed to understand the extent to which the experiments /may/ have external validity are usually inadequately reported <https://academic.oup.com/wbro/article/33/1/34/4951685>. This has led even critics within the mainstream <https://www.nber.org/papers/w22595> to argue that there are misunderstandings about what RCTs are capable of accomplishing. A deeper epistemological critique <https://www.oxfordhandbooks.com/view/10.1093/oxfordhb/9780195392753.001.0001/oxfordhb-9780195392753-e-13#oxfordhb-9780195392753-div1-78> involves the problematic underlying assumption that there is one specific true impact that can be uncovered through experiments.

Recent research <https://tandfonline.com/doi/full/10.1080/19452829.2018.1536696> has found that alternative attempts to assess the success of programs transferring assets to women in extreme poverty in West Bengal and Sindh have been far superior to RCTs, which provide very limited explanations for the patterns of outcomes observed. The research concludes that it is unlikely that RCTs will be able to acknowledge the central role of human agency in project success if they confine themselves to quantitative methods alone.

There are also serious ethical problems at stake. Among these are issues such as <https://ore.exeter.ac.uk/repository/bitstream/handle/10871/17048/EthicsRCT.pdf?sequence=1&isAllowed=y> lying, instrumentalizing people, the role of consent, accountability, and foreign intervention, in addition to the choice of who gets treatment. While ethical concerns regarding potential harm to groups is discussed extensively in the medical literature, it receives less attention in economics, despite the many ethically dubious experimental studies (e.g. allowing bribes for people to get their drivers’ licences in India <https://timesofindia.indiatimes.com/blogs/Swaminomics/heavy-fines-a-way-but-crush-apathy-for-road-safety/> or incentivizing Hong Kong university students <http://conference.nber.org/conf_papers/f126621.pdf> into participation in an antiauthoritarian protest). Finally, the colonial dimensions of US-based researchers intervening to estimate what is best for people in the Global South cannot be ignored.

*Why it matters: limits to knowledge and policy-making*

There will always be research that is more or less relevant for development, so why does it matter what the randomistas do? Well, as the Nobel Committee stated, their “experimental research methods now /entirely dominate/ development economics”. A serious epistemological problem arises when the definition of what rigour and evidence means gets narrowed down to one single approach that has so many limitations. This shift has taken place over the past couple of decades in development economics, and is now strengthened by the 2019 Nobel Prize. As both Banerjee and Duflo acknowledged in interviews after the prize was announced, this is not just a prize for them, but a prize for the entire movement.

The discipline has not always been this way. The history of thought on development economics is rich with debates about how capital accumulation differs across space, the role of institutions in shaping behavior and economic development, the legacies of colonialism and imperialism, unequal exchange, the global governance of technology, the role of fiscal policy, and the relationship between agriculture and industry. The larger questions have since been pushed out of the discipline, in favor of debates about smaller interventions.

The rise of the randomistas also matters because the randomistas are committed to provoke results, not just provide an understanding of the situations in which people living in poverty find themselves. In fact, it is one of their stated goals <https://www.aeaweb.org/articles?id=10.1257/jep.24.3.61> to produce a “better integration between theory and empirical practice”. A key argument by the randomistas <https://economics.mit.edu/files/2785> is that “all too often development policy is based on fads, and randomised evaluations could allow it to be based on evidence”.

However, the narrowness of the randomized trials is impractical for most forms of policies. While RCTs tend to test at most a couple of variations of a policy, in the real world of development, interventions are overlapping and synergistic. This reality recently led 15 leading economists to call to <https://www.theguardian.com/global-development/2018/jul/16/buzzwords-crazes-broken-aid-system-poverty> “evaluate whole public policies” rather than assess “short-term impacts of micro-projects,” given that what is needed is systems-level thinking to tackle the scale of overlapping crises. Furthermore, the value of experimentation in policy-making, rather than promoting pre-prescribed policies, should not be neglected <https://www.cgdev.org/publication/escaping-capability-traps-through-problem-driven-iterative-adaptation-pdia-working-paper>.

The concept of “evidence-based policy” associated with the randomistas needs some unpacking. It is important to note that policies are informed by reflections on values and objectives, which economists are not necessarily well-suited to intervene in <https://www.ideasforindia.in/topics/miscellany/evidence-policy-and-politics.html>. Of course, evidence should be a part of a policy-making process, but the pursuit of ineffective policies is often driven by political priorities rather than lack of evidence.

While randomistas might respond to this by arguing that their trials are precisely meant to de-politicize public policy, this is not necessarily a desirable step <http://eprints.lse.ac.uk/68604/1/Parkhurst_The%20Politics%20of%20Evidence.pdf>. Policy decisions are political in nature, and shielding these value judgements from public scrutiny and debate does little to strengthen democratic decision making. Suggesting that policy-making can be depoliticized is dangerous and it belittles the agency and participation of people in policy-making. After all, why should a policy that has been proven effective through an RCT carry more weight than, for example, policies driven by people’s demands and political and social mobilisation?

While the Nobel Prize does leave those of us concerned with broader political economy challenges in the world anxious, not everything is doom and gloom. Firstly, the Nobel directs attention to the persistence of poverty in the world and the need to do something about it. What we as critical development economists now need to do is to challenge the fact that the Prize also legitimizes a prescriptive view of how to find solutions to global problems.

Secondly, the fact that a woman and a person of color were awarded a prize that is usually reserved for white men is a step forward for a more open and inclusive field. Duflo herself recognizes that the gender imbalance among Nobel Prize winners reflects a "structural" problem in the economics profession and that her profession lacks ethnic diversity.

However, it is obvious that to challenge racism, sexism and Eurocentrism in economics, it is not enough to simply be more inclusive of women and people of color that are firmly placed at the top of the narrow, Eurocentric mainstream. To truly achieve a more open and democratic science it is necessary to push for a field that is welcoming of a plurality of viewpoints, methodologies, theoretical frameworks, forms of knowledge, and perspectives.

This is a massive challenge, but the systemic, global crises we face require broad, interdisciplinary engagement in debates about possible solutions.

/Thanks to Carolina Alves, Devika Dutt, Minna Lehtinen and Farwa Sial for helpful discussions on these issues and comments on a previous draft./

***

*Randomise This! *

Sanjay G. Reddy*

Few volumes in contemporary economics have been more lauded, and have summarised a zeitgeist, as much as Abhijit Banerjee and Esther Duflo’s /Poor Economics/.1 The book has received prominent international prizes (/The Financial Times /and Goldman Sachs Business Book of the Year, for example), and been widely read, reviewed, and praised, including by leading economists and philanthropists.2 It is a capstone of the authors’ work, representing their effort to project to a broad readership the perspective on development that they have communicated in influential academic papers and through their professional activities – which have also been lauded through extensive media attention and the award of prestigious prizes in the United States, France, India, and elsewhere.

It would be no exaggeration to say that the book captures an approach to development economics that has swept the field and, for a considerable period, has placed other approaches on the defensive. The approach, moreover, has been enormously influential within governments, international agencies, and nongovernmental organisations. The reasons for its success owe a great deal to the statistical arguments that have been made on behalf of the method of randomised trials that the authors have championed as a means of identifying “what works” in development. Despite their claim to have implemented a more scientific method for determining “what works,” the book gives much less attention to statistical arguments than to presenting a narrative of the authors’ own observations concerning the nature of poor people’s lives as well as their judgments, informed in some cases by randomised trials about which “interventions” can best “work” to improve those lives.

The authors insist that there exist solutions to the problems faced by poor people, and that these typically involve small modifications in the design of existing

1 Banerjee, Abhijit, and Duflo, Esther, /Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty/, Public Affairs, New York, 2011.

2 http://www.thegatesnotes.com/books/development/poor-Economics

* Department of Economics, The New School for Social Research, New York, reddys1 at newschool.edu.

| 61 efforts as well as the eventual generalisation (or “scaling up”) of
the resulting “interventions.” The focus of the narrative is upon individuals (or households) as decision-making actors whose choices in the midst of poverty can be strategically shaped so as to better enable them to exit poverty. While the authors accept that there can be poverty traps in which even the best responses of agents to their circumstances do not suffice to enable them to escape poverty, and may indeed reinforce those circumstances (extensively working a formal model familiar to economists involving an “S-shaped” return to effort, in which there may be a critical threshold before efforts are sufficient to justify the costs of undertaking them), they also insist that a slow climb out of poverty is sometimes possible even without additional resources, if individuals are helped to perceive and choose better options. In this they are in sympathy with behavioural economists who argue that the occasional “nudge” facilitates better choices on the part of individuals.

The resulting representation of the poor inevitably highlights their failings (for instance, the tendency to spend rather than save, or to spend on the “wrong” things) even as it emphasises their everyday ingenuity and heroism. The authors suggest, to paraphrase a famous line, that the poor are different from you and me – they have less money. Like us, they make the best of their circumstances, although, in part as a result of those circumstances, they can make mistakes and suffer from insufficient motivation.

The authors propose that although we do not know “what works,” careful observation of the poor to help design interventions, cemented by randomised trials to assess these interventions, can help us identify what does. Those who have the power to intervene (governments, international organisations, NGOs, philanthropists, and the global middle and upper classes) are assumed to be well motivated, so that once the deficit in their knowledge is overcome (in part through the good offices of the authors), they will act. This, in sum (to quote the sub-title of the book), is the authors’ “radical rethinking of the way to fight global poverty.”

The authors claim to offer simple solutions for seemingly difficult and even intractable problems associated with poverty, solutions that moreover can be “scaled up” (i.e. applied much more broadly) once they are found to work. Very often the prescriptions are presented as being non-obvious or even counterintuitive, thus justifying the authors’ claim that randomised studies of the kind they champion play a vital role in identifying and validating their favoured interventions.

The focus on the small but notionally critical intervention is supported by the claim to have made a decisive advance in solving the problem of “identification,” which relates to the difficulty of assessing the “true” effect of a particular causal variable upon an outcome of interest. The massive investment in costly randomised trials designed to zero in upon such an effect is driven by the argument that they provide the best means of arriving at an inference that is not polluted by the impact of “confounding” variables. (We shall discuss below whether this is in fact the case.) 62 |In turn, the obsession in relation to identification is shaped by the internal demands of the “profession” of economists as experienced in the metropolis. There has been an increase in the premium on identification, in part because, over the last thirty years, cheap computing power has enabled the proliferation of data and of econometric analyses that present often contradictory conclusions. Although the price of supposed success in identification is the focus on narrow questions, the authors would eat their cake and have it too, claiming that their findings provide an Archimedean point through which one can change the world – that the little changes their method may recommend can cumulate to form a “quiet revolution.”

How should one assess the claims of the book?

The implicit premise of the book is that interventions that work in one place can be expected to work in another. This presumes not only that the results of such “micro” interventions are substantially independent of the “macro” context, but also that a focus on such interventions, as opposed to those which reshape that context, is sufficient to address poverty. These premises of “separability” and “sufficiency”, although non-trivial, go largely undiscussed by the authors. The causal relations at work in relation to individuals or households cannot be understood in atomic isolation. Rather, the effects of individuals’ choices are shaped by the “macro” settings in which they find themselves; their individual choices can cumulate in unexpected ways, and the currents that buffet individuals and determine their fortunes can at times overpower the effects of their individual choices.3 There are reasons why it is not possible to discuss individual fates without taking note of the macro economy, history, culture, and politics. Alas, such concepts appear to play little role in the world-view of the randomisers.

Not surprisingly, one consequence of the approach to development economics championed by the authors is that the questions asked by the discipline have become much smaller. The authors’ position appears to be that this is quite all right, since the small questions are in fact large in importance. It is not easy to accept this, however. The larger questions once asked within the discipline, regarding the effect of alternative economic institutions and policies (such as those concerning property arrangements, trade, agricultural, industrial and fiscal policy, and the role of social protection mechanisms), for instance, and the impact of political dynamics and processes of social change, have been pushed to the background in favour of such questions as whether bed-nets dipped in insecticide should be distributed free of charge or not, or whether two schoolteachers in the classroom are much better than one. Even as the authors claim to be reporting on vital and potentially transformative

3 The entitlement framework of Amartya Sen (1981) provides a prominent example of a mainstream analysis in which these multiple levels of causation have a role to play. As Karl Marx famously noted in the /18th Brumaire of Louis Bonaparte/, “Men make their own history, but they do not make it just as they please.” Not only are human beings’ circumstances “encountered and inherited from the past,” but they may also experience individually as an immovable fact what can be changed only collectively.

| 63 breakthroughs, the result is a pianissimo of development economics
that provokes in the reader who is at all aware of the broader canvas, an anxiety that a great many of the important questions are unasked. This book is reflective of the most prominent current of contemporary metropolitan development economics, and its prescriptions are both exemplar and example of the resulting limitations.

One may argue, in fact, that the style of metropolitan development economics celebrated in this book leads not so much to increasing rigour as to rigor mortis, by severely limiting the questions that can be asked and shoring up a practical philosophy that is quiescent in relation to many important questions that cannot readily be analysed using the authors’ favoured method. These include questions related to the structure and dynamics of markets, governmental institutions, macroeconomic policies, the workings of social classes, castes, and networks, and so forth. Although such questions can only be approached through other methods, they are not the less important for that.

It has been widely recognised that the only questions that are amenable to being answered through randomised trials are very narrow ones, having to do with the responses of individuals or households to a well-defined “treatment.” All the questions that arise in the macroeconomic context (regarding national policies, for example), and many, if not most, of the important questions that arise in a meso or microeconomic context, cannot possibly be answered in such a way.4 However, the authors attempt to turn their weakness into an advantage by arguing that they are at least providing “evidence-based” answers where possible, thus creating a scientific basis for policy-making that would otherwise be unavailable and promising, moreover, to change the world by doing so.

It may be argued that whatever the potential contribution of such micro-interventions, the very ability to ask the “small” questions that are amenable to randomised trials derives from the prior activity of those who ask the larger questions, and, in particular, of those who seek to solve problems, exploring possible innovations along the way.

The low-dimensional nature of a question that may be answered by employing a randomised trial can be contrasted to the high-dimensional nature of economic and social reality. A randomised trial estimates an impact coefficient that relates a single policy “lever” to a measurable outcome. In contrast, those who face a complex reality must contend with it without the conceits of “rigour” and “precision.” We may think, for example, of the many different reasons for the failure of educational systems in both developing and developed countries, and the various ways in which one might address that failure. Placing two schoolteachers in the classroom rather than one, monitoring teachers’ attendance using cameras, or providing incentive pay for school performance are only some of the ideas that could occur to someone concerned with such reforms. Why focus on these ideas rather than others? The ideas 4 On this see, for example, Rodrik (2008).

64 |worth testing ultimately come from real attempts to improve schools, which in turn emerge from the practical experience and insight of those involved with them.

Many possible innovations cannot be interpreted in a sufficiently simple way to make randomisation productive. Consider a hypothetical example, of a randomised trial aiming to throw light on whether introducing parent–teacher associations will aid educational outcomes. Because there are many ways in which such associations can function, and many reasons why they may succeed or fail, a randomised trial can be expected to provide very limited guidance concerning their role. Randomised trials cannot help greatly to illuminate the merits of broader and more complex proposals for institutional design arising in the crucible of real problem-solving.

The authors treat the concept of an intervention as unproblematic: it is a welldefined and simple action that can be taken. In actual practice, measures that make a difference in people’s lives are often complex bundles of actions that change over time, and generate changing effects over time. It is often a /process /of social action and political engagement that reveals how they must evolve if they are to be effective.

One can think here of the evolution over time of complex governmental programmes such as the National Rural Employment Guarantee Scheme (NREGS) in India, through on-the-ground experience and the determined interaction of diverse actors.

Even as narrow and technical a measure as the administration of a vaccine can be undertaken effectively only through such a process, as the past and present history of immunisation campaigns, which encountered evolving obstacles as they proceeded, well demonstrates.5 Doing a randomised trial to find out what “works” can fail to recognise how learning happens over time in a complex, and often politicised, setting. Perhaps most fundamentally, however, the concept of an intervention gains its currency from an /engineering /approach, in which intervenors within a system are viewed as standing outside it and their possible actions are well-defined without reference to how the system acts upon the interventions. Normatively, this view is at odds with a democratic understanding of society, in which shared concerns can only properly be addressed through collective endeavour. Explanatorily, it is at odds with a non-mechanistic understanding of society, in which all actions are defined as well as outcomes shaped by complex and often unpredictable processes of mutual interaction.

As the methods championed by the authors have become not only common but indeed dominant (as reflected in the disproportionate number of Ph.D. students in development economics from leading US institutions who have been engaged in recent years in executing a randomised trial, and the astonishing fact that the “Poverty Lab” at the Massachusetts Institute of Technology alone has been or is presently involved with some 350 randomised trials around the world, each costing 5 The current difficulties in administering the polio vaccine in areas of the world where there is social resistance to accepting it provide an interesting, if disturbing, example.

| 65 a substantial amount of money to conduct) there has been a backlash
among some economists. This response has perhaps had the greatest impact within the discipline when it has come from econometricians deeply concerned with the very problems of identification that the authors and their randomising allies claim to solve.6 A criticism made by many is that randomised trials may lead to results that lack “external” validity, whether or not they possess “internal” validity. This is to say that the estimate of the impact of a policy “lever” upon an outcome of interest (e.g. the impact of having two schoolteachers rather than one on educational performance) may vary from place to place for any number of reasons. The idea that there is /an /impact that can be discovered derives from a mechanistic metaphor which presumes that there is a (sufficiently) universal causal logic at play. This metaphor is false because social and environmental contexts do matter, and profoundly, in giving rise to causal relations. Moreover, to know how and why this is the case is essential to understanding many phenomena, even those that can be represented as varying parametrically in some way (and many cannot). A straightforward example is that the impact of a microfinance intervention may well depend on what other such initiatives are simultaneously present in a community, and to which a borrower or peers may have access, as this will influence the ability of borrowers to borrow sustainably from a single source, the outside options of individual borrowers, and the discipline operating on lenders. For this reason, the lessons from microfinance experience in a place such as Andhra Pradesh in India, where many such lenders have been competing, most of them motivated by profit, are rather different from those that derive from Bangladesh, also saturated by microfinance institutions but ones driven by different objectives. The key to understanding the good and the bad that can come of microfinance is to recognise that these two cases together illuminate the distinct regimes of microfinance that may be possible. A more complex example might involve the effects of a policy that affects farm size, tenancy relations, or access to fertilizer or other inputs. The impact of the policy may depend crucially on aspects of the agro-ecological setting, the cropping and seasonal pattern, the dependence on and conditions of access to common resources such as groundwater, the prevalence and characteristics of hired labour, the terms of availability of credit, and the level and form of the marketisation of outputs (through producer cooperatives or through sale to intermediaries). These circumstances in turn will affect risk, return, and other economic variables. Further, the effects of such a policy may depend on aspects of the socio-political environment, which will influence its effects in a manner that does not lend itself to /ex ante /specification. (We may think of how similar declared policies providing rights to agricultural tenants had very different effects in different parts of India, depending in part on the role of political actors in promoting social awareness and mobilisation, and the effective enforcement of those policies.) Even if one has 6 See, for example, Barrett and Carter (2010), Deaton (2009), and, relatedly but prior to recent controversy, Heckman (1991, 1995).

66 |agreed to emphasise narrowly consequentialistic concerns, the very idea of empirical investigation is trivialised by the idea of finding /an /impact of an intervention.

This is not to say that one must throw up one’s hands. Rather, by examining the political economy of individual cases through deeper contextual and historical investigation, comparing such cases across space and time to understand possible variations, tracing the individual processes that are at play and recognising their commonalities, one can begin to understand how and why policies do or do not work, and go beyond the conclusion that “it is complicated.” Even those who are committed to a more sophisticated mode of analysis must strive to identify policies that do and do not work, and why, without simply reproducing ideological presuppositions. The large number of respects in which cases can vary and the small number of cases available for study, as well as the fact that non-deterministic factors operate in each case, imply that judgment will necessarily be involved in such an exercise. This is not an embarrassment but rather the very condition of confronting reality.

The idea that there is /an /impact of an intervention is questionable in another way too. An intervention may lack in external validity not only because its effects vary from context to context, but also because they vary within a given context as the intervention is “scaled up.” We refer here to the average effects that are the focus of randomised trials and not to differing effects in different segments (e.g. geographical or social groups) of a population, which may also be relevant for decision-making but which such trials in any case cannot easily identify. Such average effects themselves can vary as more of a population is covered by an intervention. The first fraction of a population covered by an intervention, even if randomly selected so as to do away to an extent with selection effects, may exhibit a very different impact of the intervention than later fractions of the same population. One of the reasons that this might occur is that the effects of an intervention may depend on the perception of “specialness” arising from receiving the intervention, or on incentive effects associated with doing relatively better than non-recipients of the intervention.

Another reason is that an environment may become more “congested” as more people receive an intervention. We may think of the returns from microfinancesupported petty production or trade being driven down by many people engaging in the same activity. The success of an intervention can also depend on network effects related to the fraction of the persons tied in some way to the recipient of a treatment who also receive the treatment (we may think, for instance, of the effects of any educational initiative in which either knowledge or the effects of knowledge are transmissible). Still another reason is that the political and social factors that arise against (or in favour of) an intervention, and make it relatively successful or unsuccessful, may depend on how widespread it is. Only an understanding of the substantive nature of the intervention and the causal processes likely to apply in a specific context can allow one to make the judgments necessary to characterise the findings of a randomised trial.

| 67 Other econometric issues (also arising in the biomedical context,
although arguably more muted there) that have been raised by critics of randomisation in economics include the possibility that, in practice, randomisation may not exclude the very selection bias – in which those who randomly receive the treatment differ in some unobserved respects from those who do not – that it is meant to eliminate. This can happen for any number of reasons, such as the possibility that those who do not receive the treatment may shift location or other criteria in order that they do; that those implementing the treatment at the local level may non-randomly select individuals to receive the treatment because of their special concern for specific persons, or to establish programme efficacy or inefficacy; that if the number of sampled units is not large enough, there may still be a correlation between the receipt of the treatment and the characteristics of the units; and that the very knowledge that one is receiving the treatment on a randomised basis may influence one’s response to receiving it, or indeed to not receiving it.

Quite apart from the issues of econometric identification arising in any real context, and in no way secondary to them, are the ethical requirements of conducting a randomised trial. Strikingly, these considerations are not discussed at all in the book.

The elementary Kantian notion that one should treat others as ends and not only as means has obvious implications in regard to randomisation. The case for conducting a randomised trial involves the idea that the considerable resources expended on it may be justified by the value of the knowledge garnered from the trial in application there and elsewhere. This argument typically presupposes the external validity of the trial results and that the benefits of the knowledge garnered justify the direct cost. More subtly but equally importantly, where there are /ex ante /reasons to believe that a particular benefit should go to one person rather than another (for ethical reasons or for reasons of efficacy), to allocate the benefit randomly instead is to deliberately misallocate resources, at the cost of the individuals denied the treatment.

Of course, it is also important that the randomly administered treatment should provide a benefit rather than create a harm, either to those receiving the treatment or to others, if it is not to raise ethical concerns. It is not evident whether this test has been met in a number of instances where randomised trials have been applied in development economics. The case of random provision to applicants for drivers’ licenses of financial inducements to get licenses quickly (presumably by paying bribes) – which forseeably imposed potential harm on unqualified drivers and third parties – is illustrative.7 That there exist institutional failures in the provision of drivers’ licences cannot be a sufficient defence of active involvement in the creation of potential harm. Ethical issues of this kind have been discussed extensively in the biomedical literature on randomisation, and attention to them is a part of the 7 It is beside the point that the study authors (Bertrand /et al/. 2007) find ultimately that actual driving ability matters little in the provision of licences in Delhi. Barrett and Carter (2010), perhaps uniquely, recognised this study as raising ethical questions. It is mentioned as an informative study by Banerjee and Duflo (p. 242) without any such comment. The study was moreover sanctified by publication in one of the discipline’s preeminent journals.

68 |recognised obligation of medical researchers. In contrast, with very rare exceptions, they scarcely find a mention in the economics literature.

Although there is some history of social experiments involving randomised trials in the development of government policy-making in developed countries, it is notable that these have been most often experiments aiming to compare the responses of the poor to alternate policy interventions (see Bhatt 2011 and O’Connor 2002).

The current boom in randomised trials in development economics, in contrast, has almost entirely involved experimentation on poor persons in poor countries. Sadly, there may be reasons for this that are less savoury than the putative philanthropic objectives of the experimenters. It is not implausible to argue that among these reasons is the relative disorganisation of poor people and poor communities, which makes it possible to experiment upon them without facing much resistance. The relative pliability of governmental authorities in providing consent on their behalf, in lieu of the informed consent of individuals, also facilitates such experimentation. To illustrate, in many of the well-known randomised trials that have been undertaken in development economics, higher-level government officials concerned with a programme have agreed to randomisation over geographical units in which groups of persons live, arguing that a programme must in any case be “rolled out” slowly, with only certain areas benefiting at first. It would very likely be politically unpalatable to provide a recognisable benefit randomly to middle-class or upper-class individuals or communities, and, at a minimum, to do so would involve complex political negotiation. It is hardly a surprise that a case of randomisation of benefits among middle- or upper-class recipients is hard to identify.

The implicit philosophy of the authors of the book, and the broader constellation of like-minded economists, is based on the following premises. First, “what works”

is simple, even deceptively simple. Hence, in common with the freakonomists, the randomistas reveal that there are counterintuitively straightforward answers to seemingly difficult problems arising in a complex world.8 Secondly, “what works” in one place will work in another place. This is presumably (as it is not made explicit) because of the existence of “deep structural” causal underpinnings that are uniformly present. It is not surprising that the authors use concepts such as that of the policy “lever”: their epistemic framework is modular, reductive, and mechanical. Thirdly, individuals can pull themselves up by their bootstraps if only they are given boots, i.e. the resources necessary to reach a critical threshold or other critical individuallevel supports. Fourthly, individuals respond reliably to economic incentives in the form of carrots and sticks. However, it is important to understand the specificities of poor persons’ situations in order to see why they respond as they do. This will sometimes involve recognising such phenomena as weakness of the will, behavioural mistakes, and low motivation (which, the authors claim, can be a consequence of poverty because escape from it can seem a distant prospect). This premise is revealed 8 On the former, see Spiegler (2012).

| 69 in many of the authors’ examples and studies. For instance, their
approach to dealing with teacher absenteeism is to place a camera in the classroom to register teacher attendance, and to put in place penalties for being absent so as to create incentives for compliance. They do not aim to change underlying attitudes. The prescription is driven by the presumed motivational structure of the actors as well as by a narrow way of posing the problem.

The central role of intrinsic motivations in shaping attitudes to work and other choices has long been recognised (see Kreps 1997 and, more foundationally, Weber 2003). Some may suggest that the authors of the book have a richer view of human motivation than many economists, who are still narrower. Still, their understanding of human psychology is simplistic. Although the authors draw on their field visits to motivate their informal narratives concerning individual responses to poverty, much of their description of individual motivations involves /a priori /assumptions invoking presumed economic calculations needed to rationalise observed behaviour.

Despite the fact that numerous anecdotes underline the ostensible familiarity of the authors with the poor and their experiences, and offer an implicit admission that ethnographic reportage has something to offer (if only as a supplement to the real work done by the scientific but open-minded randomising economist), the authors do little to validate a systematic role for other ways of knowing about the world (e.g.

the ethnographic), ways that would give a window on the motivations of actors and their social interdependencies. The authors’ attitude to interpersonal and societal differences reflects what is often called “economic imperialism.” They recognise that motivations may have local specificities (for example, because some persons and societies deem it important that weddings must be celebrated grandly and save accordingly), but ultimately assume that we are all reliably to be treated as /homines economici/.

Tellingly, the book makes almost no reference to any economics literature other than the very recent and other than what has emanated from the charmed metropolitan circle (as represented by a small number of “leading” economics journals). Still less does it refer to contributions of the other social sciences, whether recent or historical. Although the authors underline the ostensibly rigorous empiricism of their approach by way of contrast to the unrigorous empiricism of non-randomising econometricians and the non-empirical conceits of once-prestigious economic theorists, they show little interest in the diverse sources of empirical knowledge, actual as well as potential.

The point is not that they have failed to offer symbolic recognition to such alternatives, but that the limitations of their world-view would be exposed if one were to verge beyond the few references in their field of vision. What is the cost of such ignorance?

At a bare minimum, it is to imagine too few ways to change the world. Moreover, as increasingly recognised even by those who accord a role to randomised trials, it is important to understand the causal processes that are involved when something 70 |“works” in order to determine what exactly has been identified to work (e.g. a principle undergirding an intervention as opposed to the specific intervention), and whether and how to implement it more generally. Drawing on an understanding of economic and social reality using other resources is unavoidable (see Barrett and Carter 2010; Cartwright 2010). At a less refined level, does any serious student of development (even one who has not ever heard of a randomised trial) gain much from denials that “microfinance is a cure-all, that schooling equals learning, that poverty at the level of 99 cents a day is just a more extreme version of the experience any of us have when our income falls uncomfortably low”?9 As can be demonstrated in each of these cases, and others, the authors’ casual observations and randomised trials may not deliver much at this level of generality that thoughtful field observers, and indeed the existing literature, have not recognised.

There are discursive considerations that must be introduced in order to understand the inordinate success of the book, as well as the perspective that it offers to academic power brokers and opinion makers, policy entrepreneurs and public officials, and the metropolitan public generally. A “reception-theoretic” analysis of the enormous success of the book, and more generally of the approach it exemplifies, can identify at least three discursive factors – beyond the ostensible econometric benefits of randomised trials in dealing with selection bias – that have played a role in generating an environment conducive to its success.

The first discursive factor underlying the influence of the authors’ perspective is the widespread appeal of an ambient doctrine that we could call “Washington Consensus plus.” This is the idea that although the private property-entrenching, marketoriented reforms of the Washington Consensus, or something akin to them, are necessary for poverty reduction, they may not suffice. Additional steps are needed at the level of individuals and nations – in particular, prior interventions to alleviate poverty traps or other obstacles to taking advantage of market opportunities. This position (forcefully articulated by, for instance, Sachs 2005) criticises the Washington Consensus only for its insufficiency, and calls for a focus on what is necessary to alleviate obstacles to spontaneous and self-sustaining growth and development based on the grasping of market opportunity.

To be clear, the authors do not explicitly state their own position on larger questions, leaving open their views on macroeconomic policies, alternative institutional modes of organisation of an economy, and so forth. Indeed, they often seem to emphasise the micro-interventions which they believe can alleviate poverty, preferring little fixes to large ones, precisely because they are sceptical of coming to conclusions with regard to “what works” more generally.

9 See http://www.publicaffairsbooks.com/publicaffairsbooks-cgi-bin/display?book=9781586487980.

| 71 The book is concerned with the amelioration of individual distress,
but pays little attention to the conditions of sustained and socially inclusive structural transformation of national economies except in so far as it views individual life improvement as bringing such change in train. In this sense, the authors are concerned with poverty but not with development. To the extent that they take a view of contextual and structural concerns, it is in the form of an implicit attitude toward contemporary market capitalism that is conformist rather than critical, and indeed does not dwell long on uncomfortable or difficult questions concerning the institutional setting of the intervention. It is reflective of this that they take a largely celebratory view of microfinance (of course, seeing it as “only one of the possible arrows in the fight against poverty,” and recognising such qualifications as that not everyone is an entrepreneur). They do not appear to make a fundamental distinction between the for-profit microfinance firms that they profile (some of which have been involved in rather substantial controversies concerning loan-pushing in Andhra Pradesh, India) and not-for-profit microfinance organisations. From a perspective that prioritises narrow interventions there may be no crucial distinction here, since the authors’ focus is a common intervention ostensibly being delivered to individuals by both.

Their focus is on finding the elixir of individual success, and not on understanding or reshaping the economic environment within which individuals are asked to succeed.

Those individuals who turn a small amount of seed capital into prosperity through their thrifty habits and entrepreneurial ways are the heroes of the book.

The second discursive factor underpinning the influence of the authors’ perspective is the appeal of the technocratic orientation to development, in which change is conceived of as being brought about by technical intervention from above (a “technofix”) and relies upon the knowledge of experts as to “what works.” Such knowledge, once arrived at, can be applied in a modular form, making it possible to replicate elsewhere because of a mechanical understanding of causal relations.

The presumption that there are near-universal and observable empirical regularities underlying the connection between inputs and outputs corresponds to a narrow engineering approach to causation in social affairs. There is little room to take note of contextually variable social relations, let alone the role of political factors that undermine such a mechanistic image of society. Although the energies of ordinary people may be ultimately involved in bringing about change, the experts act as midwives who recommend expertise-grounded interventions necessary to unleash these energies, and the interventions are presumed to be tested and faithfully implemented by benevolent political actors. The perspective of the authors leaves little room for politics of any kind, not to mention the demands of open-ended democratic self-government.

The third discursive factor is the appeal of the “soundbite” – the simple and typically simplistic description or explanation that travels easily and can thus smoothly gain currency among policy entrepreneurs, power brokers, and well-meaning and interested laypersons. The breezy style of the book, peppered with anecdotal 72 |observations, combines with the authors’ self-presentation as technical experts (who, ironically, favour randomised trials as a method that ostensibly improves upon anecdotes) to provide the legitimation necessary for such soundbites to garner ready acceptance.

It is, perhaps, not too difficult to understand why the prescription of /Poor Economics /has enjoyed as much circulation as it has, in particular among metropolitan development policy-making elites, although increasingly also elsewhere.10 It appeals to powerful but flawed metropolitan predispositions: a desire to “fix” things with simpleminded mono-causal reasoning, allied with the conviction that technology, through the analysis of data using randomised trials, makes it possible to do so. Its technocratic premises, its naïve view of politics and society, and its unselfconscious do-goodism make for a self-affirming picture of the world.11 It is unfortunate that it does so little to explain it.

Keywords: Poor Economics, Randomized controlled trials (RCTs), MIT Poverty Lab, Abhijit Banerjee, Esther Duflo, randomistas, Washington consensus, econometric identification, technofix.

Acknowledgements: I am deeply grateful to V. K. Ramachandran for his sustained encouragement and many invaluable comments; to David Grewal, Sanjay Krishnan, Michael Pollak, Peter Spiegler, and Nidhi Srinivas for their extensive written suggestions; and to Arnab Acharya, Tanweer Akram, Jigar Bhatt, C. P. Chandrasekhar, Stuart Corbridge, Siddhartha Deb, Jayati Ghosh, Radha Gopalan, John Harriss, Atul Kohli, Pratap Mehta, Uday Mehta, S. Nanthikesan, Raj Patel, Thomas Pogge, Bharat Punjabi, Hugh Roberts, Rathin Roy, Anwar Shaikh, Ajit Sinha, Joseph Stiglitz, S. Subramanian, S. Vela Velupillai, Robert Wade, Thomas Wallgren, and other friends and colleagues for their helpful reactions. I do not imply that any of these persons agree in part or in full with my views.

/References /Banerjee, A., and Duflo, E. (2011), /Poor Economics: A Radical Rethinking of the Way to Fight Global Poverty/, Public Affairs, New York.

Barrett, C., and Carter, M. (2010), “The Power and Pitfalls of Experiments in Development Economics: Some Non-Random Reflections,” /Applied Economic Perspectives and Policy/, vol.

32, no. 4.

10 One can think of at least three interpretations of the title, not all favourable to the authors.

11 The authors are at least clear in their hierarchy of value, in which economists figure favourably: “Some of these people almost surely had the potential to be professors of economics or captains of industry. Instead they became daily labourers or shopkeepers” (Banerjee and Duflo 2011, p. 95).

| 73 Bertrand, M., Djankov, S., Hanna, R., and Mullainathan, S. (2007),
“Obtaining a Driver’s Licence in India: An Experimental Approach to Studying Corruption,” /The Quarterly Journal of Economics/, vol. 122, no. 4.

Bhatt, J. (2011), “Causality and the Experimental Turn in Development Economics,” /New School Economic Review/, vol. 5, no. 1.

Cartwright, N. (2010), “What Are Randomised Controlled Trials Good For?” /Philosophical Studies/, 147 (1), pp. 59–70.

Deaton, A. (2009), /Instruments of Development: Randomisation in the Tropics, and the Search for the Elusive Keys to Economic Development/, National Bureau of Economic Research, Cambridge, Mass.

Heckman, J. (1991), /Randomisation and Social Policy Evaluation/, National Bureau of Economic Research, Cambridge, Mass.

Heckman, J., and Smith, J. (1995), “Assessing the Case for Social Experiments,” /The Journal of Economic Perspectives/, vol. 9, no. 2.

Kreps, D. M. (1997), “Intrinsic Motivation and Extrinsic Incentives,” /American Economic Review/, vol. 87, no. 2.

O’Connor, A. (2002), /Poverty Knowledge: Social Science, Social Policy, and the Poor in Twentieth Century U.S. History/, Princeton University Press, Princeton.

Rodrik, D. (2008), “The New Development Economics: We Shall Experiment, But How Shall We Learn?” Faculty Research Working Paper Series, RWP08-055, Harvard Kennedy School, Cambridge, Mass.

Sachs, J. (2005), /The End of Poverty: Economic Possibilities for Our Time/, Penguin, New York.

Sen, A. (1981), /Poverty and Famines: An Essay on Entitlement and Deprivation/, Clarendon Press, Oxford.

Spiegler, P. (2012), “The Unbearable Lightness of the Economics-Made-Fun Genre,” /Journal of Economic Methodology/, vol. 19, no. 3.

Weber, M. (2003), /General Economic History/, translated by Frank Knight, Dover, New York, 1927.

Previous message: [lbo-talk] Economists project Trump will win easily in 2020 — and by a bigger margin
Messages sorted by: [ date ] [ thread ] [ subject ] [ author ]

More information about the lbo-talk mailing list