Does the micro evidence tell us anything important about development?

Another plug for grand theory, and disparagement of empirics.

Development economists spend their time these days performing randomized controlled experiments, in which a particular intervention like co-payments for mosquito bed nets are introduced into one group of villages and not into another matched set. This approach establishes causality with a level of certainty approaching that of the randomized trials used in pharmaceutical testing. But while such experiments are useful for evaluating the effectiveness of certain types of public policies, they all operate at a very micro level and don’t aggregate upwards into an understanding of the broader phenomenon of development. It is hard to imagine that all the work being done under this approach will leave anything behind of a conceptual nature that people will remember fifty years from now.

That is Francis Fukuyama, starting off an essay on Albert Hirschman, the great development economist who charted a very different path. (h/t to this commenter)

One of the great puzzles to me is why great thinkers commonly disparage other paths to knowledge. It’s politically effective, if your aim is to promote your agenda over theirs, but most of the time it’s tribal, self-serving, and hinders progress.

It’s certainly tempting to think that the quantitative and micro evidence doesn’t add up to much. Experiments and micro evidence are no panacea, but to say they haven’t changed our fundamental concepts of development and poverty reveals ignorance of the best literature.

Here’s a sample of review articles where the evidence shook my basic conceptions of how development works. All are required reading in my PhD course on political economy of development (along with Hirschman):

  • If you wanted to see how micro evidence has changed our understanding of who is poor and why, on everything from nutrition to capital, I would read Poor Economics by Banerjee and Duflo.
  • If you wanted to understand the new micro evidence helps explain growth puzzles and patterns of cross-country development, I’d recommend their more technical Growth Theory Through the Lens of Development Economics.
  • Psychology and behavioral economics is changing the way we think about economic change small and large. Sendhil Mullainathan’s new book will probably help fill this gap when it’s out, but in the meantime you can read his 2004 piece.
  • Pande and Udry and Besley and Ghatak both summarize how the micro evidence has changed our fundamental concepts of property rights and institutions, and ways (expected and not) they influence development.
  • Olken and Pande and Banerjee, Hama, and Mullainathan do the same for corruption and development.

To me, the great problem with some of the most interesting questions of political economy (“Who votes, protests, riots or rebels, and why?”, for instance) suffer from a terrific absence of microfoundations and good evidence. Grand theories and case studies are abundant, and important, but alone ultimately fail to answer the questions.

37 Responses

  1. I am deeply miserable that Francis Fukushima, the great Iraq War supporter, believes that others should seek knowledge in a way different from the one that he prefers.

  2. What i really like about your post is that you talk about the principle (the worth of micro evidence for development). I think that most debate on this topic gets bogged down on worthiness of one over another methods (e.g. RTCs) whereas very few get down to make a sound argument for the principle and moving forward its applications in the field… with the world becoming more connected, local context and local solutions will only become more important. I spoke recently with someone who told me a story of a community of devworkers in Washington interpreting a problem in a village in Kenya as that of water quality (solution= purify water, build more wells) whereas that same community asked for ways to help them build cohesion, and that wells and all other things will come as a result… a mismatch that’s not a rarity in development today. In any case, i’ll use the post to shamefully advertise some of the work we at UNDP will be doing in terms of getting down to micro evidence through using stories http://europeandcis.undp.org/blog/2013/01/14/disruption-incoming-storytelling-vs-polls/

    thanks again for the post!
    millie

  3. Excellent post and nice reading list. I might also suggest, as a counterpoint to Banerjee/Duflo and Karlan/Appel (More than Good Intentions) @ MIT J-PAL and IPA, two other works:

    1. Bill Easterly and Jessica Cohen’s “What Works in Development? Thinking Big and Thinking Small”, Brookings – excellent edited collection on both the promises and pitfalls of RCTs, etc. to examining microfoundations.

    2. Nancy Cartwright and Jeremy Hardie’s “Evidence-Based Policy: A Practical Guide to Doing it Better” which outlines the epistemological problems related to identifying exactly what causal principles are identified in an RCT (and why much of the advice on porting evidence from one RCT to another really doesn’t work that well, with suggestions on how to improve the process).

  4. Pingback: Micro-Evidence as it helps Broader Development | The World on Safari

  5. thanks for this! i am, however, confused, by this statement: “Grand theories and case studies are abundant, and important, but alone ultimately fail to answer the questions.”

    this suggests that micro-evidence/RCTs and case studies (by which i assume you mean qualitative or process work?) are fundamentally different. i am not sure this is the case: both can achieve high levels of internal validity, have rigorous data collection and analysis, and can function to test a hypothesis (from middle-range or grand theory) or generate new ones (or both). both case studies and RCTs have issues with external validity; both benefit from replication, cross-case comparison, and organizing theory or framework.

    my sense is that those using case studies go through a good deal more effort to explain the selection of their case/site(s), not just the sampling procedure within it (e.g. geddes, gerring), which is very helpful in overcoming concerns about *site* selection bias and generalizability. my sense is that case studies and RCTs are complementary in function in working between grounded/idiographic, middle-range, and grand/nomothetic theories — and that we would learn more if we used them in a more complementary way.

    what am i missing?

  6. but most of the time it’s tribal, self-serving, and hinders progress.

    For conservatives (who are almost by definition opposed to progress, and who tend to promote tribalism), this is a feature, not a bug.

  8. A few points:
    I would advocate for a middle ground between micro-empirics and grand theories. The micro-foundations suffer from questions about external validity, much more so than the authors of the above literature reviews (and you) probably acknowledge. Some have likened them to many small bricks that together construct a wall (of knowledge). Perhaps an analogy involving many irregular brick-like things aggregating to an unstable mound is more apt.
    Grand theories, of course, lack empirical tractability and application to specific cases. So what’s one to do? Natural experiments provide a middle ground. They often involv phenomenom that no researcher could manipulate on her own, so they are able to speak to big questions in social science with empirical rigor. Consider, for example, recent natural experiments that speak to the effects arbitrary colonial borders, the slave trade, media on various outcomes (incumbent vote share, genocide perpetration, fertility), property rights and development, diversity and public goods, and on and on.
    A second point is that the universality of causal forces varies across phenomenon. Thus, some phenomenon will lend themselves to generalizable study, others much less so. We all suffer from loss aversion or hyperbolic discounting, for example, and this means that studies applying these concepts in development economics probably have some external validity. I am much more skeptical of the generalizability of findings from more messy (but often more interesting) phenomenon. Corruption, clientelism, and political violence come to mind.
    Re: lamenting the tribal critic: I would disagree, for the same reason that we don’t silence dissent or stifle competition in society. And critics are useful for highlighting shortcomings that should be managed or improved upon.

  9. I think it depends on what one means by “microfoundations.” If we mean grounding models in rational actors, then I don’t see how that is much better than grand theories built on fuzzy evidence. If one means observing political and economic phenomena at the ground-level and using that to build more general, macro-level theories, then sure.

  10. Pingback: On the value of Randomized Trials in Development | Saad Gulzar

  11. Hi Tim, as a former philosophy of science hack I feel compelled to note that you are citing the Tom ‘paradigm shift’ Kuhn view of how science ‘advances’ — which is popular but not at all universally accepted. I tend to favor Karl Popper’s view, which gives evidence and methodology a bit more credit.

  12. Two posts that bring to mind the corrupted quotation from Planck: “Science advances one funeral at a time.”

    And according to WikiQuote, the actual: “A new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its opponents eventually die, and a new generation grows up that is familiar with it.”

    I find it particularly interesting that these salvos are intermingled in my twitter stream with pointers to Jerven and Yglesias material on how bad the underlying data being used for grand theories really is.