All across the social sciences, from development economics to political science departments, researchers are going into the field to collect data and learn about the world. While much has been gained from the successes of randomized controlled trials, stories of failed projects often do not get told. In Failing in the Field, Dean Karlan and Jacob Appel delve into the common causes of failure in field research, so that researchers might avoid similar pitfalls in future work.
Drawing on the experiences of top social scientists working in developing countries, this book delves into failed projects and helps guide practitioners as they embark on their research. From experimental design and implementation to analysis and partnership agreements, Karlan and Appel show that there are important lessons to be learned from failures at every stage. They describe five common categories of failures, review six case studies in detail, and conclude with some reflections on best (and worst) practices for designing and running field projects, with an emphasis on randomized controlled trials. There is much to be gained from investigating what has previously not worked, from misunderstandings by staff to errors in data collection.
Cracking open the taboo subject of the stumbles that can take place in the implementation of research studies, Failing in the Field is a valuable "how-not-to" handbook for conducting fieldwork and running randomized controlled trials in development settings.
Dean Karlan is professor of economics at Yale University and president of Innovations for Poverty Action. He is the recipient of a Guggenheim Fellowship. Jacob Appel previously worked with Innovations for Poverty Action, and is currently pursuing his MPA at Princeton University's Woodrow Wilson School of Public and International Affairs. Karlan and Appel are the coauthors of More Than Good Intentions: How a New Economics Is Helping to Solve Global Poverty.
"Highly recommended."--Karen Shook, Times Higher Education
"When we read about a successful program that mitigates poverty or improves health, we seldom consider how much work went into its evaluation and how many ways the process could go wrong. Drawing on their own and others' extensive experience in the field, Karlan and Appel provide vivid examples of failure to help future evaluators avoid common pitfalls."--Paul Brest, Stanford Law School
"This unusual and refreshing book is a good complement to the existing literature on impact evaluation and documents the practical issues involved in implementing randomized control trials. The authors look at real examples of what goes wrong in the field and provide a nice framework for thinking about how to avoid failures."--Rachel Glennerster, coauthor of Running Randomized Evaluations
"Using a rich set of examples, Failing in the Field describes failures that occur because the design or implementation of a research study does not yield data that can answer the questions it was intended to. By showing that mistakes in research design can be systematic, this book could benefit many students before they embark on their own studies. It was a pleasure to read."--Karla Hoff, World Bank
Table of Contents:
Introduction: Why Failures? 1
Part I Leading Causes of Research Failures 17
1 Inappropriate Research Setting 19
2 Technical Design Flaws 29
3 Partner Organization Challenges 40
4 Survey and Measurement Execution Problems 51
5 Low Participation Rates 62
Part II Case Studies 71
6 Credit and Financial Literacy Training: No Delivery Means No Impact 73
7 Interest Rate Sensitivity: Ignoring the Elephant in the Room 84
8 Youth Savings: Real Money Drumming up Fake People 94
9 Poultry Loans: Trying to Fly without a Pilot 105
10 Child Health and Business Training with Credit: No Such Thing as a Simple Study 114
11 Bundling Credit and Insurance: Turns Out More Is Less 125
Appendix | Checklist for Avoiding Failures 138