In his 2018 book ‘Randomistas: How Radical Researchers Changed Our World’, Andrew Leigh told the stories of overturning conventional wisdom in medicine, politics, economics, and more. Incidentally, three ‘Randomistas’ – Abhijit Banerjee, Esther Duflo and Michael Kremer – won the Nobel prize in Economics for their study on poverty, by incorporating a century- old purely statistical concept called ‘randomized controlled trials’ (RCT).
They certainly worked on ‘poverty’, but let’s try to understand how ‘rich’ and novel their nobel-winning technique was. Of course, RCT was not the idea of Banerjee-Duflo-Kremer – they just implemented a century-old technique. In the past, experimenters conducted RCT to prove compression shocks after races speed up recovery for a marathon runner; for example.
RCT has been used to exhibit that a cup of java each morning provides protection against DNA breaks. Even in the context of controlling crime, the city of Chicago’s creative Becoming a Man (BAM) program reduced arrests by one-third to one-half. Some such interesting examples are illustrated by Andrew Leigh in ‘Randomistas’ with care with a pro-RCT intent of the author.
And, what’s more, people have been using this technique in several branches of science and social science. For example, the British Cabinet Office team, nicknamed the ‘nudge unit’, is an organisation that was set up to apply nudge theory to try to improve government policy and services as well as to save the UK government money.
It carried out many low-budget, behavioral economics experiments (essentially RCTs) on nudge treatments in oder to induce people to pay their taxes, attend jobtraining seminars, and donate their organs. So, RCT is not quite new in the domain of economics as well. The 2019 Nobel winning trio, on the other hand, popularised RCT in development economics. What’s an RCT? Assume the effectiveness of a treatment/intervention in a particular disease is under experimentation.
If patients are treated only with that treatment, the knowledge obtained from this trial may not be applicable in general due to several socio-economic-environmental factors influencing the responses. Here comes the idea of a ‘control’ group, where the performance of the treatment group is compared with a group of individuals treated in a similar condition with no treatment or an existing treatment other than the treatment under experimentation.
Now, how to allocate the participants between the two groups? The answer is ‘Randomisation’ – a random mechanism like tossing a coin for every participant. It prevents ‘selection bias’ (which occurs due to the prior knowledge of treatment any participant will receive) and insures against ‘accidental bias’. And, moreover, the theory of probability comes into play to analyse the data. The differences between the outcome from the two groups are used to assess the cause-effect relation between the intervention and the outcome.
There are historical instances of RCTs, as early as in 1537, by Ambroïse Paré, the French barber surgeon who orchestrated the renaissance of surgery. However, the statistical framework of RCT was laid by legendary statistician Sir RA Fisher about a century ago (see his books: ‘Statistical Methods for Research Workers’, 1925; ‘Design of Experiments’, 1935). So, RCT was not a new idea at all. In contrast, it’s a century-old idea with properly framed theories and millions of applications, specially in medical science.
In fact, I studied RCT as a PhD student of statistics, and subsequently I have been teaching RCT for statistics students in my classes for more than the last 20 years! And, interestingly, the so-called randomistas movement of the development economists is about two decades old! However, it is to be noted that Fisher himself was reluctant to apply statistics to social sciences, due to their ‘non-experimental’ nature. However, RCTs were formally used for drug development much later – mostly from the 1960s.
Also, hundreds of randomised field experiments had been conducted in the US for social policy evaluations before 1980. One of the pioneering RCTs is the famous ‘Perry Preschool program’ (1961) where randomly chosen half of 123 children born between 1958- 62 in Michigan entered the program at 3 or 4 years of age, and were educated by skilled professionals in nurseries and kindergarten, and their parents were also helped to improve their involvement.
Follow-up surveys showed that there was significant improvement in the test group in terms of cognitive skills at 15 years of age, access to university, criminal record, welfare, and employment status at 19 years of age. So, the benefits of RCTs, even in social sciences, were known a-priori, for more than half a century. Randomisation just ensures that different treatment groups are comparable. It also helps to estimate the associated error in estimation. The 2019 Nobel winner trio know that well – they mentioned ‘randomization’ as a ‘toolkit’ in their works.
However, the RCT movement has almost taken over the development economics field, starting in the mid- 1990s, through Michael Kremer’s studies on Kenya (1994) and PROGRESA experiment in Mexico (1997). With Banerjee and Duflo joining the movement, the growth has been very rapid during the last two decades. Till date, nearly 1,000 RCTs have been completed in 83 countries, studying various aspects of poverty.
Incidentally renowned economists like Martin Ravallion, Dani Rodrik, William Easterly, and 2015 Nobel prize winner Angus Deaton are against such RCTs in economic experiments. Ideologically this is close to the belief of Sir RA Fisher. Some believe that a biased observational study with a reasonably large sample size may well be closer to the truth in specific trials than an underpowered RCT. Critics argue that by reducing problems to manageable sizes for an RCT, ‘randomistas’ are avoiding the really big, hard questions whether a systemic change is needed.
Angus Deaton even believes that any dilution of the scientific method for social science renders the conclusions questionable. Critics also point at imperfect randomization, attrition, etc., and the fact that distribution of treatment effects, and numerous other external effects remain unidentified even by RCTs. An RCT doesn’t guarantee external validity. Even the ‘randomistas’, as Deaton called them in 2007, can’t dismiss their criticisms.
However, randomistas are among the most influential of development experts now. Overall, the idea of RCT has been taken over by the development economists, specially after the 2019 Economics Nobel prize. Did Sir RA Fisher, one of the greatest statisticians in history, ever think that implementation of the simple yet powerful idea of RCT in economic experiments could bag the Nobel prize after a century?
The statisticians who laid the foundation of RCT had confined its applications mostly within design of experiments concerning agriculture, and subsequently in medicine. However, some development economists were smart enough to borrow the century-old technique of RCT effectively, and revolutionize development economics. This is a story of great effort and tremendous success, no doubt about that. And, this is also an example where statisticians failed to touch the heartbeats of the mass despite theorising nice ideas; they quickly got engaged to theorise more advanced techniques. Poor statisticians would now keep on wondering whether the tremendous success of RCT by economists should be viewed as Statistics’ defeat to or victory over Economics!
And an economist would wonder how the implementation of just an age-old statistical technique can become more important in the field of Economics than even the economic theories and insights!
(The writer is Professor of Statistics, Indian Statistical Institute, Kolkata)