Maximilian Kasy: Why would you add random noise?
Maximilian Kasy enjoys life on the edge or, more precisely, at the intersection: “…between applied research, statistical theory, and general methodological … issues,.” At a time when borders are much in the news, he finds it “very fruitful and exciting to cross the boundaries between these.”
Maximilian Kasy received degrees in economics and statistics at UC Berkeley. His recent research challenges a widespread practice in experimental science: the use of randomized controlled trials (RCTs). According to Kasy, randomization makes sense as long as experimental units are observationally homogeneous, even though unobserved heterogeneity remains. However, in fields such as medicine and economics, there is not only unobserved heterogeneity but observed heterogeneity as well: individuals might differ in their age or gender, for example. The objective of controlled trials is to have balanced treatment groups so we can “compare apples with apples.” Kasy’s approach asserts that in studies where there is such observed heterogeneity, there is generally a unique optimal non-random way to assign people/study subjects to each group!
Based on a decision theory perspective, the optimal assignment is determined by minimizing the “risk” or “mean squared error” (MSE) between the estimator and what is being estimated. The term “risk” turns out to yield a way to define and measure “balance” between treatment and control groups. You don’t just want to balance the means of experimental and control groups, but rather the entire distribution of observed heterogeneity. Randomized assignment to groups is like adding random noise to the estimators, which nobody would do, so: “Why would you add random noise to experimental designs?” Kasy asks, noting that adding noise never reduces risk.
In his lecture at a conference at Harvard, Kasy described the Project STAR study of class size, well-known in the field of education. Children at about 80 Tennessee schools were assigned randomly to small or large classes in their home school. When the class assignments were examined closely, it was noted that the average birthdates of kids assigned in some large classes was about 4 months earlier than the average birthdates of kids in the smaller classes – a large difference when considering test scores.; at another school, the larger classes contained 45% boys, smaller classes had 40% boys; and about 86% of students in large classes, but only 33% of students in small classes, received free lunches – a typical indicator of poverty. The Project STAR was widely respected, but it shows a common weakness of randomized assignment to experimental groups.
Use of the optimal design identified by Kasy’s approach reduced the MSE in the STAR experiment by 19%, compared to the MSE obtained with randomized assignment. Using the optimal design increased the precision of the estimates, making them much closer to the values being estimated. “I would hope that the methods proposed and arguments made in my research have some impact on the actual practice of empirical research in economics and neighboring fields,” said Kasy. His pursuit of more precise estimates introduces a fresh perspective in experimental design, one that is especially well-suited to studies of economic inequality, labor markets, public finances, and development economics.
The preceding article is part of a series featuring the scientific work of 20 young Austrian researchers, all who are active members of the OSTA's Research and Innovation Network Austria. The initial presentation of their work took place at the ASCINA poster session under the auspices of the "Austrian Research and Innovation Talk" in Toronto on October 21, 2016. Three of these scientists were subsequently awarded the ASCINA award the same evening, honoring their outstanding scientific work.