Attributing Effects to A Cluster Randomized Get-Out-The-Vote Campaign: An Application of Randomization Inference Using Full Matching.
(July 2005)Presented at annual meeting of the Political Methodology Section of the American Political Science Association.Abstract
Statistical analysis requires a probability model: commonly, a model for the dependence of outcomes Y on confounders X and a potentially causal variable Z. When the goal of the analysis is to infer Z's effects on Y , this requirement introduces an element of circularity: in order to decide how Z affects Y , the analyst first determines, speculatively, the manner of Y 's dependence on Z and other variables. This paper takes a statistical perspective that avoids such circles, permitting analysis of Z's effects on Y even as the statistician remains entirely agnostic about the conditional distribution of Y given X and Z, or perhaps even denies that such a distribution exists. Our assumptions instead pertain to the conditional distribution Z|X, and the role of speculation in settling them is reduced by the use of such techniques as propensity scores, poststratification, testing for overt bias before accepting a poststratification, and optimal full matching. Such beginnings pave the way for ``randomization inference'', an approach which, despite a long history in the analysis of designed experiments, is relatively new to political science and to other fields in which experimental data are rarely available. The approach applies to both experiments and observational studies. We illustrate this by applying it to analyze A. Gerber and D. Green's New Haven Vote 98 campaign. Conceived as both a get-out-the-vote campaign and a field experiment in political participation, the campaign as it turned out was in some ways more similar to an observational study than to a randomized experiment. Our analysis uses the strengths of the design of their study while adjusting for irregularities ignored by the original analysis. We estimate the number of voters who would not have voted had the campaign not prompted them to --- that is, the total number of votes attributable to the interventions of the campaigners. Both our statistical inferences about these attributable effects and the stratification and matching that precede them rely on quite recent developments from statistics; our matching, in particular, has novel features of potentially wide applicability. Our broad findings resemble those of the original analysis by Gerber and Green (2000), although in the small, the method offers additional information as to the campaign's effects upon interestingly different subgroups, such as older voters or those who have not voted in a previous election.