An official website of the United States government
Abstract Policy interventions can spill over to units of a population that is not directly exposed to the policy but are geographically close to the units receiving the intervention. In recent work, investigations of spillover effects on neighbouring regions have focused on estimating the average treatment effect of a particular policy in an observed setting. Our research question broadens this scope by asking what policy consequences would the treated units have experienced under counterfactual exposure settings. When we only observe treated unit(s) surrounded by controls—as is common when a policy intervention is implemented in a single city or state—this effect inquires about the policy effects under a counterfactual neighbourhood policy status that we do not, in actuality, observe. In this work, we extend difference-in-differences approaches to spillover settings and develop identification conditions required to evaluate policy effects in counterfactual treatment scenarios. These causal quantities are policy-relevant for designing effective policies for populations subject to various neighbourhood statuses. We develop several estimators that have desirable properties. We provide an illustrative data application to the Philadelphia beverage tax study.
more »
« less
Doubly robust estimation of policy-relevant causal effects under interference
Abstract To comprehensively evaluate a public policy intervention, researchers must consider the effects of the policy not just on the implementing region, but also nearby, indirectly affected regions. For example, an excise tax on sweetened beverages in Philadelphia, Pennsylvania was shown to not only be associated with a decrease in volume sales of taxed beverages in Philadelphia, but also an increase in sales in nontaxed bordering counties. The latter association may be explained by cross-border shopping behaviours of Philadelphia residents and indicate a causal effect of the tax on nearby regions, which may drastically offset the total effect of the intervention. In this paper, we adapt doubly robust difference-in-differences methodology to estimate distinct causal effects on the implementing and neighbouring control regions when they are geographically separable and data exists from an unaffected control region. Our approach adjusts for potential confounding in quasi-experimental evaluations and relaxes standard assumptions on model specification while accounting for geographically separable interference, repeated observations, spatial correlation, and unknown effect heterogeneity. We apply these methods to evaluate the effect of the Philadelphia beverage tax on taxed beverage sales in 231 Philadelphia and bordering county stores. We also use our methods to explore effect heterogeneity across geographical features.
more »
« less
- Award ID(s):
- 2149716
- PAR ID:
- 10556081
- Publisher / Repository:
- Oxford University Press
- Date Published:
- Journal Name:
- Journal of the Royal Statistical Society Series C: Applied Statistics
- Volume:
- 74
- Issue:
- 2
- ISSN:
- 0035-9254
- Format(s):
- Medium: X Size: p. 530-549
- Size(s):
- p. 530-549
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
ABSTRACT Researchers commonly use difference-in-differences (DiD) designs to evaluate public policy interventions. While methods exist for estimating effects in the context of binary interventions, policies often result in varied exposures across regions implementing the policy. Yet, existing approaches for incorporating continuous exposures face substantial limitations in addressing confounding variables associated with intervention status, exposure levels, and outcome trends. These limitations significantly constrain policymakers’ ability to fully comprehend policy impacts and design future interventions. In this work, we propose new estimators for causal effect curves within the DiD framework, accounting for multiple sources of confounding. Our approach accommodates misspecification of a subset of intervention, exposure, and outcome models while avoiding any parametric assumptions on the effect curve. We present the statistical properties of the proposed methods and illustrate their application through simulations and a study investigating the heterogeneous effects of a nutritional excise tax under different levels of accessibility to cross-border shopping.more » « less
-
Behavioral science interventions have the potential to address longstanding policy problems, but their effects are typically heterogeneous across contexts (e.g., teachers, schools, and geographic regions). This contextual heterogeneity is poorly understood, however, which reduces the field’s impact and its understanding of mechanisms. Here, we present an efficient way to interrogate heterogeneity and address these gaps in knowledge. This method a) presents scenarios that vividly represent different moderating contexts, b) measures a short-term behavioral outcome (e.g., an academic choice) that is known to relate to typical intervention outcomes (e.g., academic achievement), and c) assesses the causal effect of the moderating context on the link between the psychological variable typically targeted by interventions and this short-term outcome. We illustrated the utility of this approach across four experiments (total n = 3,235) that directly tested contextual moderators of the links between growth mindset, which is the belief that ability can be developed, and students’ academic choices. The present results showed that teachers’ growth mindset-supportive messages and the structural opportunities they provide moderated the link between students’ mindsets and their choices (studies 1 to 3). This pattern was replicated in a nationally representative sample of adolescents and did not vary across demographic subgroups (study 2), nor was this pattern the result of several possible confounds (studies 3 to 4). Discussion centers on how this method of interrogating contextual heterogeneity can be applied to other behavioral science interventions and broaden their impact in other policy domains.more » « less
-
Predicting how different interventions will causally affect a specific individual is important in a variety of domains such as personalized medicine, public policy, and online marketing. There are a large number of methods to predict the effect of an existing intervention based on historical data from individuals who received it. However, in many settings it is important to predict the effects of novel interventions (e.g., a newly invented drug), which these methods do not address. Here, we consider zero-shot causal learning: predicting the personalized effects of a novel intervention. We propose CaML, a causal meta-learning framework which formulates the personalized prediction of each intervention’s effect as a task. CaML trains a single meta-model across thousands of tasks, each constructed by sampling an intervention, its recipients, and its nonrecipients. By leveraging both intervention information (e.g., a drug’s attributes) and individual features (e.g., a patient’s history), CaML is able to predict the personalized effects of novel interventions that do not exist at the time of training. Experimental results on real world datasets in large-scale medical claims and cell-line perturbations demonstrate the effectiveness of our approach. Most strikingly, CaML’s zero-shot predictions outperform even strong baselines trained directly on data from the test interventions.more » « less
-
Many policies in the US are determined locally, e.g., at the county-level. Local policy regimes provide flexibility between regions, but may become less effective in the presence of geographic spillovers, where populations circumvent local restrictions by traveling to less restricted regions nearby. Due to the endogenous nature of policymaking, there have been few opportunities to reliably estimate causal spillover effects or evaluate their impact on local policies. In this work, we identify a novel setting and develop a suitable methodology that allow us to make unconfounded estimates of spillover effects of local policies. Focusing on California’s Blueprint for a Safer Economy, we leverage how county-level mobility restrictions were deterministically set by public COVID-19 severity statistics, enabling a regression discontinuity design framework to estimate spillovers between counties. We estimate these effects using a mobility network with billions of timestamped edges and find significant spillover movement, with larger effects in retail, eating places, and gyms. Contrasting local and global policy regimes, our spillover estimates suggest that county-level restrictions are only 54% as effective as statewide restrictions at reducing mobility. However, an intermediate strategy of macro-county restrictions—where we optimize county partitions by solving a minimum k-cut problem on a graph weighted by our spillover estimates— can recover over 90% of statewide mobility reductions, while maintaining substantial flexibility between counties.more » « less
