More Like this

1. Cascade-Based Randomization for Inferring Causal Effects under Diffusion Interference

   https://doi.org/10.1609/icwsm.v18i1.31322  

   Fatemi, Zahra; Pouget-Abadie, Jean; Zheleva, Elena (May 2024, Proceedings of the International AAAI Conference on Web and Social Media)

   The presence of interference, where the outcome of an individual may depend on the treatment assignment and behavior of neighboring nodes, can lead to biased causal effect estimation. Current approaches to network experiment design focus on limiting interference through cluster-based randomization, in which clusters are identified using graph clustering, and cluster randomization dictates the node assignment to treatment and control. However, cluster-based randomization approaches perform poorly when interference propagates in cascades, whereby the response of individuals to treatment propagates to their multi-hop neighbors. When we have knowledge of the cascade seed nodes, we can leverage this interference structure to mitigate the resulting causal effect estimation bias. With this goal, we propose a cascade-based network experiment design that initiates treatment assignment from the cascade seed node and propagates the assignment to their multi-hop neighbors to limit interference during cascade growth and thereby reduce the overall causal effect estimation error. Our extensive experiments on real-world and synthetic datasets demonstrate that our proposed framework outperforms the existing state-of-the-art approaches in estimating causal effects in network data. 

   more » « less
2. Network experiment designs for inferring causal effects under interference

   https://doi.org/10.3389/fdata.2023.1128649  

   Fatemi, Zahra; Zheleva, Elena (April 2023, Frontiers in Big Data)

   Current approaches to A/B testing in networks focus on limiting interference, the concern that treatment effects can “spill over” from treatment nodes to control nodes and lead to biased causal effect estimation. In the presence of interference, two main types of causal effects are direct treatment effects and total treatment effects. In this paper, we propose two network experiment designs that increase the accuracy of direct and total effect estimations in network experiments through minimizing interference between treatment and control units. For direct treatment effect estimation, we present a framework that takes advantage of independent sets and assigns treatment and control only to a set of non-adjacent nodes in a graph, in order to disentangle peer effects from direct treatment effect estimation. For total treatment effect estimation, our framework combines weighted graph clustering and cluster matching approaches to jointly minimize interference and selection bias. Through a series of simulated experiments on synthetic and real-world network datasets, we show that our designs significantly increase the accuracy of direct and total treatment effect estimation in network experiments. 

   more » « less
3. Experimental Design in Two-Sided Platforms: An Analysis of Bias

   https://doi.org/10.1287/mnsc.2021.4247  

   Johari, Ramesh; Li, Hannah; Liskovich, Inessa; Weintraub, Gabriel Y. (October 2022, Management Science)

   We develop an analytical framework to study experimental design in two-sided marketplaces. Many of these experiments exhibit interference, where an intervention applied to one market participant influences the behavior of another participant. This interference leads to biased estimates of the treatment effect of the intervention. We develop a stochastic market model and associated mean field limit to capture dynamics in such experiments and use our model to investigate how the performance of different designs and estimators is affected by marketplace interference effects. Platforms typically use two common experimental designs: demand-side “customer” randomization ([Formula: see text]) and supply-side “listing” randomization ([Formula: see text]), along with their associated estimators. We show that good experimental design depends on market balance; in highly demand-constrained markets, [Formula: see text] is unbiased, whereas [Formula: see text] is biased; conversely, in highly supply-constrained markets, [Formula: see text] is unbiased, whereas [Formula: see text] is biased. We also introduce and study a novel experimental design based on two-sided randomization ([Formula: see text]) where both customers and listings are randomized to treatment and control. We show that appropriate choices of [Formula: see text] designs can be unbiased in both extremes of market balance while yielding relatively low bias in intermediate regimes of market balance. This paper was accepted by David Simchi-Levi, revenue management and market analytics. 

   more » « less
4. Near-Optimal A-B Testing

   https://doi.org/10.1287/mnsc.2019.3424  

   Bhat, Nikhil; Farias, Vivek F.; Moallemi, Ciamac C.; Sinha, Deeksha (April 2020, Management Science)

   We consider the problem of A-B testing when the impact of the treatment is marred by a large number of covariates. Randomization can be highly inefficient in such settings, and thus we consider the problem of optimally allocating test subjects to either treatment with a view to maximizing the precision of our estimate of the treatment effect. Our main contribution is a tractable algorithm for this problem in the online setting, where subjects arrive, and must be assigned, sequentially, with covariates drawn from an elliptical distribution with finite second moment. We further characterize the gain in precision afforded by optimized allocations relative to randomized allocations, and show that this gain grows large as the number of covariates grows. Our dynamic optimization framework admits several generalizations that incorporate important operational constraints such as the consideration of selection bias, budgets on allocations, and endogenous stopping times. In a set of numerical experiments, we demonstrate that our method simultaneously offers better statistical efficiency and less selection bias than state-of-the-art competing biased coin designs. 

   more » « less
5. Adaptive experiments toward learning treatment effect heterogeneity

   https://doi.org/10.1093/jrsssb/qkaf006  

   Wei, Waverly; Ma, Xinwei; Wang, Jingshen (February 2025, Journal of the Royal Statistical Society Series B: Statistical Methodology)

   Abstract Understanding treatment effect heterogeneity has become an increasingly popular task in various fields, as it helps design personalized advertisements in e-commerce or targeted treatment in biomedical studies. However, most of the existing work in this research area focused on either analysing observational data based on strong causal assumptions or conducting post hoc analyses of randomized controlled trial data, and there has been limited effort dedicated to the design of randomized experiments specifically for uncovering treatment effect heterogeneity. In the manuscript, we develop a framework for designing and analysing response adaptive experiments toward better learning treatment effect heterogeneity. Concretely, we provide response adaptive experimental design frameworks that sequentially revise the data collection mechanism according to the accrued evidence during the experiment. Such design strategies allow for the identification of subgroups with the largest treatment effects with enhanced statistical efficiency. The proposed frameworks not only unify adaptive enrichment designs and response-adaptive randomization designs but also complement A/B test designs in e-commerce and randomized trial designs in clinical settings. We demonstrate the merit of our design with theoretical justifications and in simulation studies with synthetic e-commerce and clinical trial data. 

   more » « less