More Like this

1. FLAME: A Fast Large-scale Almost Matching Exactly Approach to Causal Inference

   Wang, Tianyu; Morucci, Marco; Awan, M. Usaid; Liu, Yameng; Roy, Sudeepa; Rudin, Cynthia; Volfovsky, Alexander (January 2021, Journal of machine learning research)

   null (Ed.)

   A classical problem in causal inference is that of matching, where treatment units need to be matched to control units based on covariate information. In this work, we propose a method that computes high quality almost-exact matches for high-dimensional categorical datasets. This method, called FLAME (Fast Large-scale Almost Matching Exactly), learns a distance metric for matching using a hold-out training data set. In order to perform matching efficiently for large datasets, FLAME leverages techniques that are natural for query processing in the area of database management, and two implementations of FLAME are provided: the first uses SQL queries and the second uses bit-vector techniques. The algorithm starts by constructing matches of the highest quality (exact matches on all covariates), and successively eliminates variables in order to match exactly on as many variables as possible, while still maintaining interpretable high-quality matches and balance between treatment and control groups. We leverage these high quality matches to estimate conditional average treatment effects (CATEs). Our experiments show that FLAME scales to huge datasets with millions of observations where existing state-of-the-art methods fail, and that it achieves significantly better performance than other matching methods. 

   more » « less
2. Adaptive Hyper-box Matching for Interpretable Individualized Treatment Effect Estimation

   Morucci, Marco; Orlandi, Vittorio; Roy, Sudeepa; Rudin, Cynthia; Volfovsky, Alexander (January 2020, Proceedings of the 36th Conference on Uncertainty in Artificial Intelligence (UAI))

   null (Ed.)

   We propose a matching method for observational data that matches units with others in unit-specific, hyper-box-shaped regions of the covariate space. These regions are large enough that many matches are created for each unit and small enough that the treatment effect is roughly constant throughout. The regions are found as either the solution to a mixed integer program, or using a (fast) approximation algorithm. The result is an interpretable and tailored estimate of the causal effect for each unit. 

   more » « less
3. An Approximation Algorithm for Blocking of an Experimental Design

   https://doi.org/10.1111/rssb.12545  

   Karmakar, Bikram (October 2022, Journal of the Royal Statistical Society Series B: Statistical Methodology)

   Abstract Blocked randomized designs are used to improve the precision of treatment effect estimates compared to a completely randomized design. A block is a set of units that are relatively homogeneous and consequently would tend to produce relatively similar outcomes if the treatment had no effect. The problem of finding the optimal blocking of the units into equal sized blocks of any given size larger than two is known to be a difficult problem—there is no polynomial time method guaranteed to find the optimal blocking. All available methods to solve the problem are heuristic methods. We propose methods that run in polynomial time and guarantee a blocking that is provably close to the optimal blocking. In all our simulation studies, the proposed methods perform better, create better homogeneous blocks, compared with the existing methods. Our blocking method aims to minimize the maximum of all pairwise differences of units in the same block. We show that bounding this maximum difference ensures that the error in the average treatment effect estimate is similarly bounded for all treatment assignments. In contrast, if the blocking bounds the average or sum of these differences, the error in the average treatment effect estimate can still be large in several treatment assignments. 

   more » « less
4. MALTS: Matching After Learning to Stretch

   Harsh Parikh; Cynthia Rudin; Alexander Volfovsky (January 2022, Journal of machine learning research)

   We introduce a flexible framework that produces high-quality almost-exact matches for causal inference. Most prior work in matching uses ad-hoc distance metrics, often leading to poor quality matches, particularly when there are irrelevant covariates. In this work, we learn an interpretable distance metric for matching, which leads to substantially higher quality matches. The learned distance metric stretches the covariate space according to each covariate's contribution to outcome prediction: this stretching means that mismatches on important covariates carry a larger penalty than mismatches on irrelevant covariates. Our ability to learn flexible distance metrics leads to matches that are interpretable and useful for the estimation of conditional average treatment effects. 

   more » « less
5. Calibration of Heterogeneous Treatment Effects in Randomized Experiments

   Leng, Yan; Drew, Dimmery (January 2024, Information systems research)

   Bapna, Ravi; Burtch, Gordon (Ed.)

   Machine learning is commonly used to estimate the heterogeneous treatment effects (HTEs) in randomized experiments. Using large-scale randomized experiments on the Facebook and Criteo platforms, we observe substantial discrepancies between machine learning-based treatment effect estimates and difference-in-means estimates directly from the randomized experiment. This paper provides a two-step framework for practitioners and researchers to diagnose and rectify this discrepancy. We first introduce a diagnostic tool to assess whether bias exists in the model-based estimates from machine learning. If bias exists, we then offer a model-agnostic method to calibrate any HTE estimates to known, unbiased, subgroup difference-in-means estimates, ensuring that the sign and magnitude of the subgroup estimates approximate the model-free benchmarks. This calibration method requires no additional data and can be scaled for large data sets. To highlight potential sources of bias, we theoretically show that this bias can result from regularization and further use synthetic simulation to show biases result from misspecification and high-dimensional features. We demonstrate the efficacy of our calibration method using extensive synthetic simulations and two real-world randomized experiments. We further demonstrate the practical value of this calibration in three typical policy-making settings: a prescriptive, budget-constrained optimization framework; a setting seeking to maximize multiple performance indicators; and a multitreatment uplift modeling setting. 

   more » « less