Marginal structural models (MSMs) can be used to estimate the causal effect of a potentially timevarying treatment in the presence of timedependent confounding via weighted regression. The standard approach of using inverse probability of treatment weighting (IPTW) can be sensitive to model misspecification and lead to highvariance estimates due to extreme weights. Various methods have been proposed to partially address this, including covariate balancing propensity score (CBPS) to mitigate treatment model misspecification, and truncation and stabilizedIPTW (sIPTW) to temper extreme weights. In this article, we present kernel optimal weighting (KOW), a convexoptimizationbased approach that finds weights for fitting the MSMs that flexibly balance timedependent confounders while simultaneously penalizing extreme weights, directly addressing the above limitations. We further extend KOW to control for informative censoring. We evaluate the performance of KOW in a simulation study, comparing it with IPTW, sIPTW, and CBPS. We demonstrate the use of KOW in studying the effect of treatment initiation on timetodeath among people living with human immunodeficiency virus and the effect of negative advertising on elections in the United States.
Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to nonfederal websites. Their policies may differ from this site.

Abstract 
Statistical distances (SDs), which quantify the dissimilarity between probability distributions, are central to machine learning and statistics. A modern method for estimating such distances from data relies on parametrizing a variational form by a neural network (NN) and optimizing it. These estimators are abundantly used in practice, but corresponding performance guarantees are partial and call for further exploration. In particular, there seems to be a fundamental tradeoff between the two sources of error involved: approximation and estimation. While the former needs the NN class to be rich and expressive, the latter relies on controlling complexity. This paper explores this tradeoff by means of nonasymptotic error bounds, focusing on three popular choices of SDs—KullbackLeibler divergence, chisquared divergence, and squared Hellinger distance. Our analysis relies on nonasymptotic function approximation theorems and tools from empirical process theory. Numerical results validating the theory are also provided.more » « less

In many informationtheoretic channel coding problems, adding an input cost constraint to the operational setup amounts to restricting the optimization domain in the capacity formula. This paper shows that, in contrast to common belief, such a simple modification does not hold for the costconstrained (CC) wiretap channel (WTC). The secrecycapacity of the discrete memoryless (DM) WTC without cost constraints is described by a single auxiliary random variable. For the CC DMWTC, however, we show that two auxiliaries are necessary to achieve capacity. Specifically, we first derive the secrecycapacity formula, proving the direct part via superposition coding. Then, we provide an example of a CC DMWTC whose secrecycapacity cannot be achieved using a single auxiliary. This establishes the fundamental role of superposition coding over CC WTCs.more » « less

Statistical divergences (SDs), which quantify the dissimilarity between probability distributions, are a basic constituent of statistical inference and machine learning. A modern method for estimating those divergences relies on parametrizing an empirical variational form by a neural network (NN) and optimizing over parameter space. Such neural estimators are abundantly used in practice, but corresponding performance guarantees are partial and call for further exploration. We establish nonasymptotic absolute error bounds for a neural estimator realized by a shallow NN, focusing on four popular 𝖿divergencesKullbackLeibler, chisquared, squared Hellinger, and total variation. Our analysis relies on nonasymptotic function approximation theorems and tools from empirical process theory to bound the two sources of error involved: function approximation and empirical estimation. The bounds characterize the effective error in terms of NN size and the number of samples, and reveal scaling rates that ensure consistency. For compactly supported distributions, we further show that neural estimators of the first three divergences above with appropriate NN growthrate are minimax rateoptimal, achieving the parametric convergence rate.more » « less

Inverse probability of treatment weighting (IPTW), which has been used to estimate average treatment effects (ATE) using observational data, tenuously relies on the positivity assumption and the correct specification of the treatment assignment model, both of which are problematic assumptions in many observational studies. Various methods have been proposed to overcome these challenges, including truncation, covariate‐balancing propensity scores, and stable balancing weights. Motivated by an observational study in spine surgery, in which positivity is violated and the true treatment assignment model is unknown, we present the use of optimal balancing by kernel optimal matching (KOM) to estimate ATE. By uniformly controlling the conditional mean squared error of a weighted estimator over a class of models, KOM simultaneously mitigates issues of possible misspecification of the treatment assignment model and is able to handle practical violations of the positivity assumption, as shown in our simulation study. Using data from a clinical registry, we apply KOM to compare two spine surgical interventions and demonstrate how the result matches the conclusions of clinical trials that IPTW estimates spuriously refute.

null (Ed.)In many informationtheoretic channel coding prob lems, adding an input cost constraint to the operational setup amounts to restricting the optimization domain in the capacity formula. This paper shows that, in contrast to common belief, such a simple modification does not hold for the costconstrained (CC) wiretap channel (WTC). The secrecycapacity of the discrete memoryless (DM) WTC without cost constraints is described by a single auxiliary random variable. For the CC DMWTC, however, we show that two auxiliaries are necessary to achieve capacity. Specifically, we first derive the secrecycapacity formula, proving the direct part via superposition coding. Then, we provide an example of a CC DMWTC whose secrecycapacity cannot be achieved using a single auxiliary. This establishes the fundamental role of superposition coding over CC WTCs.more » « less

null (Ed.)Statistical distances (SDs), which quantify the dissimilarity between probability distri butions, are central to machine learning and statistics. A modern method for esti mating such distances from data relies on parametrizing a variational form by a neu ral network (NN) and optimizing it. These estimators are abundantly used in prac tice, but corresponding performance guar antees are partial and call for further ex ploration. In particular, there seems to be a fundamental tradeoff between the two sources of error involved: approximation and estimation. While the former needs the NN class to be rich and expressive, the latter relies on controlling complexity. This paper explores this tradeoff by means of nonasymptotic error bounds, focusing on three popular choices of SDs—Kullback Leibler divergence, chisquared divergence, and squared Hellinger distance. Our analysis relies on nonasymptotic function approxima tion theorems and tools from empirical pro cess theory. Numerical results validating the theory are also provided.more » « less

We consider the problem of softcovering with constant composition superposition codes and characterize the optimal softcovering exponent. A doubleexponential concentration bound for deviation of the exponent from its mean is also established. We demonstrate an application of the result to achieving the secrecycapacity region of a broadcast channel with confidential messages under a percodeword cost constraint. This generalizes the recent characterization of the wiretap channel secrecycapacity under an average cost constraint, highlighting the potential utility of the superposition softcovering result to the analysis of coding problems.more » « less

Minimax optimal convergence rates for numerous classes of stochastic convex optimization problems are well characterized, where the majority of results utilize iterate averaged stochastic gradient descent (SGD) with polynomially decaying step sizes. In contrast, the behavior of SGD’s final iterate has received much less attention despite the widespread use in practice. Motivated by this observation, this work provides a detailed study of the following question: what rate is achievable using the final iterate of SGD for the streaming least squares regression problem with and without strong convexity? First, this work shows that even if the time horizon T (i.e. the number of iterations that SGD is run for) is known in advance, the behavior of SGD’s final iterate with any polynomially decaying learning rate scheme is highly suboptimal compared to the statistical minimax rate (by a condition number factor in the strongly convex √ case and a factor of shows that Step Decay schedules, which cut the learning rate by a constant factor every constant number of epochs (i.e., the learning rate decays geometrically) offer significant improvements over any polynomially decaying step size schedule. In particular, the behavior of the final iterate with step decay schedules is off from the statistical minimax rate by only log factors (in the condition number for the strongly convex case, and in T in the nonstrongly convex case). Finally, in stark contrast to the known horizon case, this paper shows that the anytime (i.e. the limiting) behavior of SGD’s final iterate is poor (in that it queries iterates with highly suboptimal function value infinitely often, i.e. in a limsup sense) irrespective of the stepsize scheme employed. These results demonstrate the subtlety in establishing optimal learning rate schedules (for the final iterate) for stochastic gradient procedures in fixed time horizon settings.more » « less

In offline reinforcement learning (RL), the goal is to learn a highly rewarding policy based solely on a dataset of historical interactions with the environment. The ability to train RL policies offline would greatly expand where RL can be applied, its data efficiency, and its experimental velocity. Prior work in offline RL has been confined almost exclusively to modelfree RL approaches. In this work, we present MOReL, an algorithmic framework for modelbased offline RL. This framework consists of two steps: (a) learning a pessimistic MDP (PMDP) using the offline dataset; (b) learning a nearoptimal policy in this PMDP. The learned PMDP has the property that for any policy, the performance in the real environment is approximately lowerbounded by the performance in the PMDP. This enables it to serve as a good surrogate for purposes of policy evaluation and learning, and overcome common pitfalls of modelbased RL like model exploitation. Theoretically, we show that MOReL is minimax optimal (up to log factors) for offline RL. Through experiments, we show that MOReL matches or exceeds stateoftheart results in widely studied offline RL benchmarks. Moreover, the modular design of MOReL enables future advances in its components (e.g., in model learning, planning etc.) to directly translate into improvements for offline RL.more » « less