An official website of the United States government
Background: Outcome measures that are count variables with excessive zeros are common in health behaviors research. Examples include the number of standard drinks consumed or alcohol‐related problems experienced over time. There is a lack of empirical data about the relative performance of prevailing statistical models for assessing the efficacy of interventions when outcomes are zero‐inflated, particularly compared with recently developed marginalized count regression approaches for such data.Methods: The current simulation study examined five commonly used approaches for analyzing count outcomes, including two linear models (with outcomes on raw and log‐transformed scales, respectively) and three prevailing count distribution‐based models (ie, Poisson, negative binomial, and zero‐inflated Poisson (ZIP) models). We also considered the marginalized zero‐inflated Poisson (MZIP) model, a novel alternative that estimates the overall effects on the population mean while adjusting for zero‐inflation. Motivated by alcohol misuse prevention trials, extensive simulations were conducted to evaluate and compare the statistical power and Type I error rate of the statistical models and approaches across data conditions that varied in sample size ( to 500), zero rate (0.2 to 0.8), and intervention effect sizes.Results: Under zero‐inflation, the Poisson model failed to control the Type I error rate, resulting in higher than expected false positive results. When the intervention effects on the zero (vs. non‐zero) and count parts were in the same direction, the MZIP model had the highest statistical power, followed by the linear model with outcomes on the raw scale, negative binomial model, and ZIP model. The performance of the linear model with a log‐transformed outcome variable was unsatisfactory.Conclusions: The MZIP model demonstrated better statistical properties in detecting true intervention effects and controlling false positive results for zero‐inflated count outcomes. This MZIP model may serve as an appealing analytical approach to evaluating overall intervention effects in studies with count outcomes marked by excessive zeros.
more »
« less
Evaluating the power of the causal impact method in observational studies of HCV treatment as prevention
Objectives: The causal impact method (CIM) was recently introduced for evaluation of binary interventions using observational time-series data. The CIM is appealing for practical use as it can adjust for temporal trends and account for the potential of unobserved confounding. However, the method was initially developed for applications involving large datasets and hence its potential in small epidemiological studies is still unclear. Further, the effects that measurement error can have on the performance of the CIM have not been studied yet. The objective of this work is to investigate both of these open problems. Methods: Motivated by an existing dataset of HCV surveillance in the UK, we perform simulation experiments to investigate the effect of several characteristics of the data on the performance of the CIM and extend the method to deal with this problem. Results: We identify multiple characteristics of the data that affect the ability of the CIM to detect an intervention effect including the length of time-series, the variability of the outcome and the degree of correlation between the outcome of the treated unit and the outcomes of controls. We show that measurement error can introduce biases in the estimated intervention effects and heavily reduce the power of the CIM. Using an extended CIM, some of these adverse effects can be mitigated. Conclusions: The CIM can provide satisfactory power in public health interventions. The method may provide misleading results in the presence of measurement error.
more »
« less
- Award ID(s):
- 1854934
- PAR ID:
- 10357267
- Date Published:
- Journal Name:
- Statistical communications in infectious diseases
- Volume:
- 30
- Issue:
- 1
- ISSN:
- 1948-4690
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Purpose This study synthesized effects of interventions on language outcomes of young children (ages 0–8 years) with autism and evaluated the extent to which summary effects varied by intervention, participant, and outcome characteristics. Method A subset of effect sizes gathered for a larger meta-analysis (the Autism Intervention Meta-analysis or Project AIM) examining the effects of interventions for young children with autism, which were specific to language outcomes, was analyzed. Robust variance estimation and metaregression were used to calculate summary and moderated effects while controlling for intercorrelation among outcomes within studies. Results A total of 221 outcomes were gathered from 60 studies. The summary effect of intervention on language outcomes was small but significant. Summary effects were larger for expressive and composite language outcomes compared to receptive language outcomes. Interventions implemented by clinicians, or by clinicians and caregivers together, had summary effects that were significantly larger than interventions implemented by caregivers alone. Participants' pretreatment language age equivalent scores positively and significantly moderated intervention effects, such that effects were significantly larger on average when samples of children had higher pretreatment language levels. Effects were not moderated by cumulative intervention intensity, intervention type, autism symptomatology, chronological age, or the proximity or boundedness of outcomes. Study quality concerns were apparent for a majority of included outcomes. Conclusions We found evidence that intervention can facilitate improvements in language outcomes for young children with autism. Effects were largest for expressive and composite language outcomes, for children with initially higher language abilities, and for interventions implemented by clinicians or by caregivers and clinicians combined. However, quality concerns of included studies and borderline significance of some results temper our conclusions regarding intervention effectiveness and corresponding moderators.more » « less
-
The premise of identifiable and causal representation learning is to improve the current representation learning paradigm in terms of generalizability or robustness. Despite recent progress in questions of identifiability, more theoretical results demonstrating concrete advantages of these methods for downstream tasks are needed. In this paper, we consider the task of intervention extrapolation: predicting how interventions affect an outcome, even when those interventions are not observed at training time, and show that identifiable representations can provide an effective solution to this task even if the interventions affect the outcome non-linearly. Our setup includes an outcome variable Y , observed features X, which are generated as a non-linear transformation of latent features Z, and exogenous action variables A, which influence Z. The objective of intervention extrapolation is then to predict how interventions on A that lie outside the training support of A affect Y . Here, extrapolation becomes possible if the effect of A on Z is linear and the residual when regressing Z on A has full support. As Z is latent, we combine the task of intervention extrapolation with identifiable representation learning, which we call Rep4Ex: we aim to map the observed features X into a subspace that allows for non-linear extrapolation in A. We show that the hidden representation is identifiable up to an affine transformation in Z-space, which, we prove, is sufficient for intervention extrapolation. The identifiability is characterized by a novel constraint describing the linearity assumption of A on Z. Based on this insight, we propose a flexible method that enforces the linear invariance constraint and can be combined with any type of autoencoder. We validate our theoretical findings through a series of synthetic experiments and show that our approach can indeed succeed in predicting the effects of unseen interventions.more » « less
-
Escorpizo, Reuben (Ed.)The rate of adjustment in a movement, driven by feedback error, is referred to as the adaptation rate, and the rate of recovery of a newly adapted movement to its unperturbed condition is called the de-adaptation rate. The rates of adaptation and de-adaptation are dependent on the training mechanism and intrinsic factors such as the participant's sensorimotor abilities. This study investigated the facilitation of the motor adaptation and de-adaptation processes for spatiotemporal features of an asymmetric gait pattern by sequentially applying split-belt treadmill (SBT) and asymmetric rhythmic auditory cueing (ARAC). Methods: Two sessions tested the individual gait characteristics of SBT and ARAC, and the remaining four sessions consisted of applying the two interventions sequentially during training. The adjustment process to the second intervention is referred to as “re-adaptation” and is driven by feedback error associated with the second intervention. Results: Ten healthy individuals participated in the randomized six-session trial. Spatiotemporal asymmetries during the adaptation and post-adaptation (when intervention is removed) stages were fitted into a two-component exponential model that reflects the explicit and implicit adaptation processes. A double component was shown to fit better than a single-component model. The decay constants of the model were indicative of the corresponding timescales and compared between trials. Results revealed that the explicit (fast) component of adaptation to ARAC was reduced for step length and step time when applied after SBT. Contrarily, the explicit component of adaptation to SBT was increased when it was applied after ARAC for step length. Additionally, the implicit (slow) component of adaptation to SBT was inhibited when applied incongruently after ARAC for step time. These outcomes show that the role of working motor memory as a translational tool between different gait interventions is dependent on (i) the adaptation mechanisms associated with the interventions, (ii) the targeted motor outcome of the interventions; the effects of factors (i) and (ii) are specific to the explicit and implicit components of the adaptation processes; these effects are unique to spatial and temporal gait characteristics.more » « less
-
Behavioral science interventions have the potential to address longstanding policy problems, but their effects are typically heterogeneous across contexts (e.g., teachers, schools, and geographic regions). This contextual heterogeneity is poorly understood, however, which reduces the field’s impact and its understanding of mechanisms. Here, we present an efficient way to interrogate heterogeneity and address these gaps in knowledge. This method a) presents scenarios that vividly represent different moderating contexts, b) measures a short-term behavioral outcome (e.g., an academic choice) that is known to relate to typical intervention outcomes (e.g., academic achievement), and c) assesses the causal effect of the moderating context on the link between the psychological variable typically targeted by interventions and this short-term outcome. We illustrated the utility of this approach across four experiments (total n = 3,235) that directly tested contextual moderators of the links between growth mindset, which is the belief that ability can be developed, and students’ academic choices. The present results showed that teachers’ growth mindset-supportive messages and the structural opportunities they provide moderated the link between students’ mindsets and their choices (studies 1 to 3). This pattern was replicated in a nationally representative sample of adolescents and did not vary across demographic subgroups (study 2), nor was this pattern the result of several possible confounds (studies 3 to 4). Discussion centers on how this method of interrogating contextual heterogeneity can be applied to other behavioral science interventions and broaden their impact in other policy domains.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1515/scid-2020-0005
