skip to main content


Title: Evaluating the power of the causal impact method in observational studies of HCV treatment as prevention
Objectives: The causal impact method (CIM) was recently introduced for evaluation of binary interventions using observational time-series data. The CIM is appealing for practical use as it can adjust for temporal trends and account for the potential of unobserved confounding. However, the method was initially developed for applications involving large datasets and hence its potential in small epidemiological studies is still unclear. Further, the effects that measurement error can have on the performance of the CIM have not been studied yet. The objective of this work is to investigate both of these open problems. Methods: Motivated by an existing dataset of HCV surveillance in the UK, we perform simulation experiments to investigate the effect of several characteristics of the data on the performance of the CIM and extend the method to deal with this problem. Results: We identify multiple characteristics of the data that affect the ability of the CIM to detect an intervention effect including the length of time-series, the variability of the outcome and the degree of correlation between the outcome of the treated unit and the outcomes of controls. We show that measurement error can introduce biases in the estimated intervention effects and heavily reduce the power of the CIM. Using an extended CIM, some of these adverse effects can be mitigated. Conclusions: The CIM can provide satisfactory power in public health interventions. The method may provide misleading results in the presence of measurement error.  more » « less
Award ID(s):
1854934
NSF-PAR ID:
10357267
Author(s) / Creator(s):
; ; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Statistical communications in infectious diseases
Volume:
30
Issue:
1
ISSN:
1948-4690
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Objective: We present a literature review on workplace physical fatigue interventions, focusing on evaluating the methodological quality and strength of evidence. Background: Physical fatigue is a recognized workplace problem, with negative effects on performance and health-related complaints. Although many studies have focused on the mechanisms and consequences of fatigue, few have considered the effectiveness of interventions to mitigate fatigue. Method: A systematic review of the workplace safety literature for controlled trials of physical fatigue interventions was conducted. Data on intervention type, subject characteristics, targeted tasks and body locations, outcome measures, and study design were extracted. The methodological quality for each study was evaluated using the PEDro scale, and the level of evidence was based on quality, amount, and consistency. Results: Forty-five controlled trials were reviewed, examining 18 interventions. We categorized those interventions into individual-focused ( N = 28 studies, nine interventions), workplace-focused ( N = 12 studies, five interventions), and multiple interventions ( N = 5 studies, four interventions). We identified moderate evidence for interventions related to assistive devices and task variation. There was moderate evidence supporting no fatigue attenuation for the garment change category of interventions. The interventions in the remaining categories had limited to minimal evidence of efficacy. The heterogeneity of the included trials precludes the determination of effect size. Conclusion: This review showed a lack of high levels of evidence for the effectiveness of most physical fatigue interventions. Application: Due to a lack of high levels of evidence for any category of reviewed physical fatigue interventions, further high-quality studies are needed to establish the efficacy of others. 
    more » « less
  2. Behavioral science interventions have the potential to address longstanding policy problems, but their effects are typically heterogeneous across contexts (e.g., teachers, schools, and geographic regions). This contextual heterogeneity is poorly understood, however, which reduces the field’s impact and its understanding of mechanisms. Here, we present an efficient way to interrogate heterogeneity and address these gaps in knowledge. This method a) presents scenarios that vividly represent different moderating contexts, b) measures a short-term behavioral outcome (e.g., an academic choice) that is known to relate to typical intervention outcomes (e.g., academic achievement), and c) assesses the causal effect of the moderating context on the link between the psychological variable typically targeted by interventions and this short-term outcome. We illustrated the utility of this approach across four experiments (total n = 3,235) that directly tested contextual moderators of the links between growth mindset, which is the belief that ability can be developed, and students’ academic choices. The present results showed that teachers’ growth mindset-supportive messages and the structural opportunities they provide moderated the link between students’ mindsets and their choices (studies 1 to 3). This pattern was replicated in a nationally representative sample of adolescents and did not vary across demographic subgroups (study 2), nor was this pattern the result of several possible confounds (studies 3 to 4). Discussion centers on how this method of interrogating contextual heterogeneity can be applied to other behavioral science interventions and broaden their impact in other policy domains. 
    more » « less
  3. null (Ed.)
    Purpose This study synthesized effects of interventions on language outcomes of young children (ages 0–8 years) with autism and evaluated the extent to which summary effects varied by intervention, participant, and outcome characteristics. Method A subset of effect sizes gathered for a larger meta-analysis (the Autism Intervention Meta-analysis or Project AIM) examining the effects of interventions for young children with autism, which were specific to language outcomes, was analyzed. Robust variance estimation and metaregression were used to calculate summary and moderated effects while controlling for intercorrelation among outcomes within studies. Results A total of 221 outcomes were gathered from 60 studies. The summary effect of intervention on language outcomes was small but significant. Summary effects were larger for expressive and composite language outcomes compared to receptive language outcomes. Interventions implemented by clinicians, or by clinicians and caregivers together, had summary effects that were significantly larger than interventions implemented by caregivers alone. Participants' pretreatment language age equivalent scores positively and significantly moderated intervention effects, such that effects were significantly larger on average when samples of children had higher pretreatment language levels. Effects were not moderated by cumulative intervention intensity, intervention type, autism symptomatology, chronological age, or the proximity or boundedness of outcomes. Study quality concerns were apparent for a majority of included outcomes. Conclusions We found evidence that intervention can facilitate improvements in language outcomes for young children with autism. Effects were largest for expressive and composite language outcomes, for children with initially higher language abilities, and for interventions implemented by clinicians or by caregivers and clinicians combined. However, quality concerns of included studies and borderline significance of some results temper our conclusions regarding intervention effectiveness and corresponding moderators. 
    more » « less
  4. The statistical practice of modeling interaction with two linear main effects and a product term is ubiquitous in the statistical and epidemiological literature. Most data modelers are aware that the misspecification of main effects can potentially cause severe type I error inflation in tests for interactions, leading to spurious detection of interactions. However, modeling practice has not changed. In this article, we focus on the specific situation where the main effects in the model are misspecified as linear terms and characterize its impact on common tests for statistical interaction. We then propose some simple alternatives that fix the issue of potential type I error inflation in testing interaction due to main effect misspecification. We show that when using the sandwich variance estimator for a linear regression model with a quantitative outcome and two independent factors, both the Wald and score tests asymptotically maintain the correct type I error rate. However, if the independence assumption does not hold or the outcome is binary, using the sandwich estimator does not fix the problem. We further demonstrate that flexibly modeling the main effect under a generalized additive model can largely reduce or often remove bias in the estimates and maintain the correct type I error rate for both quantitative and binary outcomes regardless of the independence assumption. We show, under the independence assumption and for a continuous outcome, overfitting and flexibly modeling the main effects does not lead to power loss asymptotically relative to a correctly specified main effect model. Our simulation study further demonstrates the empirical fact that using flexible models for the main effects does not result in a significant loss of power for testing interaction in general. Our results provide an improved understanding of the strengths and limitations for tests of interaction in the presence of main effect misspecification. Using data from a large biobank study “The Michigan Genomics Initiative”, we present two examples of interaction analysis in support of our results.

     
    more » « less
  5. Introduction: Back pain is one of the most common causes of pain in the United States. Spinal cord stimulation (SCS) is an intervention for patients with chronic back pain (CBP). However, SCS decreases pain in only 58% of patients and relies on self-reported pain scores as outcome measures. An SCS trial is temporarily implanted for seven days and helps to determine if a permanent SCS is needed. Patients that have a >50% reduction in pain from the trial stimulator makes them eligible for permanent implantation. However, self-reported measures reveal little on how mechanisms in the brain are altered. Other measurements of pain intensity, onset, medication, disabilities, depression, and anxiety have been used with machine learning to predict outcomes with accuracies <70%. We aim to predict long-term SCS responders at 6-months using baseline resting EEG and machine learning. Materials and Methods: We obtained 10-minutes of resting electroencephalography (EEG) and pain questionnaires from nine participants with CBP at two time points: 1) pre-trial baseline. 2) Six months after SCS permanent implant surgery. Subjects were designated as high or moderate responders based on the amount of pain relief provided by the long-term (post six months) SCS, and pain scored on a scale of 0-10 with 0 being no pain and 10 intolerable. We used the resting EEG from baseline to predict long-term treatment outcome. Resting EEG data was fed through a pipeline for classification and to map dipole sources. EEG signals were preprocessed using the EEGLAB toolbox. Independent component analysis and dipole fitting were used to linearly unmix the signal and to map dipole sources from the brain. Spectral analysis was performed to obtain the frequency distribution of the signal. Each power band, delta (1-4 Hz), theta (4-8 Hz), alpha (8-13 Hz), beta (13-30 Hz), and gamma (30-100 Hz), as well as the entire spectrum (1-100 Hz), were used for classification. Furthermore, dipole sources were ranked based on classification feature weights to determine the significance of specific regions in the brain. We used support vector machines to predict pain outcomes. Results and Discussion: We found higher frequency powerbands provide overall classification performance of 88.89%. Differences in power are seen between moderate and high responders in both the frontal and parietal regions for theta, alpha, beta, and the entire spectrum (Fig.1). This can potentially be used to predict patient response to SCS. Conclusions: We found evidence of decreased power in theta, alpha, beta, and entire spectrum in the anterior regions of the parietal cortex and posterior regions of the frontal cortex between moderate and high responders, which can be used for predicting treatment outcomes in long-term pain relief from SCS. Long-term treatment outcome prediction using baseline EEG data has the potential to contribute to decision making in terms of permanent surgery, forgo trial periods, and improve clinical efficiency by beginning to understand the mechanism of action of SCS in the human brain. 
    more » « less