More Like this

1. Does Survey Nonresponse Bias Estimates of Religious Service Attendance? Evidence from an Address-Based Sample from the Boston Area

   https://doi.org/10.1093/socrel/sry031  

   Brenner, Philip S (August 2018, Sociology of Religion)

   null (Ed.)

   Abstract This study investigates what role, if any, nonresponse plays in inflating survey estimates of religious behavior, using a multimode survey designed to allow estimation of nonresponse bias. A sample of 3,000 Boston-area households drawn from an address-based frame was randomly divided into two subsamples, contacted by mail, and invited to participate in a survey. The first subsample was asked to complete an interactive voice response interview. The second subsample was asked to complete a survey by telephone if a number was available for the address or by personal interview if not. Finally, random samples of nonrespondents were recontacted for a personal interview. Comparison of attendance estimates from initial interviews with nonrespondent interviews within sample segments yields minor or minimal differences that are not statistically significant. Findings suggest that the mechanism generating survey nonresponse is unlikely to be a major cause of bias in religious service attendance estimates in this study. 

   more » « less
2. The benefits of in situ reporting: Experimental evidence from in-the-moment surveys of public transit riders

   https://doi.org/10.3758/s13428-026-03022-z  

   Antoun, Christopher; Frías-Martínez, Vanessa; Garove, Anthony; Awasthi, Naman; Abrar, Saad Mohammad (May 2026, Behavior Research Methods)

   Abstract In-the-moment surveys administered when respondents participate in a predetermined event of interest—such as visiting a particular location or completing a specific activity—allow researchers to measure behaviors and experiences as the event occurs, situating data collection within the relevant context and reducing reliance on retrospective self-reports. However, data quality can be compromised if individuals are unwilling to participate or provide low-quality responses. In particular, the timing of survey prompts and the structure of incentives may affect data quality. We evaluated these issues in an experimental study of 610 individuals who used a custom smartphone app to answer a survey about each of their public transit trips over 2 weeks. Participants were randomly assigned to receive either daily or per-survey incentives, with surveys administered either during trips (in situ) or immediately afterward (near-time). Both daily incentives and in situ prompts significantly increased participation. In situ prompts also reduced “speeding” and increased the variation in survey responses across trips, suggesting improved accuracy in capturing event-specific details. These findings demonstrate that prompt timing—specifically, in situ prompts as opposed to near-time prompts—plays a key role in enhancing data quality for in-the-moment surveys. 

   more » « less
3. Improving Survey Inference Using Administrative Records Without Releasing Individual‐Level Continuous Data

   https://doi.org/10.1002/sim.10270  

   Williams, Sharifa Z; Zou, Jungang; Liu, Yutao; Si, Yajuan; Galea, Sandro; Chen, Qixuan (December 2024, Statistics in Medicine)

   ABSTRACT Probability surveys are challenged by increasing nonresponse rates, resulting in biased statistical inference. Auxiliary information about populations can be used to reduce bias in estimation. Often continuous auxiliary variables in administrative records are first discretized before releasing to the public to avoid confidentiality breaches. This may weaken the utility of the administrative records in improving survey estimates, particularly when there is a strong relationship between continuous auxiliary information and the survey outcome. In this paper, we propose a two‐step strategy, where the confidential continuous auxiliary data in the population are first utilized to estimate the response propensity score of the survey sample by statistical agencies, which is then included in a modified population data for data users. In the second step, data users who do not have access to confidential continuous auxiliary data conduct predictive survey inference by including discretized continuous variables and the propensity score as predictors using splines in a Bayesian model. We show by simulation that the proposed method performs well, yielding more efficient estimates of population means with 95% credible intervals providing better coverage than alternative approaches. We illustrate the proposed method using the Ohio Army National Guard Mental Health Initiative (OHARNG‐MHI). The methods developed in this work are readily available in the R packageAuxSurvey. 

   more » « less
4. Comparison of Social Media, Syndromic Surveillance, and Microbiologic Acute Respiratory Infection Data: Observational Study

   https://doi.org/10.2196/14986  

   Daughton, Ashlynn R; Chunara, Rumi; Paul, Michael J (January 2020, JMIR Public Health and Surveillance)

   Background Internet data can be used to improve infectious disease models. However, the representativeness and individual-level validity of internet-derived measures are largely unexplored as this requires ground truth data for study. Objective This study sought to identify relationships between Web-based behaviors and/or conversation topics and health status using a ground truth, survey-based dataset. Methods This study leveraged a unique dataset of self-reported surveys, microbiological laboratory tests, and social media data from the same individuals toward understanding the validity of individual-level constructs pertaining to influenza-like illness in social media data. Logistic regression models were used to identify illness in Twitter posts using user posting behaviors and topic model features extracted from users’ tweets. Results Of 396 original study participants, only 81 met the inclusion criteria for this study. Of these participants’ tweets, we identified only two instances that were related to health and occurred within 2 weeks (before or after) of a survey indicating symptoms. It was not possible to predict when participants reported symptoms using features derived from topic models (area under the curve [AUC]=0.51; P=.38), though it was possible using behavior features, albeit with a very small effect size (AUC=0.53; P≤.001). Individual symptoms were also generally not predictable either. The study sample and a random sample from Twitter are predictably different on held-out data (AUC=0.67; P≤.001), meaning that the content posted by people who participated in this study was predictably different from that posted by random Twitter users. Individuals in the random sample and the GoViral sample used Twitter with similar frequencies (similar @ mentions, number of tweets, and number of retweets; AUC=0.50; P=.19). Conclusions To our knowledge, this is the first instance of an attempt to use a ground truth dataset to validate infectious disease observations in social media data. The lack of signal, the lack of predictability among behaviors or topics, and the demonstrated volunteer bias in the study population are important findings for the large and growing body of disease surveillance using internet-sourced data. 

   more » « less
5. The Effects of Nonresponse and Sampling Omissions on Estimates on Various Topics in Federal Surveys: Telephone and IVR Surveys of Address-Based Samples

   https://doi.org/10.2478/jos-2020-0032  

   Fowler, Floyd J.; Brenner, Philip; Roman, Anthony M.; Hargraves, J. Lee (July 2020, Journal of Official Statistics)

   null (Ed.)

   Abstract With declining response rates and challenges of using RDD sampling for telephone surveys, collecting data from address-based samples has become more attractive. Two approaches are doing telephone interviews at telephone numbers matched to addresses and asking those at sampled addresses to call into an Interactive Voice Response (IVR) system to answer questions. This study used in-person interviewing to evaluate the effects of nonresponse and problems matching telephone numbers when telephone and IVR were used as the initial modes of data collection. The survey questions were selected from major US federal surveys covering a variety of topics. Both nonresponse and, for telephone, inability to find matches result in important nonresponse error for nearly half the measures across all topics, even after adjustments to fit the known demographic characteristics of the residents. Producing credible estimates requires using supplemental data collection strategies to reduce error from nonresponse. 

   more » « less