More Like this

1. Transitioning to a Mixed-Mode Study Design in a National Household Panel Study: Effects on Fieldwork Outcomes, Sample Composition and Costs

   https://doi.org/10.18148/srm/2023.v17i4.8172  

   McGonagle, Katherine A; Sastry, Narayan (December 2023, Survey research methods)

   The U.S. Panel Study of Income Dynamics (PSID) made a planned transition to a web-first mixed-mode data collection design in 2021 (web and computer-assisted telephone interviewing [CATI]), following nearly five decades of collecting data primarily using CATI with professional interviewers. To evaluate potential effects of mode on fieldwork outcomes, two sequential mixed-mode protocols were introduced using an experimental design. One protocol randomized sample families to a “web-first” treatment, which encouraged response through an online interview, followed by an offer of telephone to complete the interview; a second protocol randomized sample families to a “CATI-first” treatment until the last phase of fieldwork when the option to complete a web interview was offered. This paper examines the comparative effects of the two protocols on fieldwork outcomes, including response rates, interviewer contact attempts, fieldwork duration, and cost. Comparisons are also made with fieldwork outcomes and characteristics of non-responding sample members from the prior-wave when a traditional telephone design was used. We found that the web-first design compared to the CATI-first design led to comparably high response rates, and faster interview completion with lower effort and cost. With some notable exceptions, compared to the prior wave, the mixed-mode design reduced effort and had generally similar patterns of non-response among key respondent subgroups. The results provide new empirical evidence on the effects of mixing modes on fieldwork outcomes and costs and contribute to the small body of experimental evidence on the use of mixed-mode designs in household panel studies. 

   more » « less
2. An Experimental Evaluation of an Online Interview Scheduler: Effects on Fieldwork Outcomes

   https://doi.org/10.1093/jssam/smaa031  

   McGonagle, Katherine; Sastry, Narayan (October 2020, Journal of Survey Statistics and Methodology)

   Abstract In recent years, household surveys have expended significant effort to counter well-documented increases in direct refusals and greater difficulty contacting survey respondents. A substantial amount of fieldwork effort in panel surveys using telephone interviewing is devoted to the task of contacting the respondent to schedule the day and time of the interview. Higher fieldwork effort leads to greater costs and is associated with lower response rates. A new approach was experimentally evaluated in the 2017 wave of the Panel Study of Income Dynamics (PSID) Transition into Adulthood Supplement (TAS) that allowed a randomly selected subset of respondents to choose their own day and time of their telephone interview through the use of an online appointment scheduler. TAS is a nationally representative study of US young adults aged 18–28 years embedded within the worlds’ longest running panel study, the PSID. This paper experimentally evaluates the effect of offering the online appointment scheduler on fieldwork outcomes, including number of interviewer contact attempts and interview sessions, number of days to complete the interview, and response rates. We describe panel study members’ characteristics associated with uptake of the online scheduler and examine differences in the effectiveness of the treatment across subgroups. Finally, potential cost-savings of fieldwork effort due to the online appointment scheduler are evaluated. 

   more » « less
3. Meeting Data Collection Goals Quicker: An Experimental Evaluation to Reduce Fieldwork Duration in a Mixed-Mode Panel Study

   https://doi.org/10.1093/jssam/smaf030  

   McGonagle, Katherine A; Sastry, Narayan (January 2026, Journal of Survey Statistics and Methodology)

   Abstract An experiment was implemented in the 2023 wave of a US household panel study to assess the effects of a shortened field period on data collection outcomes. Following the recent adoption of sequential mixed-mode designs by panel studies worldwide, it has been observed that interview completion for respondents offered the initial mode of web is faster compared to those initially offered the telephone. This study describes an experiment designed to evaluate whether the new mixed-mode designs can support an accelerated field period and achieve cost savings while still meeting fieldwork goals. We assessed a shorter field period of 20 weeks against the standard 28-week field period and randomized study participants to each condition. The treatment group received accelerated fieldwork protocols over a 20-week data collection period, and a control group received the same protocols over the standard 28-week period. We compare the effect of the shortened duration on fieldwork outcomes, including response rates, sample composition, interviewer effort, time to interview completion, survey costs, and interview quality. We find that the accelerated protocol yields higher response rates, lower interviewer effort, and cost savings with no differences in sample composition or decrements to interview quality. We describe the strengths and limitations of the study and provide suggestions for future research on fieldwork duration. 

   more » « less
4. Effects of the COVID-19 crisis on survey fieldwork: Experience and lessons from two major supplements to the U.S. Panel Study of Income Dynamics

   https://doi.org/10.18148/srm/2020.v14i2.7752  

   Sastry, N.; McGonagle, K.A.; Fomby, P. (January 2020, Survey research methods)

   Two major supplements to the Panel Study of Income Dynamics (PSID) were in the field during the COVID-19 outbreak in the United States: the 2019 waves of the PSID Child Development Supplement (CDS-19) and the PSID Transition into Adulthood Supplement (TAS-19). Both CDS-19 and TAS-19 abruptly terminated all face-to-face fieldwork and, for TAS-19, shifted interviewers from working in a centralized call center to working from their homes. Overall, COVID-19 had a net negative effect on response rates in CDS-19 and terminated all home visits that represented an important study component. For TAS-19, the overall effect of Covid-19 was uncertain, but negative. The costs were high of adapting to COVID-19 and providing paid time-off benefits to staff affected by the pandemic. Longitudinal surveys, such as CDS, TAS, and PSID, that span the pandemic will provide valuable information on its life course and intergenerational consequences, making ongoing data collection of vital importance. 

   more » « less
5. A randomized experiment evaluating survey mode effects for video interviewing

   https://doi.org/10.1017/psrm.2022.30  

   Endres, Kyle; Hillygus, D. Sunshine; DeBell, Matthew; Iyengar, Shanto (January 2023, Political Science Research and Methods)

   Abstract Rising costs and challenges of in-person interviewing have prompted major surveys to consider moving online and conducting live web-based video interviews. In this paper, we evaluate video mode effects using a two-wave experimental design in which respondents were randomized to either an interviewer-administered video or interviewer-administered in-person survey waveaftercompleting a self-administered online survey wave. This design permits testing of both within- and between-subject differences across survey modes. Our findings suggest that video interviewing is more comparable to in-person interviewing than online interviewing across multiple measures of satisficing, social desirability, and respondent satisfaction. 

   more » « less