skip to main content

Title: Inconvenient Samples: Modeling Biases Related to Parental Consent by Coupling Observational and Experimental Results
In studies involving human subjects, voluntary participation may lead to sampling bias, thus limiting the generalizability of findings. This effect may be especially pronounced in developmental studies, where parents serve as both the primary environmental input and decision maker of whether their child participates in a study. We present a novel empirical and modeling approach to estimate how parental consent may bias measurements of children’s behavior. Specifically, we coupled naturalistic observations of parent–child interactions in public spaces with a behavioral test with children, and used modeling methods to impute the behavior of children who did not participate. Results showed that parents’ tendency to use questions to teach was associated with both children’s behavior in the test and parents’ tendency to participate. Exploiting these associations with a model-based multiple imputation and a propensity score–matching procedure, we estimated that the means of the participating and not-participating groups could differ as much as 0.23 standard deviations for the test measurements, and standard deviations themselves are likely underestimated. These results suggest that ignoring factors associated with consent may lead to systematic biases when generalizing beyond lab samples, and the proposed general approach provides a way to estimate these biases in future research.  more » « less
Award ID(s):
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Open Mind
Page Range / eLocation ID:
13 to 24
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Previous research has documented the benefits of making for young learners, but few studies have examined how parents engage in maker activities during family visits to museums, both as facilitators of their children’s learning and as makers in their own right. In this study, we asked how caregivers participate in making and tinkering programs, how parents describe the benefits of making (for their children and themselves), and what aspects of the physical and social setting influence parents’ engagement. Data included observations of 88 family groups participating in various making and tinkering activities at a science center (including woodworking, fashion design, virtual reality drawing, circuit blocks, etc) and exit interviews with a subset of 66 caregivers. Qualitative data analysis connected observed qualities of the physical and social setting with caregivers’ observed and reported engagement. Through this analysis, we identified specific aspects of the physical environment, tools/materials, and facilitation strategies that invited family participation in general and that were associated with specific caregiver roles, including observing children’s learning, facilitation of children’s learning, and engagement as a maker alongside children. The implications of the findings for the design and facilitation of maker programs are discussed. 
    more » « less
  2. Research on social, emotional, and academic development of children often notes the critical role of parents. Yet, how parents perceive and engage with children’s reactions to difficulty and perceived failure, to then shape their perspective and engagement with learning remains under investigated. The current study explored children and parents’ perceptions of and reactions to frustration and failure within an out-of-school, home-based engineering program. Specifically, we asked 1) How was failure perceived by participating families? and 2) What was the subsequent action/reaction to that failure? Data were derived from post-program interviews with children and parents who participated in a home-based, elementary engineering program involving take-home kits and self-identified engineering projects. Findings derived from descriptive qualitative methods and thematic analysis illustrated development of parent thinking around failure and frustration, both within themselves and their reactions to seeing such emotions in their children. Analysis further revealed how such emotions emerge within their children and impact their experiences. These findings shed light on ways child-parent engagement and the tactics employed by parents may influence a child’s perseverance and willingness to work through difficulty. This research represents an entry point for investigating how parents perceive and react to failures and challenges, and how these reactions shape their communication around failure with their children. Such parental reactions and communication may shape children’s mindset development, perspectives, and engagement. Implications for family engagement and influence on children’s learning through academic emotions in STEM and engineering are discussed. 
    more » « less
  3. Abstract Objective

    Collisions between bicycles and motor vehicles are one of the leading risk factors for injury and death in childhood and adolescence. We examined longitudinal and concurrent effortful control (EC) as predictors of risky bicycling behavior in early- to mid-adolescence, with age and gender as moderators. We also examined whether EC was associated with parent-reported real-world bicycling behavior and all lifetime unintentional injuries.


    Parent-reported EC measures were collected when children (N = 85) were 4 years old and when they were either 10 years (N = 42) or 15 years (N = 43) old. We assessed risky bicycling behavior by asking the adolescents to bicycle across roads with high-density traffic in an immersive virtual environment. Parents also reported on children’s real-world bicycling behavior and lifetime unintentional injuries at the time of the bicycling session.


    We found that both longitudinal and concurrent EC predicted adolescents’ gap choices, though these effects were moderated by age and gender. Lower parent-reported early EC in younger and older girls predicted a greater willingness to take tight gaps (3.5 s). Lower parent-reported concurrent EC in older boys predicted a greater willingness to take gaps of any size. Children lower in early EC started bicycling earlier and were rated as less cautious bicyclists as adolescents. Adolescents lower in concurrent EC were also rated as less cautious bicyclists and had experienced more lifetime unintentional injuries requiring medical attention.


    Early measures of child temperament may help to identify at-risk populations who may benefit from parent-based interventions.

    more » « less
  4. Abstract

    The present research examined children’s anger proneness, emotion understanding, and maternal sensitivity during toddlerhood as predictors of children’s hostile attribution bias (HAB) during the later preschool years. At 2.8 years (N = 128), maternal sensitivity (e.g., child‐centered behavior) was observed during mother–child play and snack, and parents reported on children’s anger proneness. At 3.3 years, emotion understanding (i.e., ability to identify emotional expressions accurately) was measured via an interactive puppet interview. At 4.8 and 5.4 years, children's HAB was assessed via child responses to hypothetical vignettes of ambiguous peer provocations. Path models revealed that maternal sensitivity predicted fewer hostile attributions. In addition, emotion understanding and maternal sensitivity emerged as buffers against the negative effect of anger proneness on HAB. Specifically, greater anger proneness was associated with more frequent hostile attributions, but only when children had lower emotion understanding or had mothers who were less sensitive. The findings highlight the interplay between intrapersonal and interpersonal factors in early childhood that contribute to a hostile attribution bias during the preschool period.

    more » « less
  5. The COVID-19 pandemic has dramatically altered family life in the United States. Over the long duration of the pandemic, parents had to adapt to shifting work conditions, virtual schooling, the closure of daycare facilities, and the stress of not only managing households without domestic and care supports but also worrying that family members may contract the novel coronavirus. Reports early in the pandemic suggest that these burdens have fallen disproportionately on mothers, creating concerns about the long-term implications of the pandemic for gender inequality and mothers’ well-being. Nevertheless, less is known about how parents’ engagement in domestic labor and paid work has changed throughout the pandemic, what factors may be driving these changes, and what the long-term consequences of the pandemic may be for the gendered division of labor and gender inequality more generally.

    The Study on U.S. Parents’ Divisions of Labor During COVID-19 (SPDLC) collects longitudinal survey data from partnered U.S. parents that can be used to assess changes in parents’ divisions of domestic labor, divisions of paid labor, and well-being throughout and after the COVID-19 pandemic. The goal of SPDLC is to understand both the short- and long-term impacts of the pandemic for the gendered division of labor, work-family issues, and broader patterns of gender inequality.

    Survey data for this study is collected using Prolifc (, an opt-in online platform designed to facilitate scientific research. The sample is comprised U.S. adults who were residing with a romantic partner and at least one biological child (at the time of entry into the study). In each survey, parents answer questions about both themselves and their partners. Wave 1 of SPDLC was conducted in April 2020, and parents who participated in Wave 1 were asked about their division of labor both prior to (i.e., early March 2020) and one month after the pandemic began. Wave 2 of SPDLC was collected in November 2020. Parents who participated in Wave 1 were invited to participate again in Wave 2, and a new cohort of parents was also recruited to participate in the Wave 2 survey. Wave 3 of SPDLC was collected in October 2021. Parents who participated in either of the first two waves were invited to participate again in Wave 3, and another new cohort of parents was also recruited to participate in the Wave 3 survey. This research design (follow-up survey of panelists and new cross-section of parents at each wave) will continue through 2024, culminating in six waves of data spanning the period from March 2020 through October 2024. An estimated total of approximately 6,500 parents will be surveyed at least once throughout the duration of the study.

    SPDLC data will be released to the public two years after data is collected; Waves 1 and 2 are currently publicly available. Wave 3 will be publicly available in October 2023, with subsequent waves becoming available yearly. Data will be available to download in both SPSS (.sav) and Stata (.dta) formats, and the following data files will be available: (1) a data file for each individual wave, which contains responses from all participants in that wave of data collection, (2) a longitudinal panel data file, which contains longitudinal follow-up data from all available waves, and (3) a repeated cross-section data file, which contains the repeated cross-section data (from new respondents at each wave) from all available waves. Codebooks for each survey wave and a detailed user guide describing the data are also available. Response Rates: Of the 1,157 parents who participated in Wave 1, 828 (72%) also participated in the Wave 2 study. Presence of Common Scales: The following established scales are included in the survey:
    • Self-Efficacy, adapted from Pearlin's mastery scale (Pearlin et al., 1981) and the Rosenberg self-esteem scale (Rosenberg, 2015) and taken from the American Changing Lives Survey
    • Communication with Partner, taken from the Marriage and Relationship Survey (Lichter & Carmalt, 2009)
    • Gender Attitudes, taken from the National Survey of Families and Households (Sweet & Bumpass, 1996)
    • Depressive Symptoms (CES-D-10)
    • Stress, measured using Cohen's Perceived Stress Scale (Cohen, Kamarck, & Mermelstein, 1983)
    Full details about these scales and all other items included in the survey can be found in the user guide and codebook
    The second wave of the SPDLC was fielded in November 2020 in two stages. In the first stage, all parents who participated in W1 of the SPDLC and who continued to reside in the United States were re-contacted and asked to participate in a follow-up survey. The W2 survey was posted on Prolific, and messages were sent via Prolific’s messaging system to all previous participants. Multiple follow-up messages were sent in an attempt to increase response rates to the follow-up survey. Of the 1,157 respondents who completed the W1 survey, 873 at least started the W2 survey. Data quality checks were employed in line with best practices for online surveys (e.g., removing respondents who did not complete most of the survey or who did not pass the attention filters). After data quality checks, 5.2% of respondents were removed from the sample, resulting in a final sample size of 828 parents (a response rate of 72%).

    In the second stage, a new sample of parents was recruited. New parents had to meet the same sampling criteria as in W1 (be at least 18 years old, reside in the United States, reside with a romantic partner, and be a parent living with at least one biological child). Also similar to the W1 procedures, we oversampled men, Black individuals, individuals who did not complete college, and individuals who identified as politically conservative to increase sample diversity. A total of 1,207 parents participated in the W2 survey. Data quality checks led to the removal of 5.7% of the respondents, resulting in a final sample size of new respondents at Wave 2 of 1,138 parents.

    In both stages, participants were informed that the survey would take approximately 20 minutes to complete. All panelists were provided monetary compensation in line with Prolific’s compensation guidelines, which require that all participants earn above minimum wage for their time participating in studies.
    To be included in SPDLC, respondents had to meet the following sampling criteria at the time they enter the study: (a) be at least 18 years old, (b) reside in the United States, (c) reside with a romantic partner (i.e., be married or cohabiting), and (d) be a parent living with at least one biological child. Follow-up respondents must be at least 18 years old and reside in the United States, but may experience changes in relationship and resident parent statuses. Smallest Geographic Unit: U.S. State

    This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit In accordance with this license, all users of these data must give appropriate credit to the authors in any papers, presentations, books, or other works that use the data. A suggested citation to provide attribution for these data is included below:            

    Carlson, Daniel L. and Richard J. Petts. 2022. Study on U.S. Parents’ Divisions of Labor During COVID-19 User Guide: Waves 1-2.  

    To help provide estimates that are more representative of U.S. partnered parents, the SPDLC includes sampling weights. Weights can be included in statistical analyses to make estimates from the SPDLC sample representative of U.S. parents who reside with a romantic partner (married or cohabiting) and a child aged 18 or younger based on age, race/ethnicity, and gender. National estimates for the age, racial/ethnic, and gender profile of U.S. partnered parents were obtained using data from the 2020 Current Population Survey (CPS). Weights were calculated using an iterative raking method, such that the full sample in each data file matches the nationally representative CPS data in regard to the gender, age, and racial/ethnic distributions within the data. This variable is labeled CPSweightW2 in the Wave 2 dataset, and CPSweightLW2 in the longitudinal dataset (which includes Waves 1 and 2). There is not a weight variable included in the W1-W2 repeated cross-section data file.
    more » « less