skip to main content

Explore Research Products in the NSF-PAR

Title: Regularization for Shuffled Data Problems via Exponential Family Priors on the Permutation Group
In the analysis of data sets consisting of (X, Y)-pairs, a tacit assumption is that each pair corresponds to the same observational unit. If, however, such pairs are obtained via record linkage of two files, this assumption can be violated as a result of mismatch error rooting, for example, in the lack of reliable identifiers in the two files. Recently, there has been a surge of interest in this setting under the term “Shuffled Data” in which the underlying correct pairing of (X, Y)-pairs is represented via an unknown permutation. Explicit modeling of the permutation tends to be associated with overfitting, prompting the need for suitable methods of regularization. In this paper, we propose an exponential family prior on the permutation group for this purpose that can be used to integrate various structures such as sparse and local shuffling. This prior turns out to be conjugate for canonical shuffled data problems in which the likelihood conditional on a fixed permutation can be expressed as product over the corresponding (X,Y)-pairs. Inference can be based on the EM algorithm in which the E-step is approximated by sampling, e.g., via the Fisher-Yates algorithm. The M-step is shown to admit a reduction from n^2 to n terms if the likelihood of (X,Y)-pairs has exponential family form. Comparisons on synthetic and real data show that the proposed approach compares favorably to competing methods.  more » « less
Award ID(s):
2120318
NSF-PAR ID:
10446725
Author(s) / Creator(s):
; ;
Editor(s):
Ruiz, Francisco; Dy, Jennifer; van de Meent, Jan-Willem
Date Published:
Journal Name:
Proceedings of Machine Learning Research
Volume:
206
ISSN:
2640-3498
Page Range / eLocation ID:
2939-2959
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ( , Zenodo)

    This dataset includes multiple fields: (i) files for monthly and annual fields for the max curl line and the zero curl line at 0.1 degree longitudinal resolutions; (ii) files for monthly and annual GS path obtained from Altimetry and originally processed by Andres (2016) at 0.1 degree longitudinal resolution. The maximum curl line (MCL) and the zero curl line (ZCL) calculations are briefly described here and are based on the original wind data (at 1.25 x 1.25 degree) provided by the Japanese reanalysis (JRA-55; Kobayashi et al., 2015) and available at https://zenodo.org/record/8200832 (Gifford et al. 2023). For details see Gifford, 2023. 

    The wind stress curl (WSC) fields used for the MCL and ZCL calculations extend from 80W to 45W and 30N to 45N at the 1.25 by 1.25-degree resolution.  The MCL is defined as the maximum WSC values greater than zero within the domain per 1.25 degree longitude. As such, it is a function of longitude and is not a constant WSC value unlike the zero contour. High wind stress curl values that occurred near the coast were not included within this calculation. After MCL at the 1.25 resolution was obtained the line was smoothed with a gaussian smoothing and interpolated on to a 0.1 longitudinal resolution. The smoothed MCL lines at 0.1 degree resolution are provided in separate files for monthly and annual averages (2 files). Similarly, 2 other files (monthly and annual) are provided for the ZCL.    

    Like the MCL, the ZCL is a line derived from 1.25 degree longitude throughout the domain under the condition that it's the line of zero WSC. The ZCL is constant at 0 and does not vary spatially like the MCL. If there are more than one location of zero curl for a given longitude the first location south of the MCL is selected. Similar to the MCL, the ZCL was smoothed with a gaussian smoothing and interpolated on to a 0.1 longitudinal resolution.   

    The above files span the years from 1980 through 2019. So, the monthly files have 480 months starting January 1980, and the annual files have 40 years of data. The files are organized with each row being a new time step and each column being a different longitude. Therefore, the monthly MCL and ZCL files are each 480 x 351 for the 0.1 resolution data. Similarly, the annual files are 40 x 351 for the 0.1 degree resolution data.  

    Note that the monthly MCLs and ZCLs are obtained from the monthly wind-stress curl fields. The annual MCLs and ZCLs are obtained from the annual wind-stress curl fields.

    Since the monthly curl fields preserves more atmospheric mesoscales than the annual curl fields, the 12-month average of the monthly MCLs and ZCLs will not match with the annual MCLs and ZCLs derived from the annual curl field.  The annual MCLs and ZCLs provided here are obtained from the annual curl fields and representative metrics of the wind forcing on an annual time-scale. 

    Furthermore, the monthly Gulf Stream axis path (25 cm isoheight from Altimeter, reprocessed by Andres (2016) technique) from 1993 through 2019 have been made available here. A total of 324 monthly paths of the Gulf Stream are tabulated. In addition, the annual GS paths for these 27 years (1993-2019) of altimetry era have been put together for ease of use. The monthly Gulf Stream paths have been resampled and reprocessed for uniqueness at every 0.1 degree longitude from 75W to 50W and smoothed with a 100 km (10 point) running average via matlab. The uniqueness has been achieved by using Consolidator algorithm (D’Errico, 2023). 

    Each monthly or annual GS path has 251 points between 75W to 50W at 0.1 degree resolution.  

    Please contact igifford@earth.miami.edu for any queries. {"references": ["Andres, M., 2016. On the recent destabilization of the Gulf Stream path downstream of Cape Hatteras. Geophysical Research Letters, 43(18), 9836-9842.", "D'Errico, J., 2023. Consolidator (https://www.mathworks.com/matlabcentral/fileexchange/ 8354-consolidator), MATLAB Central File Exchange. Retrieved June 17, 2023.", "Gifford, Ian. H., 2023. The Synchronicity of the Gulf Stream Free Jet and the Wind Induced Cyclonic Vorticity Pool. MS Thesis, University of Massachusetts Dartmouth. 75pp.", "Gifford, Ian, H., Avijit Gangopadhyay, Magdalena Andres, Glen Gawarkiewicz, Hilde Oliver, Adrienne Silver, 2023. Wind Stress, Wind Stress Curl, and Upwelling Velocities in the Northwest Atlantic (80-45W, 30-45N) during 1980-2019, https://zenodo.org/record/8200832.", "Kobayashi, S., Ota, Y., Harada, Y., Ebita, A., Moriya, M., Onoda, H., Onogi, K., Kamahori, H., Kobayashi, C., Endo, H. and Miyaoka, K., 2015. The JRA-55 reanalysis: General specifications and basic characteristics.\u202fJournal of the Meteorological Society of Japan. Ser. II,\u202f93(1), pp.5-48. Kobayashi, S., Ota, Y., Harada, Y., Ebita, A., Moriya, M., Onoda, H., Onogi, K., Kamahori, H., Kobayashi, C., Endo, H. and Miyaoka, K., 2015. The JRA-55 reanalysis: General specifications and basic characteristics.\u202fJournal of the Meteorological Society of Japan. Ser. II,\u202f93(1), pp.5-48."]} 
    more » « less
  2. ( , Mendeley)
    Data description This dataset presents the raw and augmented data that were used to train the machine learning (ML) models for classification of printing outcome in projection two-photon lithography (P-TPL). P-TPL is an additive manufacturing technique for the fabrication of cm-scale complex 3D structures with features smaller than 200 nm. The P-TPL process is further described in this article: “Saha, S. K., Wang, D., Nguyen, V. H., Chang, Y., Oakdale, J. S., and Chen, S.-C., 2019, "Scalable submicrometer additive manufacturing," Science, 366(6461), pp. 105-109.” This specific dataset refers to the case wherein a set of five line features were projected and the printing outcome was classified into three classes: ‘no printing’, ‘printing’, ‘overprinting’. Each datapoint comprises a set of ten inputs (i.e., attributes) and one output (i.e., target) corresponding to these inputs. The inputs are: optical power (P), polymerization rate constant at the beginning of polymer conversion (kp-0), radical quenching rate constant (kq), termination rate constant at the beginning of polymer conversion (kt-0), number of optical pulses, (N), kp exponential function shape parameter (A), kt exponential function shape parameter (B), quantum yield of photoinitiator (QY), initial photoinitiator concentration (PIo), and the threshold degree of conversion (DOCth). The output variable is ‘Class’ which can take these three values: -1 for the class ‘no printing’, 0 for the class ‘printing’, and 1 for the class ‘overprinting’. The raw data (i.e., the non-augmented data) refers to the data generated from finite element simulations of P-TPL. The augmented data was obtained from the raw data by (1) changing the DOCth and re-processing a solved finite element model or (2) by applying physics-based prior process knowledge. For example, it is known that if a given set of parameters failed to print, then decreasing the parameters that are positively correlated with printing (e.g. kp-0, power), while keeping the other parameters constant would also lead to no printing. Here, positive correlation means that individually increasing the input parameter will lead to an increase in the amount of printing. Similarly, increasing the parameters that are negatively correlated with printing (e.g. kq, kt-0), while keeping the other parameters constant would also lead to no printing. The converse is true for those datapoints that resulted in overprinting. The 'Raw.csv' file contains the datapoints generated from finite element simulations, the 'Augmented.csv' file contains the datapoints generated via augmentation, and the 'Combined.csv' file contains the datapoints from both files. The ML models were trained on the combined dataset that included both raw and augmented data. 
    more » « less
  3. ( , Environmental Data Initiative)
    Site description. This data package consists of data obtained from sampling surface soil (the 0-7.6 cm depth profile) in black mangrove (Avicennia germinans) dominated forest and black needlerush (Juncus roemerianus) saltmarsh along the Gulf of Mexico coastline in peninsular west-central Florida, USA. This location has a subtropical climate with mean daily temperatures ranging from 15.4 °C in January to 27.8 °C in August, and annual precipitation of 1336 mm. Precipitation falls as rain primarily between June and September. Tides are semi-diurnal, with 0.57 m median amplitudes during the year preceding sampling (U.S. NOAA National Ocean Service, Clearwater Beach, Florida, station 8726724). Sea-level rise is 4.0 ± 0.6 mm per year (1973-2020 trend, mean ± 95 % confidence interval, NOAA NOS Clearwater Beach station). The A. germinans mangrove zone is either adjacent to water or fringed on the seaward side by a narrow band of red mangrove (Rhizophora mangle). A near-monoculture of J. roemerianus is often adjacent to and immediately landward of the A. germinans zone. The transition from the mangrove to the J. roemerianus zone is variable in our study area. An abrupt edge between closed-canopy mangrove and J. roemerianus monoculture may extend for up to several hundred meters in some locations, while other stretches of ecotone present a gradual transition where smaller, widely spaced trees are interspersed into the herbaceous marsh. Juncus roemerianus then extends landward to a high marsh patchwork of succulent halophytes (including Salicornia bigellovi, Sesuvium sp., and Batis maritima), scattered dwarf mangrove, and salt pans, followed in turn by upland vegetation that includes Pinus sp. and Serenoa repens. Field design and sample collection. We established three study sites spaced at approximately 5 km intervals along the western coastline of the central Florida peninsula. The sites consisted of the Salt Springs (28.3298°, -82.7274°), Energy Marine Center (28.2903°, -82.7278°), and Green Key (28.2530°, -82.7496°) sites on the Gulf of Mexico coastline in Pasco County, Florida, USA. At each site, we established three plot pairs, each consisting of one saltmarsh plot and one mangrove plot. Plots were 50 m^2 in size. Plots pairs within a site were separated by 230-1070 m, and the mangrove and saltmarsh plots composing a pair were 70-170 m apart. All plot pairs consisted of directly adjacent patches of mangrove forest and J. roemerianus saltmarsh, with the mangrove forests exhibiting a closed canopy and a tree architecture (height 4-6 m, crown width 1.5-3 m). Mangrove plots were located at approximately the midpoint between the seaward edge (water-mangrove interface) and landward edge (mangrove-marsh interface) of the mangrove zone. Saltmarsh plots were located 20-25 m away from any mangrove trees and into the J. roemerianus zone (i.e., landward from the mangrove-marsh interface). Plot pairs were coarsely similar in geomorphic setting, as all were located on the Gulf of Mexico coastline, rather than within major sheltering formations like Tampa Bay, and all plot pairs fit the tide-dominated domain of the Woodroffe classification (Woodroffe, 2002, "Coasts: Form, Process and Evolution", Cambridge University Press), given their conspicuous semi-diurnal tides. There was nevertheless some geomorphic variation, as some plot pairs were directly open to the Gulf of Mexico while others sat behind keys and spits or along small tidal creeks. Our use of a plot-pair approach is intended to control for this geomorphic variation. Plot center elevations (cm above mean sea level, NAVD 88) were estimated by overlaying the plot locations determined with a global positioning system (Garmin GPS 60, Olathe, KS, USA) on a LiDAR-derived bare-earth digital elevation model (Dewberry, Inc., 2019). The digital elevation model had a vertical accuracy of ± 10 cm (95 % CI) and a horizontal accuracy of ± 116 cm (95 % CI). Soil samples were collected via coring at low tide in June 2011. From each plot, we collected a composite soil sample consisting of three discrete 5.1 cm diameter soil cores taken at equidistant points to 7.6 cm depth. Cores were taken by tapping a sleeve into the soil until its top was flush with the soil surface, sliding a hand under the core, and lifting it up. Cores were then capped and transferred on ice to our laboratory at the University of South Florida (Tampa, Florida, USA), where they were combined in plastic zipper bags, and homogenized by hand into plot-level composite samples on the day they were collected. A damp soil subsample was immediately taken from each composite sample to initiate 1 y incubations for determination of active C and N (see below). The remainder of each composite sample was then placed in a drying oven (60 °C) for 1 week with frequent mixing of the soil to prevent aggregation and liberate water. Organic wetland soils are sometimes dried at 70 °C, however high drying temperatures can volatilize non-water liquids and oxidize and decompose organic matter, so 50 °C is also a common drying temperature for organic soils (Gardner 1986, "Methods of Soil Analysis: Part 1", Soil Science Society of America); we accordingly chose 60 °C as a compromise between sufficient water removal and avoidance of non-water mass loss. Bulk density was determined as soil dry mass per core volume (adding back the dry mass equivalent of the damp subsample removed prior to drying). Dried subsamples were obtained for determination of soil organic matter (SOM), mineral texture composition, and extractable and total carbon (C) and nitrogen (N) within the following week. Sample analyses. A dried subsample was apportioned from each composite sample to determine SOM as mass loss on ignition at 550 °C for 4 h. After organic matter was removed from soil via ignition, mineral particle size composition was determined using a combination of wet sieving and density separation in 49 mM (3 %) sodium hexametaphosphate ((NaPO_3)_6) following procedures in Kettler et al. (2001, Soil Science Society of America Journal 65, 849-852). The percentage of dry soil mass composed of silt and clay particles (hereafter, fines) was calculated as the mass lost from dispersed mineral soil after sieving (0.053 mm mesh sieve). Fines could have been slightly underestimated if any clay particles were burned off during the preceding ignition of soil. An additional subsample was taken from each composite sample to determine extractable N and organic C concentrations via 0.5 M potassium sulfate (K_2SO_4) extractions. We combined soil and extractant (ratio of 1 g dry soil:5 mL extractant) in plastic bottles, reciprocally shook the slurry for 1 h at 120 rpm, and then gravity filtered it through Fisher G6 (1.6 μm pore size) glass fiber filters, followed by colorimetric detection of nitrite (NO_2^-) + nitrate (NO_3^-) and ammonium (NH_4^+) in the filtrate (Hood Nowotny et al., 2010,Soil Science Society of America Journal 74, 1018-1027) using a microplate spectrophotometer (Biotek Epoch, Winooski, VT, USA). Filtrate was also analyzed for dissolved organic C (referred to hereafter as extractable organic C) and total dissolved N via combustion and oxidation followed by detection of the evolved CO_2 and N oxide gases on a Formacs HT TOC/TN analyzer (Skalar, Breda, The Netherlands). Extractable organic N was then computed as total dissolved N in filtrate minus extractable mineral N (itself the sum of extractable NH_4-N and NO_2-N + NO_3-N). We determined soil total C and N from dried, milled subsamples subjected to elemental analysis (ECS 4010, Costech, Inc., Valencia, CA, USA) at the University of South Florida Stable Isotope Laboratory. Median concentration of inorganic C in unvegetated surface soil at our sites is 0.5 % of soil mass (Anderson, 2019, Univ. of South Florida M.S. thesis via methods in Wang et al., 2011, Environmental Monitoring and Assessment 174, 241-257). Inorganic C concentrations are likely even lower in our samples from under vegetation, where organic matter would dilute the contribution of inorganic C to soil mass. Nevertheless, the presence of a small inorganic C pool in our soils may be counted in the total C values we report. Extractable organic C is necessarily of organic C origin given the method (sparging with HCl) used in detection. Active C and N represent the fractions of organic C and N that are mineralizable by soil microorganisms under aerobic conditions in long-term soil incubations. To quantify active C and N, 60 g of field-moist soil were apportioned from each composite sample, placed in a filtration apparatus, and incubated in the dark at 25 °C and field capacity moisture for 365 d (as in Lewis et al., 2014, Ecosphere 5, art59). Moisture levels were maintained by frequently weighing incubated soil and wetting them up to target mass. Daily CO_2 flux was quantified on 29 occasions at 0.5-3 week intervals during the incubation period (with shorter intervals earlier in the incubation), and these per day flux rates were integrated over the 365 d period to compute an estimate of active C. Observations of per day flux were made by sealing samples overnight in airtight chambers fitted with septa and quantifying headspace CO_2 accumulation by injecting headspace samples (obtained through the septa via needle and syringe) into an infrared gas analyzer (PP Systems EGM 4, Amesbury, MA, USA). To estimate active N, each incubated sample was leached with a C and N free, 35 psu solution containing micronutrients (Nadelhoffer, 1990, Soil Science Society of America Journal 54, 411-415) on 19 occasions at increasing 1-6 week intervals during the 365 d incubation, and then extracted in 0.5 M K_2SO_4 at the end of the incubation in order to remove any residual mineral N. Active N was then quantified as the total mass of mineral N leached and extracted. Mineral N in leached and extracted solutions was detected as NH_4-N and NO_2-N + NO_3-N via colorimetry as above. This incubation technique precludes new C and N inputs and persistently leaches mineral N, forcing microorganisms to meet demand by mineralizing existing pools, and thereby directly assays the potential activity of soil organic C and N pools present at the time of soil sampling. Because this analysis commences with disrupting soil physical structure, it is biased toward higher estimates of active fractions. Calculations. Non-mobile C and N fractions were computed as total C and N concentrations minus the extractable and active fractions of each element. This data package reports surface-soil constituents (moisture, fines, SOM, and C and N pools and fractions) in both gravimetric units (mass constituent / mass soil) and areal units (mass constituent / soil surface area integrated through 7.6 cm soil depth, the depth of sampling). Areal concentrations were computed as X × D × 7.6, where X is the gravimetric concentration of a soil constituent, D is soil bulk density (g dry soil / cm^3), and 7.6 is the sampling depth in cm. 
    more » « less
  4. ; ; ; ( , Dryad)
    This dataset contains monthly average output files from the iCAM6 simulations used in the manuscript "Enhancing understanding of the hydrological cycle via pairing of process-oriented and isotope ratio tracers," in review at the Journal of Advances in Modeling Earth Systems. A file corresponding to each of the tagged and isotopic variables used in this manuscript is included. Files are at 0.9° latitude x 1.25° longitude, and are in NetCDF format. Data from two simulations are included: 1) a simulation where the atmospheric model was "nudged" to ERA5 wind and surface pressure fields, by adding an additional tendency (see section 3.1 of associated manuscript), and 2) a simulation where the atmospheric state was allowed to freely evolve, using only boundary conditions imposed at the surface and top of atmosphere. Specific information about each of the variables provided is located in the "usage notes" section below. Associated article abstract: The hydrologic cycle couples the Earth's energy and carbon budgets through evaporation, moisture transport, and precipitation. Despite a wealth of observations and models, fundamental limitations remain in our capacity to deduce even the most basic properties of the hydrological cycle, including the spatial pattern of the residence time (RT) of water in the atmosphere and the mean distance traveled from evaporation sources to precipitation sinks. Meanwhile, geochemical tracers such as stable water isotope ratios provide a tool to probe hydrological processes, yet their interpretation remains equivocal despite several decades of use. As a result, there is a need for new mechanistic tools that link variations in water isotope ratios to underlying hydrological processes. Here we present a new suite of “process-oriented tags,” which we use to explicitly trace hydrological processes within the isotopically enabled Community Atmosphere Model, version 6 (iCAM6). Using these tags, we test the hypotheses that precipitation isotope ratios respond to parcel rainout, variations in atmospheric RT, and preserve information regarding meteorological conditions during evaporation. We present results for a historical simulation from 1980 to 2004, forced with winds from the ERA5 reanalysis. We find strong evidence that precipitation isotope ratios record information about atmospheric rainout and meteorological conditions during evaporation, but little evidence that precipitation isotope ratios vary with water vapor RT. These new tracer methods will enable more robust linkages between observations of isotope ratios in the modern hydrologic cycle or proxies of past terrestrial environments and the environmental processes underlying these observations.   Details about the simulation setup can be found in section 3 of the associated open-source manuscript, "Enhancing understanding of the hydrological cycle via pairing of process‐oriented and isotope ratio tracers." In brief, we conducted two simulations of the atmosphere from 1980-2004 using the isotope-enabled version of the Community Atmosphere Model 6 (iCAM6) at 0.9x1.25° horizontal resolution, and with 30 vertical hybrid layers spanning from the surface to ~3 hPa. In the first simulation, wind and surface pressure fields were "nudged" toward the ERA5 reanalysis dataset by adding a nudging tendency, preventing the model from diverging from observed/reanalysis wind fields. In the second simulation, no additional nudging tendency was included, and the model was allowed to evolve 'freely' with only boundary conditions provided at the top (e.g., incoming solar radiation) and bottom (e.g., observed sea surface temperatures) of the model. In addition to the isotopic variables, our simulation included a suite of 'process-oriented tracers,' which we describe in section 2 of the manuscript. These variables are meant to track a property of water associated with evaporation, condensation, or atmospheric transport. Metadata are provided about each of the files below; moreover, since the attached files are NetCDF data - this information is also provided with the data files. NetCDF metadata can be accessed using standard tools (e.g., ncdump). Each file has 4 variables: the tagged quantity, and the associated coordinate variables (time, latitude, longitude). The latter three are identical across all files, only the tagged quantity changes. Twelve files are provided for the nudged simulation, and an additional three are provided for the free simulations: Nudged simulation files iCAM6_nudged_1980-2004_mon_RHevap: Mass-weighted mean evaporation source property: RH (%) with respect to surface temperature. iCAM6_nudged_1980-2004_mon_Tevap: Mass-weighted mean evaporation source property: surface temperature in Kelvin iCAM6_nudged_1980-2004_mon_Tcond: Mass-weighted mean condensation property: temperature (K) iCAM6_nudged_1980-2004_mon_columnQ: Total (vertically integrated) precipitable water (kg/m2).  Not a tagged quantity, but necessary to calculate depletion times in section 4.3 (e.g., Fig. 11 and 12). iCAM6_nudged_1980-2004_mon_d18O: Precipitation d18O (‰ VSMOW) iCAM6_nudged_1980-2004_mon_d18Oevap_0: Mass-weighted mean evaporation source property - d18O of the evaporative flux (e.g., the 'initial' isotope ratio prior to condensation), (‰ VSMOW) iCAM6_nudged_1980-2004_mon_dxs: Precipitation deuterium excess (‰ VSMOW) - note that precipitation d2H can be calculated from this file and the precipitation d18O as d2H = d-excess - 8*d18O. iCAM6_nudged_1980-2004_mon_dexevap_0: Mass-weighted mean evaporation source property - deuterium excess of the evaporative flux iCAM6_nudged_1980-2004_mon_lnf: Integrated property - ln(f) calculated from the constant-fractionation d18O tracer (see section 3.2). iCAM6_nudged_1980-2004_mon_precip: Total precipitation rate in m/s. Note there is an error in the metadata in this file - it is total precipitation, not just convective precipitation. iCAM6_nudged_1980-2004_mon_residencetime: Mean atmospheric water residence time (in days). iCAM6_nudged_1980-2004_mon_transportdistance: Mean atmospheric water transport distance (in km). Free simulation files iCAM6_free_1980-2004_mon_d18O: Precipitation d18O (‰ VSMOW) iCAM6_free_1980-2004_mon_dxs: Precipitation deuterium excess (‰ VSMOW) - note that precipitation d2H can be calculated from this file and the precipitation d18O as d2H = d-excess - 8*d18O. iCAM6_free_1980-2004_mon_precip: Total precipitation rate in m/s. Note there is an error in the metadata in this file - it is total precipitation, not just convective precipitation. 
    more » « less
  5. ; ( , ICPSR - Interuniversity Consortium for Political and Social Research)
    The COVID-19 pandemic has dramatically altered family life in the United States. Over the long duration of the pandemic, parents had to adapt to shifting work conditions, virtual schooling, the closure of daycare facilities, and the stress of not only managing households without domestic and care supports but also worrying that family members may contract the novel coronavirus. Reports early in the pandemic suggest that these burdens have fallen disproportionately on mothers, creating concerns about the long-term implications of the pandemic for gender inequality and mothers’ well-being. Nevertheless, less is known about how parents’ engagement in domestic labor and paid work has changed throughout the pandemic, what factors may be driving these changes, and what the long-term consequences of the pandemic may be for the gendered division of labor and gender inequality more generally.

    The Study on U.S. Parents’ Divisions of Labor During COVID-19 (SPDLC) collects longitudinal survey data from partnered U.S. parents that can be used to assess changes in parents’ divisions of domestic labor, divisions of paid labor, and well-being throughout and after the COVID-19 pandemic. The goal of SPDLC is to understand both the short- and long-term impacts of the pandemic for the gendered division of labor, work-family issues, and broader patterns of gender inequality.

    Survey data for this study is collected using Prolifc (www.prolific.co), an opt-in online platform designed to facilitate scientific research. The sample is comprised U.S. adults who were residing with a romantic partner and at least one biological child (at the time of entry into the study). In each survey, parents answer questions about both themselves and their partners. Wave 1 of SPDLC was conducted in April 2020, and parents who participated in Wave 1 were asked about their division of labor both prior to (i.e., early March 2020) and one month after the pandemic began. Wave 2 of SPDLC was collected in November 2020. Parents who participated in Wave 1 were invited to participate again in Wave 2, and a new cohort of parents was also recruited to participate in the Wave 2 survey. Wave 3 of SPDLC was collected in October 2021. Parents who participated in either of the first two waves were invited to participate again in Wave 3, and another new cohort of parents was also recruited to participate in the Wave 3 survey. This research design (follow-up survey of panelists and new cross-section of parents at each wave) will continue through 2024, culminating in six waves of data spanning the period from March 2020 through October 2024. An estimated total of approximately 6,500 parents will be surveyed at least once throughout the duration of the study.

    SPDLC data will be released to the public two years after data is collected; Waves 1 and 2 are currently publicly available. Wave 3 will be publicly available in October 2023, with subsequent waves becoming available yearly. Data will be available to download in both SPSS (.sav) and Stata (.dta) formats, and the following data files will be available: (1) a data file for each individual wave, which contains responses from all participants in that wave of data collection, (2) a longitudinal panel data file, which contains longitudinal follow-up data from all available waves, and (3) a repeated cross-section data file, which contains the repeated cross-section data (from new respondents at each wave) from all available waves. Codebooks for each survey wave and a detailed user guide describing the data are also available. Response Rates: Of the 1,157 parents who participated in Wave 1, 828 (72%) also participated in the Wave 2 study. Presence of Common Scales: The following established scales are included in the survey:
    • Self-Efficacy, adapted from Pearlin's mastery scale (Pearlin et al., 1981) and the Rosenberg self-esteem scale (Rosenberg, 2015) and taken from the American Changing Lives Survey
    • Communication with Partner, taken from the Marriage and Relationship Survey (Lichter & Carmalt, 2009)
    • Gender Attitudes, taken from the National Survey of Families and Households (Sweet & Bumpass, 1996)
    • Depressive Symptoms (CES-D-10)
    • Stress, measured using Cohen's Perceived Stress Scale (Cohen, Kamarck, & Mermelstein, 1983)
    Full details about these scales and all other items included in the survey can be found in the user guide and codebook
    The second wave of the SPDLC was fielded in November 2020 in two stages. In the first stage, all parents who participated in W1 of the SPDLC and who continued to reside in the United States were re-contacted and asked to participate in a follow-up survey. The W2 survey was posted on Prolific, and messages were sent via Prolific’s messaging system to all previous participants. Multiple follow-up messages were sent in an attempt to increase response rates to the follow-up survey. Of the 1,157 respondents who completed the W1 survey, 873 at least started the W2 survey. Data quality checks were employed in line with best practices for online surveys (e.g., removing respondents who did not complete most of the survey or who did not pass the attention filters). After data quality checks, 5.2% of respondents were removed from the sample, resulting in a final sample size of 828 parents (a response rate of 72%).

    In the second stage, a new sample of parents was recruited. New parents had to meet the same sampling criteria as in W1 (be at least 18 years old, reside in the United States, reside with a romantic partner, and be a parent living with at least one biological child). Also similar to the W1 procedures, we oversampled men, Black individuals, individuals who did not complete college, and individuals who identified as politically conservative to increase sample diversity. A total of 1,207 parents participated in the W2 survey. Data quality checks led to the removal of 5.7% of the respondents, resulting in a final sample size of new respondents at Wave 2 of 1,138 parents.

    In both stages, participants were informed that the survey would take approximately 20 minutes to complete. All panelists were provided monetary compensation in line with Prolific’s compensation guidelines, which require that all participants earn above minimum wage for their time participating in studies.
    To be included in SPDLC, respondents had to meet the following sampling criteria at the time they enter the study: (a) be at least 18 years old, (b) reside in the United States, (c) reside with a romantic partner (i.e., be married or cohabiting), and (d) be a parent living with at least one biological child. Follow-up respondents must be at least 18 years old and reside in the United States, but may experience changes in relationship and resident parent statuses. Smallest Geographic Unit: U.S. State

    This work is licensed under the Creative Commons Attribution 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/. In accordance with this license, all users of these data must give appropriate credit to the authors in any papers, presentations, books, or other works that use the data. A suggested citation to provide attribution for these data is included below:            

    Carlson, Daniel L. and Richard J. Petts. 2022. Study on U.S. Parents’ Divisions of Labor During COVID-19 User Guide: Waves 1-2.  

    To help provide estimates that are more representative of U.S. partnered parents, the SPDLC includes sampling weights. Weights can be included in statistical analyses to make estimates from the SPDLC sample representative of U.S. parents who reside with a romantic partner (married or cohabiting) and a child aged 18 or younger based on age, race/ethnicity, and gender. National estimates for the age, racial/ethnic, and gender profile of U.S. partnered parents were obtained using data from the 2020 Current Population Survey (CPS). Weights were calculated using an iterative raking method, such that the full sample in each data file matches the nationally representative CPS data in regard to the gender, age, and racial/ethnic distributions within the data. This variable is labeled CPSweightW2 in the Wave 2 dataset, and CPSweightLW2 in the longitudinal dataset (which includes Waves 1 and 2). There is not a weight variable included in the W1-W2 repeated cross-section data file.
     
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email