skip to main content

Title: The Visible Cash Effect with Prepaid Incentives: Evidence for Data Quality, Response Rates, Generalizability, and Cost

A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This “visible cash effect” suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample more » of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives’ effectiveness in several circumstances.

« less
Publication Date:
Journal Name:
Journal of Survey Statistics and Methodology
Oxford University Press
Sponsoring Org:
National Science Foundation
More Like this
  1. A non-response follow-up study by mail in a national sample of U.S. households had five embedded experiments to test the effects of an advance mailing, alternate survey titles, 1- or 2-page questionnaire length, the inclusion or exclusion of political questions on the 1-page questionnaire, and the position of political content on the first or second page of the 2-page questionnaire. None of these design elements affected the payout of escalated postpaid incentives. Advance mailings had no effect on response rate. A short title (National Survey of Households) had a slightly higher response rate than a longer, more descriptive one (National Survey of Households, Families, and Covid-19). Political question content, whether by inclusion, exclusion, or position, had no discernable effect on response, even among prior-study non-respondents. Questionnaire length was inversely related to response: the 2-page questionnaire depressed the overall response rate by 3.7 points (58.5 compared to 54.8 percent, weighted) and depressed response for the critical sample group of prior non-respondents by 6.9 points (36.9 compared to 29.9).
  2. Despite the growing popularity of digital payment transactions in the United States, most survey participation incentives are still paid through cash or check and then distributed to respondents or potential sample members via direct mail. Though survey researchers have explored alternative incentives, such as e-gift cards, for online samples, there has been no study of electronic cash incentives—specifically paid through mobile pay applications—to date. In this article, we briefly review the literature on incentives used in online surveys and then examine survey incentive payment preferences among respondents using a small, web-based survey of younger adults. Our results suggest a greater preference for cash incentives paid through mobile applications than through direct mail, further highlighting the need for more research on the efficacy of electronically-delivered monetary incentives.
  3. Green Lake is the deepest natural inland lake in Wisconsin, with a maximum depth of about 72 meters. In the early 1900s, the lake was believed to have very good water quality (low nutrient concentrations and good water clarity) with low dissolved oxygen (DO) concentrations occurring in only the deepest part of the lake. Because of increased phosphorus (P) inputs from anthropogenic activities in its watershed, total phosphorus (TP) concentrations in the lake have increased; these changes have led to increased algal production and low DO concentrations not only in the deepest areas but also in the middle of the water column (metalimnion). The U.S. Geological Survey has routinely monitored the lake since 2004 and its tributaries since 1988. Results from this monitoring led the Wisconsin Department of Natural Resources (WDNR) to list the lake as impaired because of low DO concentrations in the metalimnion, and they identified elevated TP concentrations as the cause of impairment. As part of this study by the U.S. Geological Survey, in cooperation with the Green Lake Sanitary District, the lake and its tributaries were comprehensively sampled in 2017–18 to augment ongoing monitoring that would further describe the low DO concentrations in the lake (especiallymore »in the metalimnion). Empirical and process-driven water-quality models were then used to determine the causes of the low DO concentrations and the magnitudes of P-load reductions needed to improve the water quality of the lake enough to meet multiple water-quality goals, including the WDNR’s criteria for TP and DO. Data from previous studies showed that DO concentrations in the metalimnion decreased slightly as summer progressed in the early 1900s but, since the late 1970s, have typically dropped below 5 milligrams per liter (mg/L), which is the WDNR criterion for impairment. During 2014–18 (the baseline period for this study), the near-surface geometric mean TP concentration during June–September in the east side of the lake was 0.020 mg/L and in the west side was 0.016 mg/L (both were above the 0.015-mg/L WDNR criterion for the lake), and the metalimnetic DO minimum concentrations (MOMs) measured in August ranged from 1.0 to 4.7 mg/L. The degradation in water quality was assumed to have been caused by excessive P inputs to the lake; therefore, the TP inputs to the lake were estimated. The mean annual external P load during 2014–18 was estimated to be 8,980 kilograms per year (kg/yr), of which monitored and unmonitored tributary inputs contributed 84 percent, atmospheric inputs contributed 8 percent, waterfowl contributed 7 percent, and septic systems contributed 1 percent. During fall turnover, internal sediment recycling contributed an additional 7,040 kilograms that increased TP concentrations in shallow areas of the lake by about 0.020 mg/L. The elevated TP concentrations then persisted until the following spring. On an annual basis, however, there was a net deposition of P to the bottom sediments. Empirical models were used to describe how the near-surface water quality of Green Lake would be expected to respond to changes in external P loading. Predictions from the models showed a relatively linear response between P loading and TP and chlorophyll-a (Chl-a) concentrations in the lake, with the changes in TP and Chl-a concentrations being less on a percentage basis (50–60 percent for TP and 30–70 percent for Chl-a) than the changes in P loading. Mean summer water clarity, quantified by Secchi disk depths, had a greater response to decreases in P loading than to increases in P loading. Based on these relations, external P loading to the lake would need to be decreased from 8,980 kg/yr to about 5,460 kg/yr for the geometric mean June–September TP concentration in the east side of the lake, with higher TP concentrations than in the west side, to reach the WDNR criterion of 0.015 mg/L. This reduction of 3,520 kg/yr is equivalent to a 46-percent reduction in the potentially controllable external P sources (all external sources except for precipitation, atmospheric deposition, and waterfowl) from those measured during water years 2014–18. The total external P loading would need to decrease to 7,680 kg/yr (a 17-percent reduction in potentially controllable external P sources) for near-surface June–September TP concentrations in the west side of the lake to reach 0.015 mg/L. Total external P loading would need to decrease to 3,870–5,320 kg/yr for the lake to be classified as oligotrophic, with a near-surface June–September TP concentration of 0.012 mg/L. Results from the hydrodynamic water-quality model GLM–AED (General Lake Model coupled to the Aquatic Ecodynamics modeling library) indicated that MOMs are driven by external P loading and internal sediment recycling that lead to high TP concentrations during spring and early summer, which in turn lead to high phytoplankton production, high metabolism and respiration, and ultimately DO consumption in the upper, warmer areas of the metalimnion. GLM–AED results indicated that settling of organic material during summer might be slowed by the colder, denser, and more viscous water in the metalimnion and thus increase DO consumption. Based on empirical evidence from a comparison of MOMs with various meteorological, hydrologic, water quality, and in-lake physical factors, MOMs were lower during summers, when metalimnetic water temperatures were warmer, near-surface Chl-a and TP concentrations were higher, and Secchi depths were lower. GLM–AED results indicated that the external P load would need to be reduced to about 4,060 kg/yr, a 57-percent reduction from that measured in 2014–18, to eliminate the occurrence of MOMs less than 5 mg/L during more than 75 percent of the years (the target provided by the WDNR). Large reductions in external P loading are expected to have an immediate effect on the near-surface TP concentrations and metalimnetic DO concentrations in Green Lake; however, it may take several years for the full effects of the external-load reduction to be observed because internal sediment recycling is an important source of P for the following spring.« less
  4. Abstract
    Excessive phosphorus (P) applications to croplands can contribute to eutrophication of surface waters through surface runoff and subsurface (leaching) losses. We analyzed leaching losses of total dissolved P (TDP) from no-till corn, hybrid poplar (Populus nigra X P. maximowiczii), switchgrass (Panicum virgatum), miscanthus (Miscanthus giganteus), native grasses, and restored prairie, all planted in 2008 on former cropland in Michigan, USA. All crops except corn (13 kg P ha−1 year−1) were grown without P fertilization. Biomass was harvested at the end of each growing season except for poplar. Soil water at 1.2 m depth was sampled weekly to biweekly for TDP determination during March–November 2009–2016 using tension lysimeters. Soil test P (0–25 cm depth) was measured every autumn. Soil water TDP concentrations were usually below levels where eutrophication of surface waters is frequently observed (> 0.02 mg L−1) but often higher than in deep groundwater or nearby streams and lakes. Rates of P leaching, estimated from measured concentrations and modeled drainage, did not differ statistically among cropping systems across years; 7-year cropping system means ranged from 0.035 to 0.072 kg P ha−1 year−1 with large interannual variation. Leached P was positively related to STP, which decreased over the 7 years in all systems. These results indicate that both P-fertilized and unfertilized cropping systems mayMore>>
  5. Abstract
    The COVID-19 pandemic has dramatically altered family life in the United States. Over the long duration of the pandemic, parents had to adapt to shifting work conditions, virtual schooling, the closure of daycare facilities, and the stress of not only managing households without domestic and care supports but also worrying that family members may contract the novel coronavirus. Reports early in the pandemic suggest that these burdens have fallen disproportionately on mothers, creating concerns about the long-term implications of the pandemic for gender inequality and mothers’ well-being. Nevertheless, less is known about how parents’ engagement in domestic labor and paid work has changed throughout the pandemic, what factors may be driving these changes, and what the long-term consequences of the pandemic may be for the gendered division of labor and gender inequality more generally. <br /><br />The Study on U.S. Parents’ Divisions of Labor During COVID-19 (SPDLC) collects longitudinal survey data from partnered U.S. parents that can be used to assess changes in parents’ divisions of domestic labor, divisions of paid labor, and well-being throughout and after the COVID-19 pandemic. The goal of SPDLC is to understand both the short- and long-term impacts of the pandemic for the genderedMore>>