An official website of the United States government
Despite the growing popularity of digital payment transactions in the United States, most survey participation incentives are still paid through cash or check and then distributed to respondents or potential sample members via direct mail. Though survey researchers have explored alternative incentives, such as e-gift cards, for online samples, there has been no study of electronic cash incentives—specifically paid through mobile pay applications—to date. In this article, we briefly review the literature on incentives used in online surveys and then examine survey incentive payment preferences among respondents using a small, web-based survey of younger adults. Our results suggest a greater preference for cash incentives paid through mobile applications than through direct mail, further highlighting the need for more research on the efficacy of electronically-delivered monetary incentives.
more »
« less
Distributing a survey using Every Door Direct Mail in an ideal use case
While paper mail-based surveys avoid much of the risk of bots and fraudulent data, they suffer from lower response rates and ever-inflating material and logistical costs. In response, there is a nascent, but growing literature investigating a lower cost, explicitly anonymous, mail-based survey distribution method called Every Door Direct Mail (EDDM). This study contributes to this growing body of literature by using EDDM to disseminate a sequential mixed-mode census-style survey that meets best use-case recommendations per past research. We make several design alterations to elicit higher response rates including using an outer envelope and cash incentive. The survey, distributed near large-scale solar developments in three urban Michigan communities (~1554 households), was geographically based, targeted a specific and limited population, and covered the potentially sensitive topic of local solar development, which may have also led to a higher response rate. The survey achieved an overall response rate of 10.2% with 158 complete surveys returned, demonstrating this work’s usefulness, use case, and flexibility.
more »
« less
- Award ID(s):
- 1934348
- PAR ID:
- 10600316
- Publisher / Repository:
- SAGE Publications
- Date Published:
- Journal Name:
- Methodological Innovations
- Volume:
- 18
- Issue:
- 2
- ISSN:
- 2059-7991
- Format(s):
- Medium: X Size: p. 89-100
- Size(s):
- p. 89-100
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
US federal and state agencies have advocated for the development of derelict, polluted, often post-industrial and urban sites, i.e. “brownfields,” with large-scale solar (LSS). So too have local officials, developers, industry experts, and rural community residents arguing LSS development of brownfields represents a ‘win-win-win’. They argue these developments face less opposition from local communities than rural development, are a pathway to remedy prior injustices, and can reutilize industrial sites rather than develop valuable farmland or natural habitats. Yet little research exists examining residents’ perceptions of LSS development of urban brownfields and the perceived local community impacts that accompany such development. This is a key gap and raises the question of whether these developments indeed promote justice and whether and how opposition may arise from local communities. Using the theories of place attachment, procedural justice, along with resident perception of benefits, this study examines urban resident perceptions and preferences of urban LSS developments. The study utilized a mail survey disseminated via the Every Door Direct Mail service provided by the United States Postal Office to gather data from urban residents in three Michigan communities living within one mile of an urban LSS development. The response rate to the survey across all three communities was 10.2%, with 158 complete surveys returned. Respondent support rates were found to be similar to previous studies that looked at large-scale grid-feeding solar developments, with the two significant predictors of support being positive perceptions of local benefits and procedural justice.more » « less
-
Abstract A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This “visible cash effect” suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives’ effectiveness in several circumstances.more » « less
-
A non-response follow-up study by mail in a national sample of U.S. households had five embedded experiments to test the effects of an advance mailing, alternate survey titles, 1- or 2-page questionnaire length, the inclusion or exclusion of political questions on the 1-page questionnaire, and the position of political content on the first or second page of the 2-page questionnaire. None of these design elements affected the payout of escalated postpaid incentives. Advance mailings had no effect on response rate. A short title (National Survey of Households) had a slightly higher response rate than a longer, more descriptive one (National Survey of Households, Families, and Covid-19). Political question content, whether by inclusion, exclusion, or position, had no discernable effect on response, even among prior-study non-respondents. Questionnaire length was inversely related to response: the 2-page questionnaire depressed the overall response rate by 3.7 points (58.5 compared to 54.8 percent, weighted) and depressed response for the critical sample group of prior non-respondents by 6.9 points (36.9 compared to 29.9).more » « less
-
Grinnell, Frederick (Ed.)Queer identities are often ignored in diversity initiatives, yet there is a growing body of research that describes notable heterosexist and gender-normative expectations in STEM that lead to unsupportive and discriminatory environments and to the lower persistence of queer individuals. Research on the experiences of queer-spectrum individuals is limited by current demographic practices. In surveys that are queer-inclusive there is no consensus on best practices, and individuals with queer genders and queer sexual, romantic, and related orientations are often lumped together in a general category (e.g. LGBTQ+). We developed two queer-inclusive demographics questions and administered them as part of a larger study in undergraduate engineering and computer science classes (n = 3698), to determine which of three survey types for gender (conventional, queered, open-ended) provided the most robust data and compared responses to national data to determine if students with queer genders and/or queer sexual, romantic, and related orientations were underrepresented in engineering and computer science programs. The gender survey with queer-identity options provided the most robust data, as measured by higher response rates and relatively high rates of disclosing queer identities. The conventional survey (male, female, other) had significantly fewer students disclose queer identities, and the open-ended survey had a significantly higher non-response rate. Allowing for multiple responses on the survey was important: 78% of those with queer gender identities and 9% of those with queer sexual, romantic and related orientations selected multiple identities within the same survey question. Queer students in our study were underrepresented relative to national data. Students who disclosed queer gender identities were 7/100ths of the expected number, and those with queer orientations were under-represented by one-quarter. Further work developing a research-based queered demographics instrument is needed for larger-scale changes in demographics practices, which will help others identify and address barriers that queer-spectrum individuals face in STEM.more » « less
