An official website of the United States government
BackgroundLaypeople have easy access to health information through large language models (LLMs), such as ChatGPT, and search engines, such as Google. Search engines transformed health information access, and LLMs offer a new avenue for answering laypeople’s questions. ObjectiveWe aimed to compare the frequency of use and attitudes toward LLMs and search engines as well as their comparative relevance, usefulness, ease of use, and trustworthiness in responding to health queries. MethodsWe conducted a screening survey to compare the demographics of LLM users and nonusers seeking health information, analyzing results with logistic regression. LLM users from the screening survey were invited to a follow-up survey to report the types of health information they sought. We compared the frequency of use of LLMs and search engines using ANOVA and Tukey post hoc tests. Lastly, paired-sample Wilcoxon tests compared LLMs and search engines on perceived usefulness, ease of use, trustworthiness, feelings, bias, and anthropomorphism. ResultsIn total, 2002 US participants recruited on Prolific participated in the screening survey about the use of LLMs and search engines. Of them, 52% (n=1045) of the participants were female, with a mean age of 39 (SD 13) years. Participants were 9.7% (n=194) Asian, 12.1% (n=242) Black, 73.3% (n=1467) White, 1.1% (n=22) Hispanic, and 3.8% (n=77) were of other races and ethnicities. Further, 1913 (95.6%) used search engines to look up health queries versus 642 (32.6%) for LLMs. Men had higher odds (odds ratio [OR] 1.63, 95% CI 1.34-1.99; P<.001) of using LLMs for health questions than women. Black (OR 1.90, 95% CI 1.42-2.54; P<.001) and Asian (OR 1.66, 95% CI 1.19-2.30; P<.01) individuals had higher odds than White individuals. Those with excellent perceived health (OR 1.46, 95% CI 1.1-1.93; P=.01) were more likely to use LLMs than those with good health. Higher technical proficiency increased the likelihood of LLM use (OR 1.26, 95% CI 1.14-1.39; P<.001). In a follow-up survey of 281 LLM users for health, most participants used search engines first (n=174, 62%) to answer health questions, but the second most common first source consulted was LLMs (n=39, 14%). LLMs were perceived as less useful (P<.01) and less relevant (P=.07), but elicited fewer negative feelings (P<.001), appeared more human (LLM: n=160, vs search: n=32), and were seen as less biased (P<.001). Trust (P=.56) and ease of use (P=.27) showed no differences. ConclusionsSearch engines are the primary source of health information; yet, positive perceptions of LLMs suggest growing use. Future work could explore whether LLM trust and usefulness are enhanced by supplementing answers with external references and limiting persuasive language to curb overreliance. Collaboration with health organizations can help improve the quality of LLMs’ health output.
more »
« less
Participant Reactions to Medical Screening: A Survey of Satisfaction With the C8 (PFOA) Health Project
We report participant perceptions of the 2005–2006 C8 Health Project, a massive medical monitoring effort in response to perfluorooctanoic acid (C8) in West Virginia and Ohio. The C8 Health Project consisted of a health survey ( n = 69,030), blood testing for ten per- and polyfluoroalkyl substances, and 50+ laboratory tests ( n = 66,899). A randomly selected subgroup was surveyed in 2007 on (1) demographics (2) satisfaction with the project, and (3) perceptions of outcomes such as contribution to personal/family, community health, and links to health outcomes. The response rate was 573/1500 (38.2 percent). Most (92.7 percent) characterized their participation experience as “excellent” or “good,” and most (96.2 percent) considered the project very “important,” “important,” or “moderately important.” No demographic variable predicted important changes in satisfaction or perception of project importance. We conclude that responses to the survey indicate strong positive assessments of project benefits.
more »
« less
- Award ID(s):
- 1456897
- PAR ID:
- 10549256
- Publisher / Repository:
- SAGE Publications
- Date Published:
- Journal Name:
- NEW SOLUTIONS: A Journal of Environmental and Occupational Health Policy
- Volume:
- 29
- Issue:
- 2
- ISSN:
- 1048-2911
- Format(s):
- Medium: X Size: p. 186-204
- Size(s):
- p. 186-204
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)In early 2020, colleges shifted abruptly from traditional in-person to remote distant instruction due to COVID-19 potentially exacerbating science, technology, engineering, and mathematics (STEM) students’ recruitment and retention. This preliminary study using survey methodology was conducted with STEM students at a small (700 students) private college to examine questions related to students’ perceptions of natural science careers, career decision-making factors, barriers influencing students’ career path, including effects of COVID-19 on career goals, mental health, and perceived quality of instruction. A Qualtrics® survey was sent to 180 STEM students, from which we received 53 responses (29.4% response rate). Consistent with other studies, family was one of the most important factors supporting their career path. Students had a relatively upbeat career outlook despite being in the middle of a global pandemic and were only moderately worried about the impact of COVID-19 on their future career. Despite these relatively positive outcomes, the abrupt switch to online instruction was viewed unfavorably by most respondents, who valued the hands-on learning experiences obtained with traditional in-person instruction. It is possible that respondents’ views of online instruction may improve over time as instructors become more adept at using new instructional tools. Future research should evaluate this aspect and whether students’ career goals change across time as the pandemic unfolds.more » « less
-
This descriptive study examines the experiences of virtually-trained new members in a hybrid distributed community of practice (CoP) focused on undergraduate genomics education. We utilized a sequential explanatory mixed methods research design consisting of an engagement survey for all community members (n=124), followed by interviews with new members (n=15). Survey analysis identifies several areas in which new members do not differ from members with longer involvement, including in motivations for involvement, levels of engagement, satisfaction, and perceived benefits of community involvement. These findings indicate ways in which virtual training and integration was able to facilitate important community outcomes within a new, online context. Our interviews reveal important elements of training new CoP members, including onboarding, implementation, and community engagement opportunities, that successfully facilitated new members’ integration into the community and contributed to their meeting the aforementioned outcomes. The findings of this study provide useful lessons and structures for growing communities through virtual means.more » « less
-
Abstract A prior study found that mailing prepaid incentives with $5 cash visible from outside the envelope increased the response rate to a mail survey by 4 percentage points compared to cash that was not externally visible. This “visible cash effect” suggests opportunities to improve survey response at little or no cost, but many unknowns remain. Among them: Does the visible cash effect generalize to different survey modes, respondent burdens, and cash amounts? Does it differ between fresh samples and reinterview samples? Does it affect data quality or survey costs? This article examines these questions using two linked studies where incentive visibility was randomized in a large probability sample for the American National Election Studies. The first study used $10 incentives with invitations to a long web questionnaire (median 71 minutes, n = 17,849). Visible cash increased response rates in a fresh sample for both screener and extended interview response (by 6.7 and 4.8 percentage points, respectively). Visible cash did not increase the response rate in a reinterview sample where the baseline reinterview response rate was very high (72 percent). The second study used $5 incentives with invitations to a mail-back paper questionnaire (n = 8,000). Visible cash increased the response rate in a sample of prior nonrespondents by 4.0 percentage points (from 31.5 to 35.5), but it did not increase the response rate in a reinterview sample where the baseline reinterview rate was very high (84 percent). In the two studies, several aspects of data quality were investigated, including speeding, non-differentiation, item nonresponse, nonserious responses, noncredible responses, sample composition, and predictive validity; no adverse effects of visible cash were detected, and sample composition improved marginally. Effects on survey costs were either negligible or resulted in net savings. Accumulated evidence now shows that visible cash can increase incentives’ effectiveness in several circumstances.more » « less
-
BackgroundMachine learning approaches, including deep learning, have demonstrated remarkable effectiveness in the diagnosis and prediction of diabetes. However, these approaches often operate as opaque black boxes, leaving health care providers in the dark about the reasoning behind predictions. This opacity poses a barrier to the widespread adoption of machine learning in diabetes and health care, leading to confusion and eroding trust. ObjectiveThis study aimed to address this critical issue by developing and evaluating an explainable artificial intelligence (AI) platform, XAI4Diabetes, designed to empower health care professionals with a clear understanding of AI-generated predictions and recommendations for diabetes care. XAI4Diabetes not only delivers diabetes risk predictions but also furnishes easily interpretable explanations for complex machine learning models and their outcomes. MethodsXAI4Diabetes features a versatile multimodule explanation framework that leverages machine learning, knowledge graphs, and ontologies. The platform comprises the following four essential modules: (1) knowledge base, (2) knowledge matching, (3) prediction, and (4) interpretation. By harnessing AI techniques, XAI4Diabetes forecasts diabetes risk and provides valuable insights into the prediction process and outcomes. A structured, survey-based user study assessed the app’s usability and influence on participants’ comprehension of machine learning predictions in real-world patient scenarios. ResultsA prototype mobile app was meticulously developed and subjected to thorough usability studies and satisfaction surveys. The evaluation study findings underscore the substantial improvement in medical professionals’ comprehension of key aspects, including the (1) diabetes prediction process, (2) data sets used for model training, (3) data features used, and (4) relative significance of different features in prediction outcomes. Most participants reported heightened understanding of and trust in AI predictions following their use of XAI4Diabetes. The satisfaction survey results further revealed a high level of overall user satisfaction with the tool. ConclusionsThis study introduces XAI4Diabetes, a versatile multi-model explainable prediction platform tailored to diabetes care. By enabling transparent diabetes risk predictions and delivering interpretable insights, XAI4Diabetes empowers health care professionals to comprehend the AI-driven decision-making process, thereby fostering transparency and trust. These advancements hold the potential to mitigate biases and facilitate the broader integration of AI in diabetes care.more » « less
