skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Friday, September 13 until 2:00 AM ET on Saturday, September 14 due to maintenance. We apologize for the inconvenience.


Title: Using Health Concept Surveying to Elicit Usable Evidence: Case Studies of a Novel Evaluation Methodology
Background Developers, designers, and researchers use rapid prototyping methods to project the adoption and acceptability of their health intervention technology (HIT) before the technology becomes mature enough to be deployed. Although these methods are useful for gathering feedback that advances the development of HITs, they rarely provide usable evidence that can contribute to our broader understanding of HITs. Objective In this research, we aim to develop and demonstrate a variation of vignette testing that supports developers and designers in evaluating early-stage HIT designs while generating usable evidence for the broader research community. Methods We proposed a method called health concept surveying for untangling the causal relationships that people develop around conceptual HITs. In health concept surveying, investigators gather reactions to design concepts through a scenario-based survey instrument. As the investigator manipulates characteristics related to their HIT, the survey instrument also measures proximal cognitive factors according to a health behavior change model to project how HIT design decisions may affect the adoption and acceptability of an HIT. Responses to the survey instrument were analyzed using path analysis to untangle the causal effects of these factors on the outcome variables. Results We demonstrated health concept surveying in 3 case studies of sensor-based health-screening apps. Our first study (N=54) showed that a wait time incentive could influence more people to go see a dermatologist after a positive test for skin cancer. Our second study (N=54), evaluating a similar application design, showed that although visual explanations of algorithmic decisions could increase participant trust in negative test results, the trust would not have been enough to affect people’s decision-making. Our third study (N=263) showed that people might prioritize test specificity or sensitivity depending on the nature of the medical condition. Conclusions Beyond the findings from our 3 case studies, our research uses the framing of the Health Belief Model to elicit and understand the intrinsic and extrinsic factors that may affect the adoption and acceptability of an HIT without having to build a working prototype. We have made our survey instrument publicly available so that others can leverage it for their own investigations.  more » « less
Award ID(s):
1813675
NSF-PAR ID:
10357271
Author(s) / Creator(s):
; ; ; ; ;
Date Published:
Journal Name:
JMIR Human Factors
Volume:
9
Issue:
1
ISSN:
2292-9495
Page Range / eLocation ID:
e30474
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Mukherjee, Amitava (Ed.)

    What influences the adoption of SARS-CoV-2 mitigation behaviors–both personal, such as mask wearing and frequent handwashing, and social, such as avoiding large gatherings and physical contact–across countries? Understanding why some individuals are more willing to change their behavior to mitigate the spread of a pandemic will not only help us to address the current SARS-CoV-2 pandemic but also to respond to future ones. Researchers have pointed to a variety of factors that may influence individual adoption of personal and social mitigation behaviors, including social inequality, risk perception, personality traits, and government policies. While not denying the importance of these factors, we argue that the role of trust and confidence has received insufficient attention to date. Our study explores whether there is a difference in the way trust and confidence in particular leaders and organizations affect individual compliance and whether this effect is consistent across different types of mitigation behaviors. Specifically, we utilize an original cross-national survey conducted during the first wave of the SARS-CoV-2 pandemic (May-June 2020) to investigate how trust in scientists, medical professionals, politicians, and religious leaders and confidence in global, national, and local health organizations affects individual compliance in 16 countries/territories across five world regions. Our analyses, which control for the aforementioned factors as well as several others, suggest that trust in politicians and confidence in national health ministries have the most consistent influence on whether individuals adopt both personal and social mitigation behaviors. Across our sample, we find that greater trust in politicians is associated with lower levels of individual compliance with public health directives, whereas greater confidence in the national health ministry is associated with higher levels of individual compliance. Our findings suggest the need to understand trust and confidence as among the most important individual level characteristics driving compliance when developing and delivering messaging about the adoption of mitigation behaviors. The content of the message, it seems, will be most effective when citizens across countries trust its source. Trusted sources, such as politicians and the national health ministry, should thus consider working closely together when determining and communicating recommended health behaviors to avoid contradicting one another.

     
    more » « less
  2. The respective benefits and drawbacks of manual food journaling and automated dietary monitoring (ADM) suggest the value of semi-automated journaling systems combining the approaches. However, the current understanding of how people anticipate strategies for implementing semi-automated food journaling systems is limited. We therefore conduct a speculative survey study with 600 responses, examining how people anticipate approaches to automatic capture and prompting for details. Participants feel the location and detection capability of ADM sensors influences anticipated physical, social, and privacy burdens. People more positively anticipate prompts which contain information relevant to their journaling goals, help them recall what they ate, and are quick to respond to. Our work suggests a tradeoff between ADM systems' detection performance and anticipated acceptability, with sensors on facial areas having higher performance but lower acceptability than sensors in other areas and more usable prompting methods like those containing specific foods being more challenging to produce than manual reminders. We suggest opportunities to improve higher-acceptability, lower-accuracy ADM sensors, select approaches based on individual and practitioner journaling needs, and better describe capabilities to potential users. 
    more » « less
  3. Multiple symptom tracking applications (apps) were created during the early phase of the COVID-19 pandemic. While they provided crowdsourced information about the state of the pandemic in a scalable manner, they also posed significant privacy risks for individuals. The present study investigates the interplay between individual privacy attitudes and the adoption of symptom tracking apps. Using the communication privacy theory as a framework, it studies how users’ privacy attitudes changed during the public health emergency compared to the pre-COVID times. Based on focus-group interviews (N = 21), this paper reports significant changes in users’ privacy attitudes toward such apps. Research participants shared various reasons for both increased acceptability (e.g., disease uncertainty, public good) and decreased acceptability (e.g., reduced utility due to changed lifestyle) during COVID. The results of this study can assist health informatics researchers and policy designers in creating more socially acceptable health apps in the future. 
    more » « less
  4. Motivation: The question of what combination of attributes drives the adoption of a particular software technology is critical to developers. It determines both those technologies that receive wide support from the community and those which may be abandoned, thus rendering developers' investments worthless. Aim and Context: We model software technology adoption by developers and provide insights on specific technology attributes that are associated with better visibility among alternative technologies. Thus, our findings have practical value for developers seeking to increase the adoption rate of their products. Approach: We leverage social contagion theory and statistical modeling to identify, define, and test empirically measures that are likely to affect software adoption. More specifically, we leverage a large collection of open source repositories to construct a software dependency chain for a specific set of R language source-code files. We formulate logistic regression models, where developers' software library choices are modeled, to investigate the combination of technological attributes that drive adoption among competing data frame (a core concept for a data science languages) implementations in the R language: tidy and data.table. To describe each technology, we quantify key project attributes that might affect adoption (e.g., response times to raised issues, overall deployments, number of open defects, knowledge base) and also characteristics of developers making the selection (performance needs, scale, and their social network). Results: We find that a quick response to raised issues, a larger number of overall deployments, and a larger number of high-score StackExchange questions are associated with higher adoption. Decision makers tend to adopt the technology that is closer to them in the technical dependency network and in author collaborations networks while meeting their performance needs. To gauge the generalizability of the proposed methodology, we investigate the spread of two popular web JavaScript frameworks Angular and React, and discuss the results. Future work: We hope that our methodology encompassing social contagion that captures both rational and irrational preferences and the elucidation of key measures from large collections of version control data provides a general path toward increasing visibility, driving better informed decisions, and producing more sustainable and widely adopted software. 
    more » « less
  5. This study aimed to investigate the key technical and psychological factors that impact the architecture, engineering, and construction (AEC) professionals’ trust in collaborative robots (cobots) powered by artificial intelligence (AI). This study seeks to address the critical knowledge gaps surrounding the establishment and reinforcement of trust among AEC professionals in their collaboration with AI-powered cobots. In the context of the construction industry, where the complexities of tasks often necessitate human–robot teamwork, understanding the technical and psychological factors influencing trust is paramount. Such trust dynamics play a pivotal role in determining the effectiveness of human–robot collaboration on construction sites. This research employed a nationwide survey of 600 AEC industry practitioners to shed light on these influential factors, providing valuable insights to calibrate trust levels and facilitate the seamless integration of AI-powered cobots into the AEC industry. Additionally, it aimed to gather insights into opportunities for promoting the adoption, cultivation, and training of a skilled workforce to effectively leverage this technology. A structural equation modeling (SEM) analysis revealed that safety and reliability are significant factors for the adoption of AI-powered cobots in construction. Fear of being replaced resulting from the use of cobots can have a substantial effect on the mental health of the affected workers. A lower error rate in jobs involving cobots, safety measurements, and security of data collected by cobots from jobsites significantly impact reliability, and the transparency of cobots’ inner workings can benefit accuracy, robustness, security, privacy, and communication and result in higher levels of automation, all of which demonstrated as contributors to trust. The study’s findings provide critical insights into the perceptions and experiences of AEC professionals toward adoption of cobots in construction and help project teams determine the adoption approach that aligns with the company’s goals workers’ welfare. 
    more » « less