An official website of the United States government
ObjectivesMicrointeraction-based Ecological Momentary Assessment (micro-EMA) is a smartwatch-based tool that delivers single-question surveys, enabling respondents to quickly report their real-time experiences. The objectives of the two studies presented here were to evaluate micro-EMA's psychometric characteristics and feasibility across three response formats (2-point, 5-point, and 10-point scales) for adults with hearing loss. DesignIn the first study, thirty-two participants completed a dual-task experiment aimed at assessing the construct validity, responsiveness, intrusiveness, and test-retest reliability of micro-EMA across the three response formats. Participants listened to sentences at five signal-to-noise ratios (SNRs) ranging from −3 to 9 dB relative to the SNR for 50% speech understanding, answered the question “Hearing well?” on smartwatches, and repeated the sentences. In the second study, twenty-one participants wore smartwatches over 6 days. Every 15 min, participants were prompted to answer the question “Hearing well?” using one of the three response formats for 2 days. Participants provided feedback on their experience with micro-EMA. ResultsIn the dual-task experiment, participants reported improved hearing performance in micro-EMA as SNRs and speech recognition scores increased across all three response formats, supporting the tool's construct validity. Statistical models indicated that the 5-point and 10-point scales yielded larger relative changes between SNRs, suggesting higher responsiveness, compared to the 2-point scale. Participants completed surveys significantly faster with the 2-point scale, indicating lower intrusiveness, compared to the 5-point and 10-point scales. Correlation analysis revealed that over two visits 1 week apart, the 2-point scale had the poorest test-retest reliability, while the 5-point scale had the highest. In the field trial, participants completed 79.6% of the prompted surveys, with each participant averaging 42.9 surveys per day. Although participants experienced interruptions due to frequent prompts, annoyance and distraction levels were low. Most participants preferred the 5-point scale. ConclusionsThe dual-task experiment suggested that micro-EMA using the 5-point scale demonstrated superior psychometric characteristics compared to the 2-point and 10-point scales at the tested SNRs. The field trial further supported its feasibility for evaluating hearing performance in adults with hearing loss. Additional research is needed to explore the potential applications of micro-EMA in audiology research.
more »
« less
Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study
BackgroundAs mobile health (mHealth) studies become increasingly productive owing to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. Many health constructs are dependent on subjective responses, and without such responses, researchers are left with little to no ground truth to accompany our ever-growing biobehavioral data. This issue can significantly impact the quality of a study, particularly for populations known to exhibit lower compliance rates. To address this challenge, researchers have proposed innovative approaches that use machine learning (ML) and sensor data to modify the timing and delivery of surveys. However, an overarching concern is the potential introduction of biases or unintended influences on participants’ responses when implementing new survey delivery methods. ObjectiveThis study aims to demonstrate the potential impact of an ML-based ecological momentary assessment (EMA) delivery system (using receptivity as the predictor variable) on the participants’ reported emotional state. We examine the factors that affect participants’ receptivity to EMAs in a 10-day wearable and EMA–based emotional state–sensing mHealth study. We study the physiological relationships indicative of receptivity and affect while also analyzing the interaction between the 2 constructs. MethodsWe collected data from 45 healthy participants wearing 2 devices measuring electrodermal activity, accelerometer, electrocardiography, and skin temperature while answering 10 EMAs daily, containing questions about perceived mood. Owing to the nature of our constructs, we can only obtain ground truth measures for both affect and receptivity during responses. Therefore, we used unsupervised and supervised ML methods to infer affect when a participant did not respond. Our unsupervised method used k-means clustering to determine the relationship between physiology and receptivity and then inferred the emotional state during nonresponses. For the supervised learning method, we primarily used random forest and neural networks to predict the affect of unlabeled data points as well as receptivity. ResultsOur findings showed that using a receptivity model to trigger EMAs decreased the reported negative affect by >3 points or 0.29 SDs in our self-reported affect measure, scored between 13 and 91. The findings also showed a bimodal distribution of our predicted affect during nonresponses. This indicates that this system initiates EMAs more commonly during states of higher positive emotions. ConclusionsOur results showed a clear relationship between affect and receptivity. This relationship can affect the efficacy of an mHealth study, particularly those that use an ML algorithm to trigger EMAs. Therefore, we propose that future work should focus on a smart trigger that promotes EMA receptivity without influencing affect during sampled time points.
more »
« less
- PAR ID:
- 10498550
- Publisher / Repository:
- JMIR publications
- Date Published:
- Journal Name:
- JMIR mHealth and uHealth
- Volume:
- 12
- ISSN:
- 2291-5222
- Page Range / eLocation ID:
- e46347
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Background Studies that use ecological momentary assessments (EMAs) or wearable sensors to track numerous attributes, such as physical activity, sleep, and heart rate, can benefit from reductions in missing data. Maximizing compliance is one method of reducing missing data to increase the return on the heavy investment of time and money into large-scale studies. Objective This paper aims to identify the extent to which compliance can be prospectively predicted from individual attributes and initial compliance. Methods We instrumented 757 information workers with fitness trackers for 1 year and conducted EMAs in the first 56 days of study participation as part of an observational study. Their compliance with the EMA and fitness tracker wearing protocols was analyzed. Overall, 31 individual characteristics (eg, demographics and personalities) and behavioral variables (eg, early compliance and study portal use) were considered, and 14 variables were selected to create beta regression models for predicting compliance with EMAs 56 days out and wearable compliance 1 year out. We surveyed study participation and correlated the results with compliance. Results Our modeling indicates that 16% and 25% of the variance in EMA compliance and wearable compliance, respectively, could be explained through a survey of demographics and personality in a held-out sample. The likelihood of higher EMA and wearable compliance was associated with being older (EMA: odds ratio [OR] 1.02, 95% CI 1.00-1.03; wearable: OR 1.02, 95% CI 1.01-1.04), speaking English as a first language (EMA: OR 1.38, 95% CI 1.05-1.80; wearable: OR 1.39, 95% CI 1.05-1.85), having had a wearable before joining the study (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.50, 95% CI 1.23-1.83), and exhibiting conscientiousness (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.34, 95% CI 1.14-1.58). Compliance was negatively associated with exhibiting extraversion (EMA: OR 0.74, 95% CI 0.64-0.85; wearable: OR 0.67, 95% CI 0.57-0.78) and having a supervisory role (EMA: OR 0.65, 95% CI 0.54-0.79; wearable: OR 0.66, 95% CI 0.54-0.81). Furthermore, higher wearable compliance was negatively associated with agreeableness (OR 0.68, 95% CI 0.56-0.83) and neuroticism (OR 0.85, 95% CI 0.73-0.98). Compliance in the second week of the study could help explain more variance; 62% and 66% of the variance in EMA compliance and wearable compliance, respectively, was explained. Finally, compliance correlated with participants’ self-reflection on the ease of participation, usefulness of our compliance portal, timely resolution of issues, and compensation adequacy, suggesting that these are avenues for improving compliance. Conclusions We recommend conducting an initial 2-week pilot to measure trait-like compliance and identify participants at risk of long-term noncompliance, performing oversampling based on participants’ individual characteristics to avoid introducing bias in the sample when excluding data based on noncompliance, using an issue tracking portal, and providing special care in troubleshooting to help participants maintain compliance.more » « less
-
Background Multiple strategies can be used when self-monitoring diet, physical activity, and perceived stress, but no gold standards are available. Although self-monitoring is a core element of self-management and behavior change, the success of mHealth behavioral tools depends on their validity and reliability, which lack evidence. African American and Latina mothers in the United States are high-priority populations for apps that can be used for self-monitoring of diet, physical activity, and stress because the body mass index (BMI) of mothers typically increases for several years after childbirth and the risks of obesity and its’ sequelae diseases are elevated among minority populations. Objective To examine the intermethod reliability and concurrent validity of smartphone-based self-monitoring via ecological momentary assessments (EMAs) and use of daily diaries for diet, stress, and physical activity compared with brief recall measures, anthropometric biomeasures, and bloodspot biomarkers. Methods A purposive sample (n=42) of primarily African American (16/42, 39%) and Latina (18/42, 44%) mothers was assigned Android smartphones for using Ohmage apps to self-monitor diet, perceived stress, and physical activity over 6 months. Participants were assessed at 3- and 6-month follow-ups. Recall measures included brief food frequency screeners, physical activity assessments adapted from the National Health and Nutrition Examination Survey, and the nine-item psychological stress measure. Anthropometric biomeasures included BMI, body fat, waist circumference, and blood pressure. Bloodspot assays for Epstein–Barr virus and C-reactive protein were used as systemic load and stress biomarkers. EMAs and daily diary questions assessed perceived quality and quantity of meals, perceived stress levels, and moderate, vigorous, and light physical activity. Units of analysis were follow-up assessments (n=29 to n=45 depending on the domain) of the participants (n=29 with sufficient data for analyses). Correlations, R2 statistics, and multivariate linear regressions were used to assess the strength of associations between variables. Results Almost all participants (39/42, 93%) completed the study. Intermethod reliability between smartphone-based EMAs and diary reports and their corresponding recall reports was highest for stress and diet; correlations ranged from .27 to .52 (P<.05). However, it was unexpectedly low for physical activity; no significant associations were observed. Concurrent validity was demonstrated for diet EMAs and diary reports on systolic blood pressure (r=−.32), C-reactive protein level (r=−.34), and moderate and vigorous physical activity recalls (r=.35 to.48), suggesting a covariation between healthy diet and physical activity behaviors. EMAs and diary reports on stress were not associated with Epstein–Barr virus and C-reactive protein level. Diary reports on moderate and vigorous physical activity were negatively associated with BMI and body fat (r=−.35 to −.44, P<.05). Conclusions Brief smartphone-based EMA use may be valid and reliable for long-term self-monitoring of diet, stress, and physical activity. Lack of intermethod reliability for physical activity measures is consistent with prior research, warranting more research on the efficacy of smartphone-based self-monitoring of self-management and behavior change support.more » « less
-
BackgroundPredicting the likelihood of success of weight loss interventions using machine learning (ML) models may enhance intervention effectiveness by enabling timely and dynamic modification of intervention components for nonresponders to treatment. However, a lack of understanding and trust in these ML models impacts adoption among weight management experts. Recent advances in the field of explainable artificial intelligence enable the interpretation of ML models, yet it is unknown whether they enhance model understanding, trust, and adoption among weight management experts. ObjectiveThis study aimed to build and evaluate an ML model that can predict 6-month weight loss success (ie, ≥7% weight loss) from 5 engagement and diet-related features collected over the initial 2 weeks of an intervention, to assess whether providing ML-based explanations increases weight management experts’ agreement with ML model predictions, and to inform factors that influence the understanding and trust of ML models to advance explainability in early prediction of weight loss among weight management experts. MethodsWe trained an ML model using the random forest (RF) algorithm and data from a 6-month weight loss intervention (N=419). We leveraged findings from existing explainability metrics to develop Prime Implicant Maintenance of Outcome (PRIMO), an interactive tool to understand predictions made by the RF model. We asked 14 weight management experts to predict hypothetical participants’ weight loss success before and after using PRIMO. We compared PRIMO with 2 other explainability methods, one based on feature ranking and the other based on conditional probability. We used generalized linear mixed-effects models to evaluate participants’ agreement with ML predictions and conducted likelihood ratio tests to examine the relationship between explainability methods and outcomes for nested models. We conducted guided interviews and thematic analysis to study the impact of our tool on experts’ understanding and trust in the model. ResultsOur RF model had 81% accuracy in the early prediction of weight loss success. Weight management experts were significantly more likely to agree with the model when using PRIMO (χ2=7.9; P=.02) compared with the other 2 methods with odds ratios of 2.52 (95% CI 0.91-7.69) and 3.95 (95% CI 1.50-11.76). From our study, we inferred that our software not only influenced experts’ understanding and trust but also impacted decision-making. Several themes were identified through interviews: preference for multiple explanation types, need to visualize uncertainty in explanations provided by PRIMO, and need for model performance metrics on similar participant test instances. ConclusionsOur results show the potential for weight management experts to agree with the ML-based early prediction of success in weight loss treatment programs, enabling timely and dynamic modification of intervention components to enhance intervention effectiveness. Our findings provide methods for advancing the understandability and trust of ML models among weight management experts.more » « less
-
BACKGROUNDEffective communication is crucial during health crises, and social media has become a prominent platform for public health experts to inform and to engage with the public. At the same time, social media also platforms pseudo-experts who may promote contrarian views. Despite the significance of social media, key elements of communication such as the use of moral or emotional language and messaging strategy, particularly during the COVID-19 pandemic, has not been explored. OBJECTIVEThis study aims to analyze how notable public health experts (PHEs) and pseudo-experts communicated with the public during the COVID-19 pandemic. Our focus is the emotional and moral language they used in their messages across a range of pandemic issues. We also study their engagement with political elites and how the public engaged with PHEs to better understand the impact of these health experts on the public discourse. METHODSWe gathered a dataset of original tweets from 489 PHEs and 356 pseudo- experts on Twitter (now X) from January 2020 to January 2021, as well as replies to the original tweets from the PHEs. We identified the key issues that PHEs and pseudo- experts prioritized. We also determined the emotional and moral language in both the original tweets and the replies. This approach enabled us to characterize key priorities for PHEs and pseudo-experts, as well as differences in messaging strategy between these two groups. We also evaluated the influence of PHE language and strategy on the public response. RESULTSOur analyses revealed that PHEs focus on masking, healthcare, education, and vaccines, whereas pseudo-experts discuss therapeutics and lockdowns more frequently. PHEs typically used positive emotional language across all issues, expressing optimism and joy. Pseudo-experts often utilized negative emotions of pessimism and disgust, while limiting positive emotional language to origins and therapeutics. Along the dimensions of moral language, PHEs and pseudo-experts differ on care versus harm, and authority versus subversion, across different issues. Negative emotional and moral language tends to boost engagement in COVID-19 discussions, across all issues. However, the use of positive language by PHEs increases the use of positive language in the public responses. PHEs act as liberal partisans: they express more positive affect in their posts directed at liberals and more negative affect directed at conservative elites. In contrast, pseudo-experts act as conservative partisans. These results provide nuanced insights into the elements that have polarized the COVID-19 discourse. CONCLUSIONSUnderstanding the nature of the public response to PHE’s messages on social media is essential for refining communication strategies during health crises. Our findings emphasize the need for experts to consider the strategic use of moral and emotional language in their messages to reduce polarization and enhance public trust.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.2196/46347
