Background Studies that use ecological momentary assessments (EMAs) or wearable sensors to track numerous attributes, such as physical activity, sleep, and heart rate, can benefit from reductions in missing data. Maximizing compliance is one method of reducing missing data to increase the return on the heavy investment of time and money into large-scale studies. Objective This paper aims to identify the extent to which compliance can be prospectively predicted from individual attributes and initial compliance. Methods We instrumented 757 information workers with fitness trackers for 1 year and conducted EMAs in the first 56 days of study participation as part of an observational study. Their compliance with the EMA and fitness tracker wearing protocols was analyzed. Overall, 31 individual characteristics (eg, demographics and personalities) and behavioral variables (eg, early compliance and study portal use) were considered, and 14 variables were selected to create beta regression models for predicting compliance with EMAs 56 days out and wearable compliance 1 year out. We surveyed study participation and correlated the results with compliance. Results Our modeling indicates that 16% and 25% of the variance in EMA compliance and wearable compliance, respectively, could be explained through a survey of demographics and personality in a held-out sample. The likelihood of higher EMA and wearable compliance was associated with being older (EMA: odds ratio [OR] 1.02, 95% CI 1.00-1.03; wearable: OR 1.02, 95% CI 1.01-1.04), speaking English as a first language (EMA: OR 1.38, 95% CI 1.05-1.80; wearable: OR 1.39, 95% CI 1.05-1.85), having had a wearable before joining the study (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.50, 95% CI 1.23-1.83), and exhibiting conscientiousness (EMA: OR 1.25, 95% CI 1.04-1.51; wearable: OR 1.34, 95% CI 1.14-1.58). Compliance was negatively associated with exhibiting extraversion (EMA: OR 0.74, 95% CI 0.64-0.85; wearable: OR 0.67, 95% CI 0.57-0.78) and having a supervisory role (EMA: OR 0.65, 95% CI 0.54-0.79; wearable: OR 0.66, 95% CI 0.54-0.81). Furthermore, higher wearable compliance was negatively associated with agreeableness (OR 0.68, 95% CI 0.56-0.83) and neuroticism (OR 0.85, 95% CI 0.73-0.98). Compliance in the second week of the study could help explain more variance; 62% and 66% of the variance in EMA compliance and wearable compliance, respectively, was explained. Finally, compliance correlated with participants’ self-reflection on the ease of participation, usefulness of our compliance portal, timely resolution of issues, and compensation adequacy, suggesting that these are avenues for improving compliance. Conclusions We recommend conducting an initial 2-week pilot to measure trait-like compliance and identify participants at risk of long-term noncompliance, performing oversampling based on participants’ individual characteristics to avoid introducing bias in the sample when excluding data based on noncompliance, using an issue tracking portal, and providing special care in troubleshooting to help participants maintain compliance.
more »
« less
Investigating Receptivity and Affect Using Machine Learning: Ecological Momentary Assessment and Wearable Sensing Study
As mobile health (mHealth) studies become increasingly productive owing to the advancements in wearable and mobile sensor technology, our ability to monitor and model human behavior will be constrained by participant receptivity. Many health constructs are dependent on subjective responses, and without such responses, researchers are left with little to no ground truth to accompany our ever-growing biobehavioral data. This issue can significantly impact the quality of a study, particularly for populations known to exhibit lower compliance rates. To address this challenge, researchers have proposed innovative approaches that use machine learning (ML) and sensor data to modify the timing and delivery of surveys. However, an overarching concern is the potential introduction of biases or unintended influences on participants’ responses when implementing new survey delivery methods.
This study aims to demonstrate the potential impact of an ML-based ecological momentary assessment (EMA) delivery system (using receptivity as the predictor variable) on the participants’ reported emotional state. We examine the factors that affect participants’ receptivity to EMAs in a 10-day wearable and EMA–based emotional state–sensing mHealth study. We study the physiological relationships indicative of receptivity and affect while also analyzing the interaction between the 2 constructs.
We collected data from 45 healthy participants wearing 2 devices measuring electrodermal activity, accelerometer, electrocardiography, and skin temperature while answering 10 EMAs daily, containing questions about perceived mood. Owing to the nature of our constructs, we can only obtain ground truth measures for both affect and receptivity during responses. Therefore, we used unsupervised and supervised ML methods to infer affect when a participant did not respond. Our unsupervised method used k-means clustering to determine the relationship between physiology and receptivity and then inferred the emotional state during nonresponses. For the supervised learning method, we primarily used random forest and neural networks to predict the affect of unlabeled data points as well as receptivity.
Our findings showed that using a receptivity model to trigger EMAs decreased the reported negative affect by >3 points or 0.29 SDs in our self-reported affect measure, scored between 13 and 91. The findings also showed a bimodal distribution of our predicted affect during nonresponses. This indicates that this system initiates EMAs more commonly during states of higher positive emotions.
Our results showed a clear relationship between affect and receptivity. This relationship can affect the efficacy of an mHealth study, particularly those that use an ML algorithm to trigger EMAs. Therefore, we propose that future work should focus on a smart trigger that promotes EMA receptivity without influencing affect during sampled time points.
- PAR ID:
- 10498550
- Publisher / Repository:
- JMIR publications
- Date Published:
- Journal Name:
- JMIR mHealth and uHealth
- Volume:
- 12
- ISSN:
- 2291-5222
- Page Range / eLocation ID:
- e46347
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
While machine learning (ML) can validly score psychological constructs from behavior, several conditions often change across studies, making it difficult to understand why the psychometric properties of ML models differ across studies. We address this gap in the context of automatically scored interviews. Across multiple datasets, for interview- or question-level scoring of self-reported, tested, and interviewer-rated constructs, we manipulate the training sample size and natural language processing (NLP) method while observing differences in ground truth reliability. We examine how these factors influence the ML model scores’ test–retest reliability and convergence, and we develop multilevel models for estimating the convergent-related validity of ML model scores in similar interviews. When the ground truth is interviewer ratings, hundreds of observations are adequate for research purposes, while larger samples are recommended for practitioners to support generalizability across populations and time. However, self-reports and tested constructs require larger training samples. Particularly when the ground truth is interviewer ratings, NLP embedding methods improve upon count-based methods. Given mixed findings regarding ground truth reliability, we discuss future research possibilities on factors that affect supervised ML models’ psychometric properties.
-
more » « less
Abstract Sleep and stress independently enhance emotional memory consolidation. In particular, theta oscillations (4–7 Hz) during rapid eye movement (REM) sleep increase coherence in an emotional memory network (i.e., hippocampus, amygdala, and prefrontal cortex) and enhance emotional memory. However, little is known about how stress during learning might
interact with subsequent REM theta activity to affect emotional memory. In the current study, we examined whether the relationship between REM theta activity and emotional memory differs as a function of pre‐encoding stress exposure and reactivity. Participants underwent a psychosocial stressor (the Trier Social Stress Task;n = 32) or a comparable control task (n = 32) prior to encoding. Task‐evoked cortisol reactivity was assessed by salivary cortisol rise from pre‐ to post‐stressor, and participants in the stress condition were additionally categorized as high or low cortisol responders via a median split. During incidental encoding, participants studied 150 line drawings of negative, neutral, and positive images, followed by the complete color photo. All participants then slept overnight in the lab with polysomnographic recording. The next day, they were given a surprise recognition memory task. Results showed that memory was better for emotional relative to neutral information. Critically, these findings were observed only in the stress condition. No emotional memory benefit was observed in the control condition. In stressed participants, REM theta power significantly predicted memory for emotional information, specifically for positive items. This relationship was observed only in high cortisol responders. For low responders and controls, there was no relationship between REM theta and memory of any valence. These findings provide evidence that elevated stress at encoding, and accompanying changes in neuromodulators such as cortisol, may interact with theta activity during REM sleep to promote selective consolidation of emotional information. -
Background Multiple strategies can be used when self-monitoring diet, physical activity, and perceived stress, but no gold standards are available. Although self-monitoring is a core element of self-management and behavior change, the success of mHealth behavioral tools depends on their validity and reliability, which lack evidence. African American and Latina mothers in the United States are high-priority populations for apps that can be used for self-monitoring of diet, physical activity, and stress because the body mass index (BMI) of mothers typically increases for several years after childbirth and the risks of obesity and its’ sequelae diseases are elevated among minority populations. Objective To examine the intermethod reliability and concurrent validity of smartphone-based self-monitoring via ecological momentary assessments (EMAs) and use of daily diaries for diet, stress, and physical activity compared with brief recall measures, anthropometric biomeasures, and bloodspot biomarkers. Methods A purposive sample (n=42) of primarily African American (16/42, 39%) and Latina (18/42, 44%) mothers was assigned Android smartphones for using Ohmage apps to self-monitor diet, perceived stress, and physical activity over 6 months. Participants were assessed at 3- and 6-month follow-ups. Recall measures included brief food frequency screeners, physical activity assessments adapted from the National Health and Nutrition Examination Survey, and the nine-item psychological stress measure. Anthropometric biomeasures included BMI, body fat, waist circumference, and blood pressure. Bloodspot assays for Epstein–Barr virus and C-reactive protein were used as systemic load and stress biomarkers. EMAs and daily diary questions assessed perceived quality and quantity of meals, perceived stress levels, and moderate, vigorous, and light physical activity. Units of analysis were follow-up assessments (n=29 to n=45 depending on the domain) of the participants (n=29 with sufficient data for analyses). Correlations, R2 statistics, and multivariate linear regressions were used to assess the strength of associations between variables. Results Almost all participants (39/42, 93%) completed the study. Intermethod reliability between smartphone-based EMAs and diary reports and their corresponding recall reports was highest for stress and diet; correlations ranged from .27 to .52 (P<.05). However, it was unexpectedly low for physical activity; no significant associations were observed. Concurrent validity was demonstrated for diet EMAs and diary reports on systolic blood pressure (r=−.32), C-reactive protein level (r=−.34), and moderate and vigorous physical activity recalls (r=.35 to.48), suggesting a covariation between healthy diet and physical activity behaviors. EMAs and diary reports on stress were not associated with Epstein–Barr virus and C-reactive protein level. Diary reports on moderate and vigorous physical activity were negatively associated with BMI and body fat (r=−.35 to −.44, P<.05). Conclusions Brief smartphone-based EMA use may be valid and reliable for long-term self-monitoring of diet, stress, and physical activity. Lack of intermethod reliability for physical activity measures is consistent with prior research, warranting more research on the efficacy of smartphone-based self-monitoring of self-management and behavior change support.more » « less
-
more » « less
BACKGROUND Effective communication is crucial during health crises, and social media has become a prominent platform for public health experts to inform and to engage with the public. At the same time, social media also platforms pseudo-experts who may promote contrarian views. Despite the significance of social media, key elements of communication such as the use of moral or emotional language and messaging strategy, particularly during the COVID-19 pandemic, has not been explored.
OBJECTIVE This study aims to analyze how notable public health experts (PHEs) and pseudo-experts communicated with the public during the COVID-19 pandemic. Our focus is the emotional and moral language they used in their messages across a range of pandemic issues. We also study their engagement with political elites and how the public engaged with PHEs to better understand the impact of these health experts on the public discourse.
METHODS We gathered a dataset of original tweets from 489 PHEs and 356 pseudo- experts on Twitter (now X) from January 2020 to January 2021, as well as replies to the original tweets from the PHEs. We identified the key issues that PHEs and pseudo- experts prioritized. We also determined the emotional and moral language in both the original tweets and the replies. This approach enabled us to characterize key priorities for PHEs and pseudo-experts, as well as differences in messaging strategy between these two groups. We also evaluated the influence of PHE language and strategy on the public response.
RESULTS Our analyses revealed that PHEs focus on masking, healthcare, education, and vaccines, whereas pseudo-experts discuss therapeutics and lockdowns more frequently. PHEs typically used positive emotional language across all issues, expressing optimism and joy. Pseudo-experts often utilized negative emotions of pessimism and disgust, while limiting positive emotional language to origins and therapeutics. Along the dimensions of moral language, PHEs and pseudo-experts differ on care versus harm, and authority versus subversion, across different issues. Negative emotional and moral language tends to boost engagement in COVID-19 discussions, across all issues. However, the use of positive language by PHEs increases the use of positive language in the public responses. PHEs act as liberal partisans: they express more positive affect in their posts directed at liberals and more negative affect directed at conservative elites. In contrast, pseudo-experts act as conservative partisans. These results provide nuanced insights into the elements that have polarized the COVID-19 discourse.
CONCLUSIONS Understanding the nature of the public response to PHE’s messages on social media is essential for refining communication strategies during health crises. Our findings emphasize the need for experts to consider the strategic use of moral and emotional language in their messages to reduce polarization and enhance public trust.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.2196/46347
