skip to main content

Explore Research Products in the NSF-PAR

Title: Multimodal Time-Series Activity Forecasting for Adaptive Lifestyle Intervention Design
Physical activity is a cornerstone of chronic conditions and one of the most critical factors in reducing the risks of cardiovascular diseases, the leading cause of death in the United States. App-based lifestyle interventions have been utilized to promote physical activity in people with or at risk for chronic conditions. However, these mHealth tools have remained largely static and do not adapt to the changing behavior of the user. In a step toward designing adaptive interventions, we propose BeWell24Plus, a framework for monitoring activity and user engagement and developing computational models for outcome prediction and intervention design. In particular, we focus on devising algorithms that combine data about physical activity and engagement with the app to predict future physical activity performance. Knowing in advance how active a person is going to be in the next day can help with designing adaptive interventions that help individuals achieve their physical activity goals. Our technique combines the recent history of a person's physical activity with app engagement metrics such as when, how often, and for how long the app was used to forecast the near future's activity. We formulate the problem of multimodal activity forecasting and propose an LSTM-based realization of our proposed model architecture, which estimates physical activity outcomes in advance by examining the history of app usage and physical activity of the user. We demonstrate the effectiveness of our forecasting approach using data collected with 58 prediabetic people in a 9-month user study. We show that our multimodal forecasting approach outperforms single-modality forecasting by 2.2$ to 11.1% in mean-absolute-error.  more » « less
Award ID(s):
2227002 2210133 1954372
NSF-PAR ID:
10389477
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
2022 IEEE-EMBS International Conference on Wearable and Implantable Body Sensor Networks (BSN)
Page Range / eLocation ID:
1 to 4
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ( , JMIR Research Protocols)
    null (Ed.)
    Background Cardiovascular disease (CVD) disparities are a particularly devastating manifestation of health inequity. Despite advancements in prevention and treatment, CVD is still the leading cause of death in the United States. Additionally, research indicates that African American (AA) and other ethnic-minority populations are affected by CVD at earlier ages than white Americans. Given that AAs are the fastest-growing population of smartphone owners and users, mobile health (mHealth) technologies offer the unparalleled potential to prevent or improve self-management of chronic disease among this population. Objective To address the unmet need for culturally tailored primordial prevention CVD–focused mHealth interventions, the MOYO app was cocreated with the involvement of young people from this priority community. The overall project aims to develop and evaluate the effectiveness of a novel smartphone app designed to reduce CVD risk factors among urban-AAs, 18-29 years of age. Methods The theoretical underpinning will combine the principles of community-based participatory research and the agile software development framework. The primary outcome goals of the study will be to determine the usability, acceptability, and functionality of the MOYO app, and to build a cloud-based data collection infrastructure suitable for digital epidemiology in a disparity population. Changes in health-related parameters over a 24-week period as determined by both passive (eg, physical activity levels, sleep duration, social networking) and active (eg, use of mood measures, surveys, uploading pictures of meals and blood pressure readings) measures will be the secondary outcome. Participants will be recruited from a majority AA “large city” school district, 2 historically black colleges or universities, and 1 urban undergraduate college. Following baseline screening for inclusion (administered in person), participants will receive the beta version of the MOYO app. Participants will be monitored during a 24-week pilot period. Analyses of varying data including social network dynamics, standard metrics of activity, percentage of time away from a given radius of home, circadian rhythm metrics, and proxies for sleep will be performed. Together with external variables (eg, weather, pollution, and socioeconomic indicators such as food access), these metrics will be used to train machine-learning frameworks to regress them on the self-reported quality of life indicators. Results This 5-year study (2015-2020) is currently in the implementation phase. We believe that MOYO can build upon findings of classical epidemiology and longitudinal studies like the Jackson Heart Study by adding greater granularity to our knowledge of the exposures and behaviors that affect health and disease, and creating a channel for outreach capable of launching interventions, clinical trials, and enhancements of health literacy. Conclusions The results of this pilot will provide valuable information about community cocreation of mHealth programs, efficacious design features, and essential infrastructure for digital epidemiology among young AA adults. International Registered Report Identifier (IRRID) DERR1-10.2196/16699 
    more » « less
  2. ; ; ; ; ; ; ( , JMIR Formative Research)
    Background Over half of US adults have at least one chronic disease, including obesity. Although physical activity is an important component of chronic disease self-management, few reach the recommended physical activity goals. Individuals who identify as racial and ethnic minorities are disproportionally affected by chronic diseases and physical inactivity. Interventions using consumer-based wearable devices have shown promise for increasing physical activity among patients with chronic diseases; however, populations with the most to gain, such as minorities, have been poorly represented to date. Objective This study aims to assess the feasibility, acceptability, and preliminary outcomes of an 8-week text-based coaching and Fitbit program aimed at increasing the number of steps in a predominantly overweight ethnic minority population. Methods Overweight patients (BMI >25 kg/m2) were recruited from an internal medicine clinic located in an inner-city academic medical center. Fitbit devices were provided. Using 2-way SMS text messaging, health coaches (HCs) guided patients to establish weekly step goals that were specific, measurable, attainable, realistic, and time-bound. SMS text messaging and Fitbit activities were managed using a custom-designed app. Program feasibility was assessed via the recruitment rate, retention rate (the proportion of eligible participants completing the 8-week program), and patient engagement (based on the number of weekly text message goals set with the HC across the 8-week period). Acceptability was assessed using a qualitative, summative evaluation. Exploratory statistical analysis included evaluating the average weekly steps in week 1 compared with week 8 using a paired t test (2-tailed) and modeling daily steps over time using a linear mixed model. Results Of the 33 patients initially screened; 30 (91%) patients were enrolled in the study. At baseline, the average BMI was 39.3 (SD 9.3) kg/m2, with 70% (23/33) of participants presenting as obese. A total of 30% (9/30) of participants self-rated their health as either fair or poor, and 73% (22/30) of participants set up ≥6 weekly goals across the 8-week program. In total, 93% (28/30) of participants completed a qualitative summative evaluation, and 10 themes emerged from the evaluation: patient motivation, convenient SMS text messaging experience, social support, supportive accountability, technology support, self-determined goals, achievable goals, feedback from Fitbit, challenges, and habit formation. There was no significant group change in the average weekly steps for week 1 compared with week 8 (mean difference 7.26, SD 6209.3; P=.99). However, 17% (5/30) of participants showed a significant increase in their daily steps. Conclusions Overall, the results demonstrate the feasibility and acceptability of a remotely delivered walking study that included an HC; SMS text messaging; a wearable device (Fitbit); and specific, measurable, attainable, realistic, and time-bound goals within an ethnic minority patient population. Results support further development and testing in larger samples to explore efficacy. 
    more » « less
  3. ; ( , NASA Human Research Program Investigators Conference)
    INTRODUCTION: Apollo-11 (A-11) was the first manned space mission to successfully bring astronauts to the moon and return them safely. Effective team based communications is required for mission specialists to work collaboratively to learn, engage, and solve complex problems. As part of NASA’s goal in assessing team and mission success, all vital speech communications between these personnel were recorded using the multi-track SoundScriber system onto analog tapes, preserving their contribution in the success of one of the greatest achievements in human history. More than +400 personnel served as mission specialists/support who communicated across 30 audio loops, resulting in +9k hours of data for A-11. To ensure success of this mission, it was necessary for teams to communicate, learn, and address problems in a timely manner. Previous research has found that compatibility of individual personalities within teams is important for effective team collaboration of those individuals. Hence, it is essential to identify each speaker’s role during an Apollo mission and analyze group communications for knowledge exchange and problem solving to achieve a common goal. Assessing and analyzing speaker roles during the mission can allow for exploring engagement analysis for multi-party speaker situations. METHOD: The UTDallas Fearless steps Apollo data is comprised of 19,000 hours (A-11,A-13,A-1) possessing unique and multiple challenges as it is characterized by severe noise and degradation as well as overlap instances over the 30 channels. For our study, we have selected a subset of 100 hours manually transcribed by professional annotators for speaker labels. The 100 hours are obtained from three mission critical events: 1. Lift-Off (25 hours) 2. Lunar-Landing (50 hours) 3. Lunar-Walking (25 hours). Five channels of interest, out of 30 channels were selected with the most speech activity, the primary speakers operating these five channels are command/owners of these channels. For our analysis, we select five speaker roles: Flight Director (FD), Capsule Communicator (CAPCOM), Guidance, Navigation and, Control (GNC), Electrical, environmental, and consumables manager (EECOM), and Network (NTWK). To track and tag individual speakers across our Fearless Steps audio dataset, we use the concept of ‘where’s Waldo’ to identify all instances of our speakers-of-interest across a cluster of other speakers. Also, to understand speaker roles of our speaker-of-interests, we use speaker duration of primary speaker vs secondary speaker and speaker turns as our metrics to determine the role of the speaker and to understand their responsibility during the three critical phases of the mission. This enables a content linking capability as well as provide a pathway to analyzing group engagement, group dynamics of people working together in an enclosed space, psychological effects, and cognitive analysis in such individuals. IMPACT: NASA’s Apollo Program stands as one of the most significant contributions to humankind. This collection opens new research options for recognizing team communication, group dynamics, and human engagement/psychology for future deep space missions. Analyzing team communications to achieve such goals would allow for the formulation of educational and training technologies for assessment of STEM knowledge, task learning, and educational feedback. Also, identifying these personnel can help pay tribute and yield personal recognition to the hundreds of notable engineers and scientist who made this feat possible. ILLUSTRATION: In this work, we propose to illustrate how a pre-trained speech/language network can be used to obtain powerful speaker embeddings needed for speaker diarization. This framework is used to build these learned embeddings to label unique speakers over sustained audio streams. To train and test our system, we will make use of Fearless Steps Apollo corpus, allowing us to effectively leverage a limited label information resource (100 hours of labeled data out of +9000 hours). Furthermore, we use the concept of 'Finding Waldo' to identify key speakers of interest (SOI) throughout the Apollo-11 mission audio across multiple channel audio streams. 
    more » « less
  4. ; ; ( , Proceedings of the ACM on Human-Computer Interaction)
    Online discussion platforms provide a forum to strengthen and propagate belief in misinformed conspiracy theories. Yet, they also offer avenues for conspiracy theorists to express their doubts and experiences of cognitive dissonance. Such expressions of dissonance may shed light on who abandons misguided beliefs and under what circumstances. This paper characterizes self-disclosures of dissonance about QAnon-a conspiracy theory initiated by a mysterious leader "Q" and popularized by their followers ?anons"-in conspiratorial subreddits. To understand what dissonance and disbelief mean within conspiracy communities, we first characterize their social imaginaries-a broad understanding of how people collectively imagine their social existence. Focusing on 2K posts from two image boards, 4chan and 8chan, and 1.2 M comments and posts from 12 subreddits dedicated to QAnon, we adopt a mixed-methods approach to uncover the symbolic language representing the movement,expectations,practices,heroes and foes of the QAnon community. We use these social imaginaries to create a computational framework for distinguishing belief and dissonance from general discussion about QAnon, surfacing in the 1.2M comments. We investigate the dissonant comments to characterize the dissonance expressed along QAnon social imaginaries. Further, analyzing user engagement with QAnon conspiracy subreddits, we find that self-disclosures of dissonance correlate with a significant decrease in user contributions and ultimately with their departure from the community. Our work offers a systematic framework for uncovering the dimensions and coded language related to QAnon social imaginaries and can serve as a toolbox for studying other conspiracy theories across different platforms. We also contribute a computational framework for identifying dissonance self-disclosures and measuring the changes in user engagement surrounding dissonance. Our work provide insights into designing dissonance based interventions that can potentially dissuade conspiracists from engaging in online conspiracy discussion communities. 
    more » « less
  5. ; ; ; ; ; ( , Journal of Medical Internet Research)
    Background Home health aides (HHAs) provide necessary hands-on care to older adults and those with chronic conditions in their homes. Despite their integral role, HHAs experience numerous challenges in their work, including their ability to communicate with other health care professionals about patient care while caring for patients and access to educational resources. Although technological interventions have the potential to address these challenges, little is known about the technological landscape and existing technology-based interventions designed for and used by this workforce. Objective We conducted a scoping review of the scientific literature to identify existing studies that have described, designed, deployed, or tested technology-based tools and apps intended for use by HHAs to care for patients at home. To complement our literature review, we conducted a landscape analysis of existing mobile apps intended for HHAs providing in-home care. Methods We searched the following databases from their inception to October 2020: Ovid MEDLINE, Ovid Embase, Cochrane Library, and CINAHL (EBSCO). A total of 3 researchers screened the yield using prespecified inclusion and exclusion criteria. In addition, 4 researchers independently reviewed these articles, and a fifth researcher arbitrated when needed. Among studies that met the inclusion criteria, data were extracted and summarized narratively. An analysis of mobile health apps designed for HHAs was performed using a predefined set of terms to search Google Play and Apple App stores. Overall, 2 researchers independently screened the resulting apps, and those that met the inclusion criteria were categorized according to their intended purpose and functionality. Results Of the 8643 studies retrieved, 182 (2.11%) underwent full-text review, and 4.9% (9/182) met our inclusion criteria. Approximately half (4/9, 44%) of the studies were descriptive in nature, proposing technology-based systems (eg, web portals and dashboards) or prototypes without a technical or user-based evaluation of the technology. In most (7/9, 78%) papers, HHAs were just one of several users and not the sole or primary intended users of the technology. Our review of mobile apps yielded 166 Android and iOS apps, of which 48 (29%) met the inclusion criteria. These apps provided HHAs with one or more of the following functions: electronic visit verification (29/48, 60%), clocking in and out (23/48, 48%), documentation (22/48, 46%), task checklist (19/48, 40%), communication between HHA and agency (14/48, 29%), patient information (6/48, 13%), resources (5/48, 10%), and communication between HHA and patients (4/48, 8%). Of the 48 apps, 25 (52%) performed monitoring functions, 4 (8%) performed supporting functions, and 19 (40%) performed both. Conclusions A limited number of studies and mobile apps have been designed to support HHAs in their work. Further research and rigorous evaluation of technology-based tools are needed to assess their impact on the work HHAs provide in patient’s homes. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Save / Share this Record
  • Send to Email