Search for: All records

Deep reinforcement learning (DRL) has proven capable of superhuman performance on many complex tasks. To achieve this success, DRL algorithms train a decision-making agent to select the actions that maximize some long-term performance measure. In many consequential real-world domains, however, optimal performance is not enough to justify an algorithm’s use—for example, sometimes a system’s robustness, stability, or safety must be rigorously ensured. Thus, methods for verifying DRL systems have emerged. These algorithms can guarantee a system’s properties over an infinite set of inputs, but the task is not trivial. DRL relies on deep neural networks (DNNs). DNNs are often referred to as “black boxes” because examining their respective structures does not elucidate their decision-making processes. Moreover, the sequential nature of the problems DRL is used to solve promotes significant scalability challenges. Finally, because DRL environments are often stochastic, verification methods must account for probabilistic behavior. To address these complications, a new subfield has emerged. In this survey, we establish the foundations of DRL and DRL verification, define a taxonomy for DRL verification methods, describe approaches for dealing with stochasticity, characterize considerations related to writing specifications, enumerate common testing tasks/environments, and detail opportunities for future research. 

Academic achievement in the first year of college is critical for setting students on a pathway toward long-term academic and life success, yet little is known about the factors that shape early college academic achievement. Given the important role sleep plays in learning and memory, here we extend this work to evaluate whether nightly sleep duration predicts change in end-of-semester grade point average (GPA). First-year college students from three independent universities provided sleep actigraphy for a month early in their winter/spring academic term across five studies. Findings showed that greater early-term total nightly sleep duration predicted higher end-of-term GPA, an effect that persisted even after controlling for previous-term GPA and daytime sleep. Specifically, every additional hour of average nightly sleep duration early in the semester was associated with an 0.07 increase in end-of-term GPA. Sensitivity analyses using sleep thresholds also indicated that sleeping less than 6 h each night was a period where sleep shifted from helpful to harmful for end-of-term GPA, relative to previous-term GPA. Notably, predictive relationships with GPA were specific to total nightly sleep duration, and not other markers of sleep, such as the midpoint of a student’s nightly sleep window or bedtime timing variability. These findings across five studies establish nightly sleep duration as an important factor in academic success and highlight the potential value of testing early academic term total sleep time interventions during the formative first year of college. 

This paper explores the feasibility of using sonification in delivering and communicating health and wellness status on personal devices. Ambient displays have proven to inform users of their health and wellness and help them to make healthier decisions, yet, little technology provides health assessments through sounds, which can be even more pervasive than visual displays. We developed a method to generate music from user preferences and evaluated it in a two-step user study. In the first step, we acquired general healthiness impressions from each user. In the second step, we generated customized melodies from music preferences in the first step to capture participants' perceived healthiness of those melodies. We deployed our surveys for 55 participants to complete on their own over 31 days. We analyzed the data to understand commonalities and differences in users' perceptions of music as an expression of health. Our findings show the existence of clear associations between perceived healthiness and different music features. We provide useful insights into how different musical features impact the perceived healthiness of music, how perceptions of healthiness vary between users, what trends exist between users' impressions, and what influences (or does not influence) a user's perception of healthiness in a melody. Overall, our results indicate validity in presenting health data through personalized music models. The findings can inform the design of behavior management applications on personal and ubiquitous devices. 

Modeling biological rhythms helps understand the complex principles behind the physical and psychological abnormalities of human bodies, to plan life schedules, and avoid persisting fatigue and mood and sleep alterations due to the desynchronization of those rhythms. The first step in modeling biological rhythms is to identify their characteristics, such as cyclic periods, phases, and amplitude. However, human rhythms are susceptible to external events, which cause irregular fluctuations in waveforms and affect the characterization of each rhythm. In this paper, we present our exploratory work towards developing a computational framework for automated discovery and modeling of human rhythms. We first identify cyclic periods in time series data using three different methods and test their performance on both synthetic data and real fine-grained biological data. We observe consistent periods are detected by all three methods. We then model inner cycles within each period by identifying change points to observe fluctuations in biological data that may inform the impact of external events on human rhythms. The results provide initial insights into the design of a computational framework for discovering and modeling human rhythms. 

This paper presents the first computational framework for modeling biobehavioral rhythms - the repeating cycles of physiological, psychological, social, and environmental events - from mobile and wearable data streams. The framework incorporates four main components: mobile data processing, rhythm discovery, rhythm modeling, and machine learning. We evaluate the framework with two case studies using datasets of smartphones, Fitbit, and OURA smart ring to evaluate the framework's ability to 1) detect cyclic biobehavioral, 2) model commonality and differences in rhythms of human participants in the sample datasets, and 3) predict their health and readiness status using models of biobehavioral rhythms. Our evaluation demonstrates the framework's ability to generate new knowledge and findings through rigorous micro-and macro-level modeling of human rhythms from mobile and wearable data streams collected in the wild and using them to assess and predict different life and health outcomes. 

Continuous passive sensing of daily behavior from mobile devices has the potential to identify behavioral patterns associated with different aspects of human characteristics. This paper presents novel analytic approaches to extract and understand these behavioral patterns and their impact on predicting adaptive and maladaptive personality traits. Our machine learning analysis extends previous research by showing that both adaptive and maladaptive traits can be predicted from passively sensed behavior providing initial evidence for the utility of this type of data to study personality and its pathology. The analysis also provides insights into the underlying behavior patterns that link adaptive and maladaptive variants consistent with contemporary models of personality pathology.