skip to main content


The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Friday, September 29 until 11:59 PM ET on Saturday, September 30 due to maintenance. We apologize for the inconvenience.

Search for: All records

Creators/Authors contains: "Yu, Hao"

Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher. Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?

Some links on this page may take you to non-federal websites. Their policies may differ from this site.

  1. In highly and fully automated vehicles (AV), drivers could divert their attention to non-driving-related activities. Drivers may also take over AVs if they do not trust the way AVs drive in specific driving scenarios. Existing models have been developed to predict drivers’ takeover performance in responding to takeover requests initiated by AVs in semi-AVs. However, few models predicted driver-initiated takeover behavior in highly and fully AVs. The present study develops an attention-based multiple-input Convolutional Neural Network (CNN) to predict drivers’ takeover intention in fully AVs. The results indicated that the developed model successfully predicted takeover intentions of drivers with a precision of 0.982 and an F1-Score of.989, which were found to be substantially higher than other machine learning algorithms. The developed CNN model could be applied in improving the driving algorithms of the AV by considering drivers’ driving styles to reduce drivers’ unnecessary takeover behaviors. 
    more » « less
  2. Free, publicly-accessible full text available February 1, 2024
  3. Free, publicly-accessible full text available December 1, 2023
  4. Live streaming is a form of media that allows streamers to directly interact with their audience. Previous research has explored mental health, and live streaming platforms, and users' social motivations behind watching live streams separately. However, few have explored how these all intertwine in conversations involving intimate, self-disclosing topics, such as mental health. Live streams are unique in that they are largely masspersonal in nature; streamers broadcast themselves to mostly unknown viewers, but may choose to interact with them in a personal way. This study aims to understand users' motivations, preferences, and habits behind participating in mental health discussions on live streams. We interviewed 25 Twitch viewers about the streamers they watch, how they interact in mental health discussions, and how they believe streamers should discuss mental health on live streams. Our findings are contextualized in the dynamics in which these discussions occur. Overall, we found that the innate design of the Twitch platform promotes a user-hierarchy in the ecosystem of streamers and their communities, which may affect how mental health is discussed. 
    more » « less
  5. A major challenge for online learning is the inability of systems to support student emotion and to maintain student engagement. In response to this challenge, computer vision has become an embedded feature in some instructional applications. In this paper, we propose a video dataset of college students solving math problems on the educational platform with a front facing camera collecting visual feedback of student gestures. The video dataset is annotated to indicate whether students’ attention at specific frames is engaged or wandering. In addition, we train baselines for a computer vision module that determines the extent of student engagement during remote learning. Baselines include state-of-the-art deep learning image classifiers and traditional conditional and logistic regression for head pose estimation. We then incorporate a gaze baseline into the MathSpring learning platform, and we are evaluating its performance with the currently implemented approach. 
    more » « less
  6. In this work, we propose a video-based transfer learning approach for predicting problem outcomes of students working with an intelligent tutoring system (ITS). By analyzing a student's face and gestures, our method predicts the outcome of a student answering a problem in an ITS from a video feed. Our work is motivated by the reasoning that the ability to predict such outcomes enables tutoring systems to adjust interventions, such as hints and encouragement, and to ultimately yield improved student learning. We collected a large labeled dataset of student interactions with an intelligent online math tutor consisting of 68 sessions, where 54 individual students solved 2,749 problems. We will release this dataset publicly upon publication of this paper. It will be available at Working with this dataset, our transfer-learning challenge was to design a representation in the source domain of pictures obtained “in the wild” for the task of facial expression analysis, and transferring this learned representation to the task of human behavior prediction in the domain of webcam videos of students in a classroom environment. We developed a novel facial affect representation and a user-personalized training scheme that unlocks the potential of this representation. We designed several variants of a recurrent neural network that models the temporal structure of video sequences of students solving math problems. Our final model, named ATL-BP for Affect Transfer Learning for Behavior Prediction, achieves a relative increase in mean F -score of 50 % over the state-of-the-art method on this new dataset. 
    more » « less