Note: When clicking on a Digital Object Identifier (DOI) number, you will be taken to an external site maintained by the publisher.
Some full text articles may not yet be available without a charge during the embargo (administrative interval).
What is a DOI Number?
Some links on this page may take you to non-federal websites. Their policies may differ from this site.
-
Free, publicly-accessible full text available May 11, 2025
-
Advances in large language models (LLMs) have empowered a variety of applications. However, there is still a significant gap in research when it comes to understanding and enhancing the capabilities of LLMs in the field of mental health. In this work, we present a comprehensive evaluation of multiple LLMs on various mental health prediction tasks via online text data, including Alpaca, Alpaca-LoRA, FLAN-T5, GPT-3.5, and GPT-4. We conduct a broad range of experiments, covering zero-shot prompting, few-shot prompting, and instruction fine-tuning. The results indicate a promising yet limited performance of LLMs with zero-shot and few-shot prompt designs for mental health tasks. More importantly, our experiments show that instruction finetuning can significantly boost the performance of LLMs for all tasks simultaneously. Our best-finetuned models, Mental-Alpaca and Mental-FLAN-T5, outperform the best prompt design of GPT-3.5 (25 and 15 times bigger) by 10.9% on balanced accuracy and the best of GPT-4 (250 and 150 times bigger) by 4.8%. They further perform on par with the state-of-the-art task-specific language model. We also conduct an exploratory case study on LLMs' capability on mental health reasoning tasks, illustrating the promising capability of certain models such as GPT-4. We summarize our findings into a set of action guidelines for potential methods to enhance LLMs' capability for mental health tasks. Meanwhile, we also emphasize the important limitations before achieving deployability in real-world mental health settings, such as known racial and gender bias. We highlight the important ethical risks accompanying this line of research.more » « lessFree, publicly-accessible full text available March 6, 2025
-
Free, publicly-accessible full text available January 1, 2025
-
Academic achievement in the first year of college is critical for setting students on a pathway toward long-term academic and life success, yet little is known about the factors that shape early college academic achievement. Given the important role sleep plays in learning and memory, here we extend this work to evaluate whether nightly sleep duration predicts change in end-of-semester grade point average (GPA). First-year college students from three independent universities provided sleep actigraphy for a month early in their winter/spring academic term across five studies. Findings showed that greater early-term total nightly sleep duration predicted higher end-of-term GPA, an effect that persisted even after controlling for previous-term GPA and daytime sleep. Specifically, every additional hour of average nightly sleep duration early in the semester was associated with an 0.07 increase in end-of-term GPA. Sensitivity analyses using sleep thresholds also indicated that sleeping less than 6 h each night was a period where sleep shifted from helpful to harmful for end-of-term GPA, relative to previous-term GPA. Notably, predictive relationships with GPA were specific to total nightly sleep duration, and not other markers of sleep, such as the midpoint of a student’s nightly sleep window or bedtime timing variability. These findings across five studies establish nightly sleep duration as an important factor in academic success and highlight the potential value of testing early academic term total sleep time interventions during the formative first year of college.more » « less
-
There is a growing body of research revealing that longitudinal passive sensing data from smartphones and wearable devices can capture daily behavior signals for human behavior modeling, such as depression detection. Most prior studies build and evaluate machine learning models using data collected from a single population. However, to ensure that a behavior model can work for a larger group of users, its generalizability needs to be verified on multiple datasets from different populations. We present the first work evaluating cross-dataset generalizability of longitudinal behavior models, using depression detection as an application. We collect multiple longitudinal passive mobile sensing datasets with over 500 users from two institutes over a two-year span, leading to four institute-year datasets. Using the datasets, we closely re-implement and evaluated nine prior depression detection algorithms. Our experiment reveals the lack of model generalizability of these methods. We also implement eight recently popular domain generalization algorithms from the machine learning community. Our results indicate that these methods also do not generalize well on our datasets, with barely any advantage over the naive baseline of guessing the majority. We then present two new algorithms with better generalizability. Our new algorithm, Reorder, significantly and consistently outperforms existing methods on most cross-dataset generalization setups. However, the overall advantage is incremental and still has great room for improvement. Our analysis reveals that the individual differences (both within and between populations) may play the most important role in the cross-dataset generalization challenge. Finally, we provide an open-source benchmark platform GLOBEM- short for Generalization of Longitudinal BEhavior Modeling - to consolidate all 19 algorithms. GLOBEM can support researchers in using, developing, and evaluating different longitudinal behavior modeling methods. We call for researchers' attention to model generalizability evaluation for future longitudinal human behavior modeling studies.more » « less
-
Smartphone overuse is related to a variety of issues such as lack of sleep and anxiety. We explore the application of Self-Affirmation Theory on smartphone overuse intervention in a just-in-time manner. We present TypeOut, a just-in-time intervention technique that integrates two components: an in-situ typing-based unlock process to improve user engagement, and self-affirmation-based typing con- tent to enhance effectiveness. We hypothesize that the integration of typing and self-affirmation content can better reduce smartphone overuse. We conducted a 10-week within-subject field experiment (N=54) and compared TypeOut against two baselines: one only showing the self-affirmation content (a common notification-based intervention), and one only requiring typing non-semantic content (a state-of-the-art method). TypeOut reduces app usage by over 50%, and both app opening frequency and usage duration by over 25%, all significantly outperforming baselines. TypeOut can potentially be used in other domains where an intervention may benefit from integrating self-affirmation exercises with an engaging just-in-time mechanism.more » « less
-
Abstract Taking an action research approach, we engaged in fieldwork with school-based behavioral health care teams to: observe record keeping practices, design and deploy a prototype system addressing key challenges, and reflect on its use. We describe the challenges of capturing behavioral data using both paper and electronic records. Creating records of behaviors requires direct observation, and as a result the record keeping responsibility is challenging to distribute across a care team. Behavioral data on paper must be transferred and prepared for reporting, both inside the organization and to stakeholders outside of the organization. In prototyping a computerized working record, we targeted user needs for capturing details of a behavioral incident in the moment. Challenges persisted through the transition from paper to our prototype, and based on these empirical findings over two years of fieldwork, we present five tensions in representing behavioral data in an electronic health record. These tensions reflect the differences between entering behavioral data into the record for intraorganizational use versus interorganizational use.more » « less