Attention:The NSF Public Access Repository (PAR) system and access will be unavailable from 5:00 PM ET until 8:00 PM ET on Friday, September 11 due to maintenance. We apologize for the inconvenience.

Explore Research Products in the PAR
It may take a few hours for recently added research products to appear in PAR search results.


  • NSF Public Access
  • Search Results
  • Assessing the impact of context inference error and partial observability on RL methods for just-in-time adaptive interventions
Title: Assessing the impact of context inference error and partial observability on RL methods for just-in-time adaptive interventions
Just-in-Time Adaptive Interventions (JITAIs) are a class of personalized health interventions developed within the behavioral science community. JITAIs aim to provide the right type and amount of support by iteratively selecting a sequence of intervention options from a pre-defined set of components in response to each individual's time varying state. In this work, we explore the application of reinforcement learning methods to the problem of learning intervention option selection policies. We study the effect of context inference error and partial observability on the ability to learn effective policies. Our results show that the propagation of uncertainty from context inferences is critical to improving intervention efficacy as context uncertainty increases, while policy gradient algorithms can provide remarkable robustness to partially observed behavioral state information.  more » « less
Award ID(s):
1722792
PAR ID:
10488020
Author(s) / Creator(s):
; ; ;
Editor(s):
Evans, Robin J.; Shpitser, Ilya
Publisher / Repository:
Proceedings of Machine Learning Research
Date Published:
Journal Name:
UAI '23: Proceedings of the Thirty-Ninth Conference on Uncertainty in Artificial Intelligence
Volume:
216
Page Range / eLocation ID:
1047-1057
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. ; ; ; ; ; ; ; ; ; (, Med Research)
    ABSTRACT Advances in digital phenotyping have opened the door to continuous, individualised monitoring of mental health, but realising the full potential of these data demands machine learning models that can operate effectively in ‘small‐data’ regimes—where per‐user data are sparse, irregular and noisy. This article explores the feasibility, challenges and opportunities of small‐data machine learning approaches for forecasting individual‐level mental health trajectories. We examine the limitations of traditional clinical tools and population‐level models and argue that fine‐grained time‐series forecasting, powered by models such as tabular prior‐data fitted networks (TabPFN), Gaussian processes, Kalman filters and meta‐learning strategies, offers a path towards personalised, proactive psychiatry. Emphasis is placed on key clinical requirements: real‐time adaptation, uncertainty quantification, feature‐level interpretability and respect for interindividual variability. We discuss implementation barriers including data quality, model transparency and ethical considerations and propose practical pathways for deployment—such as integrated biosensor platforms and just‐in‐time adaptive interventions (JITAIs). We highlight the emerging convergence of small‐data ML, mobile sensing and clinical insight as a transformative force in mental healthcare. With interdisciplinary collaboration and prospective validation, these technologies have the potential to shift psychiatry from reactive symptom management to anticipatory, personalised intervention. 
    more » « less
  2. ; ; (, Proceedings of the National Academy of Sciences)
    Choice context influences decision processes and is one of the primary determinants of what people choose. This insight has been used by academics and practitioners to study decision biases and to design behavioral interventions to influence and improve choices. We analyzed the effects of context-based behavioral interventions on the computational mechanisms underlying decision-making. We collected data from two large laboratory studies involving 19 prominent behavioral interventions, and we modeled the influence of each intervention using a leading computational model of choice in psychology and neuroscience. This allowed us to parametrize the biases induced by each intervention, to interpret these biases in terms of underlying decision mechanisms and their properties, to quantify similarities between interventions, and to predict how different interventions alter key choice outcomes. In doing so, we offer researchers and practitioners a theoretically principled approach to understanding and manipulating choice context in decision-making. 
    more » « less
  3. ; ; ; ; ; (, International Conference on Learning Research 2026 the 2nd Workshop on World Models)
    World models have emerged as a unifying paradigm for learning latent dynamics, simulating counterfactual futures, and supporting planning under uncertainty. In this paper, we argue that computational epidemiology is a natural and underdeveloped setting for world models. This is because epidemic decision-making requires reasoning about latent disease burden, imperfect and policy-dependent surveillance signals, and intervention effects are mediated by adaptive human behavior. We introduce a conceptual framework for epidemiological world models, formulating epidemics as controlled, partially observed dynamical systems in which (i) the true epidemic state is latent, (ii) observations are noisy and endogenous to policy, and (iii) interventions act as sequential actions whose effects propagate through behavioral and social feedback. We present three case studies that illustrate why explicit world modeling is necessary for policy-relevant reasoning: strategic misreporting in behavioral surveillance, systematic delays in time-lagged signals such as hospitalizations and deaths, and counterfactual intervention analysis where identical histories diverge under alternative action sequences. 
    more » « less
  4. ; ; ; ; ; ; ; ; (, Proceedings of the National Academy of Sciences)
    Online education is rapidly expanding in response to rising demand for higher and continuing education, but many online students struggle to achieve their educational goals. Several behavioral science interventions have shown promise in raising student persistence and completion rates in a handful of courses, but evidence of their effectiveness across diverse educational contexts is limited. In this study, we test a set of established interventions over 2.5 y, with one-quarter million students, from nearly every country, across 247 online courses offered by Harvard, the Massachusetts Institute of Technology, and Stanford. We hypothesized that the interventions would produce medium-to-large effects as in prior studies, but this is not supported by our results. Instead, using an iterative scientific process of cyclically preregistering new hypotheses in between waves of data collection, we identified individual, contextual, and temporal conditions under which the interventions benefit students. Self-regulation interventions raised student engagement in the first few weeks but not final completion rates. Value-relevance interventions raised completion rates in developing countries to close the global achievement gap, but only in courses with a global gap. We found minimal evidence that state-of-the-art machine learning methods can forecast the occurrence of a global gap or learn effective individualized intervention policies. Scaling behavioral science interventions across various online learning contexts can reduce their average effectiveness by an order-of-magnitude. However, iterative scientific investigations can uncover what works where for whom. 
    more » « less
  5. ; ; ; ; ; (, Chaos, Solitons & Fractals)
    The ongoing COVID-19 pandemic has inflicted tremendous economic and societal losses. In the absence of pharmaceutical interventions, the population behavioral response, including situational awareness and adherence to non-pharmaceutical intervention policies, has a significant impact on contagion dynamics. Game-theoretic models have been used to reproduce the concurrent evolution of behavioral responses and disease contagion, and social networks are critical platforms on which behavior imitation between social contacts, even dispersed in distant communities, takes place. Such joint contagion dynamics has not been sufficiently explored, which poses a challenge for policies aimed at containing the infection. In this study, we present a multi-layer network model to study contagion dynamics and behavioral adaptation. It comprises two physical layers that mimic the two solitary communities, and one social layer that encapsulates the social influence of agents from these two communities. Moreover, we adopt high-order interactions in the form of simplicial complexes on the social influence layer to delineate the behavior imitation of individual agents. This model offers a novel platform to articulate the interaction between physically isolated communities and the ensuing coevolution of behavioral change and spreading dynamics. The analytical insights harnessed therefrom provide compelling guidelines on coordinated policy design to enhance the preparedness for future pandemics. 
    more » « less
  • Citation Formats
  • MLA
  • APA
  • Chicago
  • BibTeX
  • Text Encoded Conversion
  • XML
  • Save / Share this Record
  • Send to Email