An official website of the United States government
We present a neural network approach for closed-loop deep brain stimulation (DBS). We cast the problem of finding an optimal neurostimulation strategy as a control problem. In this setting, control policies aim to optimize therapeutic outcomes by tailoring the parameters of a DBS system, typically via electrical stimulation, in real time based on the patient’s ongoing neuronal activity. We approximate the value function offline using a neural network to enable generating controls (stimuli) in real time via the feedback form. The neuronal activity is characterized by a nonlinear, stiff system of differential equations as dictated by the Hodgkin-Huxley model. Our training process leverages the relationship between Pontryagin’s maximum principle and Hamilton-Jacobi-Bellman equations to update the value function estimates simultaneously. Our numerical experiments illustrate the accuracy of our approach for out-of-distribution samples and the robustness to moderate shocks and disturbances in the system.
more »
« less
Neural co-processors for restoring brain function: results from a cortical model of grasping
Abstract Objective.A major challenge in designing closed-loop brain-computer interfaces is finding optimal stimulation patterns as a function of ongoing neural activity for different subjects and different objectives. Traditional approaches, such as those currently used for deep brain stimulation, have largely followed a manual trial-and-error strategy to search for effective open-loop stimulation parameters, a strategy that is inefficient and does not generalize to closed-loop activity-dependent stimulation.Approach.To achieve goal-directed closed-loop neurostimulation, we propose the use of brain co-processors, devices which exploit artificial intelligence to shape neural activity and bridge injured neural circuits for targeted repair and restoration of function. Here we investigate a specific type of co-processor called a ‘neural co-processor’ which uses artificial neural networks and deep learning to learn optimal closed-loop stimulation policies. The co-processor adapts the stimulation policy as the biological circuit itself adapts to the stimulation, achieving a form of brain-device co-adaptation. Here we use simulations to lay the groundwork for futurein vivotests of neural co-processors. We leverage a previously published cortical model of grasping, to which we applied various forms of simulated lesions. We used our simulations to develop the critical learning algorithms and study adaptations to non-stationarity in preparation for futurein vivotests.Main results.Our simulations show the ability of a neural co-processor to learn a stimulation policy using a supervised learning approach, and to adapt that policy as the underlying brain and sensors change. Our co-processor successfully co-adapted with the simulated brain to accomplish the reach-and-grasp task after a variety of lesions were applied, achieving recovery towards healthy function in the range 75%–90%.Significance.Our results provide the first proof-of-concept demonstration, using computer simulations, of a neural co-processor for adaptive activity-dependent closed-loop neurostimulation for optimizing a rehabilitation goal after injury. While a significant gap remains between simulations andin vivoapplications, our results provide insights on how such co-processors may eventually be developed for learning complex adaptive stimulation policies for a variety of neural rehabilitation and neuroprosthetic applications.
more »
« less
- Award ID(s):
- 2223495
- PAR ID:
- 10412363
- Publisher / Repository:
- IOP Publishing
- Date Published:
- Journal Name:
- Journal of Neural Engineering
- Volume:
- 20
- Issue:
- 3
- ISSN:
- 1741-2560
- Page Range / eLocation ID:
- Article No. 036004
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Abstract Despite promising advancements, closed-loop neurostimulation for drug-resistant epilepsy (DRE) still relies on manual tuning and produces variable outcomes, while automated predictable algorithms remain an aspiration. As a fundamental step towards addressing this gap, here we study predictive dynamical models of human intracranial EEG (iEEG) response under parametrically rich neurostimulation. Using data fromn= 13 DRE patients, we find that stimulation-triggered switched-linear models with ~300 ms of causal historical dependence best explain evoked iEEG dynamics. These models are highly consistent across different stimulation amplitudes and frequencies, allowing for learning a generalizable model from abundant STIM OFF and limited STIM ON data. Further, evoked iEEG in nearly all subjects exhibited a distance-dependent pattern, whereby stimulationdirectlyimpacts the actuation site and nearby regions (≲ 20 mm), affects medium-distance regions (20 ~ 100 mm) through network interactions, and hardly reaches more distal areas (≳ 100 mm). Peak network interaction occurs at 60 ~ 80 mm from the stimulation site. Due to their predictive accuracy and mechanistic interpretability, these models hold significant potential for model-based seizure forecasting and closed-loop neurostimulation design.more » « less
-
Abstract Advances in technologies that can record and stimulate deep brain activity in humans have led to impactful discoveries within the field of neuroscience and contributed to the development of novel therapies for neurological and psychiatric disorders. Further progress, however, has been hindered by device limitations in that recording of single-neuron activity during freely moving behaviors in humans has not been possible. Additionally, implantable neurostimulation devices, currently approved for human use, have limited stimulation programmability and restricted full-duplex bidirectional capability. In this study, we developed a wearable bidirectional closed-loop neuromodulation system (Neuro-stack) and used it to record single-neuron and local field potential activity during stationary and ambulatory behavior in humans. Together with a highly flexible and customizable stimulation capability, the Neuro-stack provides an opportunity to investigate the neurophysiological basis of disease, develop improved responsive neuromodulation therapies, explore brain function during naturalistic behaviors in humans and, consequently, bridge decades of neuroscientific findings across species.more » « less
-
Neurostimulation - the practice of applying exogenous excitation, e.g., via electrical current, to the brain - has been used for decades in clinical applications such as the treatment of motor disorders and neuropsychiatric illnesses. Over the past several years, more emphasis has been placed on understanding and designing neurostimulation from a systems-theoretic perspective, so as to better optimize its use. Particular questions of interest have included designing stimulation waveforms that best induce certain patterns of brain activity while minimizing expenditure of stimulus power. The pursuit of these designs faces a fundamental conundrum, insofar as they presume that the desired pattern (e.g., desyn-chronization of a neural population) is known a priori. In this paper, we present an alternative paradigm wherein the goal of the stimulation is not to induce a prescribed pattern, but rather to simply improve the functionality of the stimulated circuit/system. Here, the notion of functionality is defined in terms of an information-theoretic objective. Specifically, we seek closed loop control designs that maximize the ability of a controlled circuit to encode an afferent `hidden input,' without prescription of dynamics or output. In this way, the control attempts only to make the system `effective' without knowing beforehand the dynamics that are needed to be induced. We devote most of our effort to defining this framework mathematically, providing algorithmic procedures that demonstrate its solution and interpreting the results of this procedure for simple, prototypical dynamical systems. Simulation results are provided for more complex models, including an example involving control of a canonical neural mass model.more » « less
-
Abstract Experimental neuroscience techniques are advancing rapidly, with major recent developments in high-density electrophysiology and targeted electrical stimulation. In combination with these techniques, cortical organoids derived from pluripotent stem cells show great promise asin vitromodels of brain development and function. Although sensory input is vital to neurodevelopmentin vivo, few studies have explored the effect of meaningful input toin vitroneural cultures over time. In this work, we demonstrate the first example of goal-directed learning in brain organoids. We developed a closed-loop electrophysiology framework to embody mouse cortical organoids into a simulated dynamical task (the inverted pendulum problem known as ‘Cartpole’) and evaluate learning through high-frequency training signals. Longitudinal experiments enabled by this framework illuminate how different methods of selecting training signals enable improvement on the tasks. We found that for most organoids, training signals chosen by artificial reinforcement learning yield better performance on the task than randomly chosen training signals or the absence of a training signal. This systematic approach to studying learning mechanismsin vitroopens new possibilities for therapeutic interventions and biological computation.more » « less
