More Like this

1. Robust walking control of a lower limb rehabilitation exoskeleton coupled with a musculoskeletal model via deep reinforcement learning

   https://doi.org/10.1186/s12984-023-01147-2  

   Luo, Shuzhen ; Androwis, Ghaith ; Adamovich, Sergei ; Nunez, Erick ; Su, Hao ; Zhou, Xianlian ( December 2023 , Journal of NeuroEngineering and Rehabilitation)

   Abstract Background Few studies have systematically investigated robust controllers for lower limb rehabilitation exoskeletons (LLREs) that can safely and effectively assist users with a variety of neuromuscular disorders to walk with full autonomy. One of the key challenges for developing such a robust controller is to handle different degrees of uncertain human-exoskeleton interaction forces from the patients. Consequently, conventional walking controllers either are patient-condition specific or involve tuning of many control parameters, which could behave unreliably and even fail to maintain balance. Methods We present a novel, deep neural network, reinforcement learning-based robust controller for a LLRE based on a decoupled offline human-exoskeleton simulation training with three independent networks, which aims to provide reliable walking assistance against various and uncertain human-exoskeleton interaction forces. The exoskeleton controller is driven by a neural network control policy that acts on a stream of the LLRE’s proprioceptive signals, including joint kinematic states, and subsequently predicts real-time position control targets for the actuated joints. To handle uncertain human interaction forces, the control policy is trained intentionally with an integrated human musculoskeletal model and realistic human-exoskeleton interaction forces. Two other neural networks are connected with the control policy network to predict the interaction forces and musclemore »coordination. To further increase the robustness of the control policy to different human conditions, we employ domain randomization during training that includes not only randomization of exoskeleton dynamics properties but, more importantly, randomization of human muscle strength to simulate the variability of the patient’s disability. Through this decoupled deep reinforcement learning framework, the trained controller of LLREs is able to provide reliable walking assistance to patients with different degrees of neuromuscular disorders without any control parameter tuning. Results and conclusion A universal, RL-based walking controller is trained and virtually tested on a LLRE system to verify its effectiveness and robustness in assisting users with different disabilities such as passive muscles (quadriplegic), muscle weakness, or hemiplegic conditions without any control parameter tuning. Analysis of the RMSE for joint tracking, CoP-based stability, and gait symmetry shows the effectiveness of the controller. An ablation study also demonstrates the strong robustness of the control policy under large exoskeleton dynamic property ranges and various human-exoskeleton interaction forces. The decoupled network structure allows us to isolate the LLRE control policy network for testing and sim-to-real transfer since it uses only proprioception information of the LLRE (joint sensory state) as the input. Furthermore, the controller is shown to be able to handle different patient conditions without the need for patient-specific control parameter tuning.« less
2. Dynamics and Control of a Magnetic Transducer Array Using Multi-Physics Models and Artificial Neural Networks

   https://doi.org/10.3390/s21206788  

   Tsakyridis, Georgios ; Xiros, Nikolaos I. ( October 2021 , Sensors)

   A linear mechanical oscillator is non-linearly coupled with an electromagnet and its driving circuit through a magnetic field. The resulting non-linear dynamics are investigated using magnetic circuit approximations without major loss of accuracy and in the interest of brevity. Different computational approaches to simulate the setup in terms of dynamical system response and design parameters optimization are pursued. A current source operating in baseband without modulation directly feeds the electromagnet, which consists commonly of a solenoid and a horseshoe-shaped core. The electromagnet is then magnetically coupled to a mass made of soft magnetic material and attached to a spring with damping. The non-linear system is described by a linearized steady-space representation while is examined for controllability and observability. A controller using a pole placement approach is built to stabilize the element. Drawing upon the fact that coupling works both ways, enabling estimation of the mass position and velocity (state variables) by processing the induced voltage across the electromagnet, a state observer is constructed. Accurate and fast tracking of the state variables, along with the possibility of driving more than one module from the same source using modulation, proves the applicability of the electro-magneto-mechanical transducer for sensor applications. Next, a three-layermore »feed-forward artificial neural network (ANN) system equivalent was trained using the non-linear plant-linear controller-linear observer configuration. Simulations to investigate the robustness of the system with respect to different equilibrium points and input currents were carried out. The ANN proved robust with respect to position accuracy.« less
3. Absolutely robust controllers for chemical reaction networks

   https://doi.org/10.1098/rsif.2020.0031  

   Kim, Jinsu ; Enciso, German ( May 2020 , Journal of The Royal Society Interface)

   In this work, we design a type of controller that consists of adding a specific set of reactions to an existing mass-action chemical reaction network in order to control a target species. This set of reactions is effective for both deterministic and stochastic networks, in the latter case controlling the mean as well as the variance of the target species. We employ a type of network property called absolute concentration robustness (ACR). We provide applications to the control of a multisite phosphorylation model as well as a receptor–ligand signalling system. For this framework, we use the so-called deficiency zero theorem from chemical reaction network theory as well as multiscaling model reduction methods. We show that the target species has approximately Poisson distribution with the desired mean. We further show that ACR controllers can bring robust perfect adaptation to a target species and are complementary to a recently introduced antithetic feedback controller used for stochastic chemical reactions.
4. Risk of Stochastic Systems for Temporal Logic Specifications

   https://doi.org/10.1145/3580490  

   Lindemann, Lars ; Jiang, Lejun ; Matni, Nikolai ; Pappas, George J. ( May 2023 , ACM Transactions on Embedded Computing Systems)

   The wide availability of data coupled with the computational advances in artificial intelligence and machine learning promise to enable many future technologies such as autonomous driving. While there has been a variety of successful demonstrations of these technologies, critical system failures have repeatedly been reported. Even if rare, such system failures pose a serious barrier to adoption without a rigorous risk assessment. This article presents a framework for the systematic and rigorous risk verification of systems. We consider a wide range of system specifications formulated in signal temporal logic (STL) and model the system as a stochastic process, permitting discrete-time and continuous-time stochastic processes. We then define the STL robustness risk as the risk of lacking robustness against failure . This definition is motivated as system failures are often caused by missing robustness to modeling errors, system disturbances, and distribution shifts in the underlying data generating process. Within the definition, we permit general classes of risk measures and focus on tail risk measures such as the value-at-risk and the conditional value-at-risk. While the STL robustness risk is in general hard to compute, we propose the approximate STL robustness risk as a more tractable notion that upper bounds the STL robustnessmore »risk. We show how the approximate STL robustness risk can accurately be estimated from system trajectory data. For discrete-time stochastic processes, we show under which conditions the approximate STL robustness risk can even be computed exactly. We illustrate our verification algorithm in the autonomous driving simulator CARLA and show how a least risky controller can be selected among four neural network lane-keeping controllers for five meaningful system specifications.« less
5. Reinforcement Learning for Spacecraft Attitude Control

   Vedant, James T ( October 2019 , 70th International Astronautical Congress)

   Reinforcement learning (RL) has recently shown promise in solving difficult numerical problems and has discovered non-intuitive solutions to existing problems. This study investigates the ability of a general RL agent to find an optimal control strategy for spacecraft attitude control problems. Two main types of Attitude Control Systems (ACS) are presented. First, the general ACS problem with full actuation is considered, but with saturation constraints on the applied torques, representing thruster-based ACSs. Second, an attitude control problem with reaction wheel based ACS is considered, which has more constraints on control authority. The agent is trained using the Proximal Policy Optimization (PPO) RL method to obtain an attitude control policy. To ensure robustness, the inertia of the satellite is unknown to the control agent and is randomized for each simulation. To achieve efficient learning, the agent is trained using curriculum learning. We compare the RL based controller to a QRF (quaternion rate feedback) attitude controller, a well-established state feedback control strategy. We investigate the nominal performance and robustness with respect to uncertainty in system dynamics. Our RL based attitude control agent adapts to any spacecraft mass without needing to re-train. In the range of 0.1 to 100,000 kg, our agent achievesmore »2% better performance to a QRF controller tuned for the same mass range, and similar performance to the QRF controller tuned specifically for a given mass. The performance of the trained RL agent for the reaction wheel based ACS achieved 10 higher better reward then that of a tuned QRF controller« less