Abstract Background Few studies have systematically investigated robust controllers for lower limb rehabilitation exoskeletons (LLREs) that can safely and effectively assist users with a variety of neuromuscular disorders to walk with full autonomy. One of the key challenges for developing such a robust controller is to handle different degrees of uncertain human-exoskeleton interaction forces from the patients. Consequently, conventional walking controllers either are patient-condition specific or involve tuning of many control parameters, which could behave unreliably and even fail to maintain balance. Methods We present a novel, deep neural network, reinforcement learning-based robust controller for a LLRE based on a decoupled offline human-exoskeleton simulation training with three independent networks, which aims to provide reliable walking assistance against various and uncertain human-exoskeleton interaction forces. The exoskeleton controller is driven by a neural network control policy that acts on a stream of the LLRE’s proprioceptive signals, including joint kinematic states, and subsequently predicts real-time position control targets for the actuated joints. To handle uncertain human interaction forces, the control policy is trained intentionally with an integrated human musculoskeletal model and realistic human-exoskeleton interaction forces. Two other neural networks are connected with the control policy network to predict the interaction forces and muscle coordination. To further increase the robustness of the control policy to different human conditions, we employ domain randomization during training that includes not only randomization of exoskeleton dynamics properties but, more importantly, randomization of human muscle strength to simulate the variability of the patient’s disability. Through this decoupled deep reinforcement learning framework, the trained controller of LLREs is able to provide reliable walking assistance to patients with different degrees of neuromuscular disorders without any control parameter tuning. Results and conclusion A universal, RL-based walking controller is trained and virtually tested on a LLRE system to verify its effectiveness and robustness in assisting users with different disabilities such as passive muscles (quadriplegic), muscle weakness, or hemiplegic conditions without any control parameter tuning. Analysis of the RMSE for joint tracking, CoP-based stability, and gait symmetry shows the effectiveness of the controller. An ablation study also demonstrates the strong robustness of the control policy under large exoskeleton dynamic property ranges and various human-exoskeleton interaction forces. The decoupled network structure allows us to isolate the LLRE control policy network for testing and sim-to-real transfer since it uses only proprioception information of the LLRE (joint sensory state) as the input. Furthermore, the controller is shown to be able to handle different patient conditions without the need for patient-specific control parameter tuning.
more »
« less
Robust decentralized frequency control: A leaky integrator approach
We investigate the robustness of the
so-called leaky integral frequency controller for the
power network. In particular, using a strict Lyapunov
function, we show the closed-loop system is robust
in the input-to-state stability sense to measurement
noise in the controller. Moreover, an interesting and
explicit trade-o between controller performance and
robustness is discussed and illustrated using a bench-
mark study of the 39-bus New England reference
network.
more »
« less
- PAR ID:
- 10078988
- Date Published:
- Journal Name:
- European Control Conference
- Page Range / eLocation ID:
- 764 to 769
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Deterministic and stochastic approaches to handle uncertainties may incur very different complexities in computation and memory, in addition to different uncertainty models. For linear systems with delay and rate constrained communications between the observer and controller, previous work shows that the deterministic approach l_infty control has low complexity but only handles bounded disturbance. In this paper, we take a stochastic approach and propose an LQ controller that can handle arbitrarily large disturbance but has large complexity in time/space. The differences in robustness and complexity of the l_infty and LQ controllers motivate the design of a hybrid controller that interpolates between the two: The l_infty controller is applied when the disturbance is not too large (normal mode) and the LQ controller is resorted to otherwise (acute mode). We characterize the switching behavior between the normal and acute modes. Using theoretical bounds and supplementary numerical experiments, we show that the hybrid controller can achieve a sweet spot in robustness-complexity tradeoff, ie, reject occasional large disturbance while operating with low complexity most of the time.more » « less
-
Software-Defined Networking (SDN) is a dynamic, and manageable network architecture which is more cost-effective than existing network architectures. The idea behind this architecture is to centralize intelligence from the network hardware and funnel this intelligence to the management system (controller) [2]-[4]. Since the centralized SDN controller controls the entire network and manages policies and the flow of the traffic throughout the network, it can be considered as the single point of failure [1]. It is important to find some ways to identify different types of attacks on the SDN controller [8]. Distributed Denial of Service (DDoS) attack is one of the most dangerous attacks on SDN controller. In this work, we implement DDoS attack on the Ryu controller in a tree network topology using Mininet emulator. Also, we use a machine learning method, Vector Machines (SVM) to detect DDoS attack. We propose to install flows in switches, and we consider time attack pattern of the DDoS attack for detection. Simulation results show the effects of DDoS attacks on the Ryu controller is reduced by 36% using our detection method.more » « less
-
null (Ed.)In this work, we design a type of controller that consists of adding a specific set of reactions to an existing mass-action chemical reaction network in order to control a target species. This set of reactions is effective for both deterministic and stochastic networks, in the latter case controlling the mean as well as the variance of the target species. We employ a type of network property called absolute concentration robustness (ACR). We provide applications to the control of a multisite phosphorylation model as well as a receptor–ligand signalling system. For this framework, we use the so-called deficiency zero theorem from chemical reaction network theory as well as multiscaling model reduction methods. We show that the target species has approximately Poisson distribution with the desired mean. We further show that ACR controllers can bring robust perfect adaptation to a target species and are complementary to a recently introduced antithetic feedback controller used for stochastic chemical reactions.more » « less
-
Reinforcement learning (RL) has recently shown promise in solving difficult numerical problems and has discovered non-intuitive solutions to existing problems. This study investigates the ability of a general RL agent to find an optimal control strategy for spacecraft attitude control problems. Two main types of Attitude Control Systems (ACS) are presented. First, the general ACS problem with full actuation is considered, but with saturation constraints on the applied torques, representing thruster-based ACSs. Second, an attitude control problem with reaction wheel based ACS is considered, which has more constraints on control authority. The agent is trained using the Proximal Policy Optimization (PPO) RL method to obtain an attitude control policy. To ensure robustness, the inertia of the satellite is unknown to the control agent and is randomized for each simulation. To achieve efficient learning, the agent is trained using curriculum learning. We compare the RL based controller to a QRF (quaternion rate feedback) attitude controller, a well-established state feedback control strategy. We investigate the nominal performance and robustness with respect to uncertainty in system dynamics. Our RL based attitude control agent adapts to any spacecraft mass without needing to re-train. In the range of 0.1 to 100,000 kg, our agent achieves 2% better performance to a QRF controller tuned for the same mass range, and similar performance to the QRF controller tuned specifically for a given mass. The performance of the trained RL agent for the reaction wheel based ACS achieved 10 higher better reward then that of a tuned QRF controllermore » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.23919/ECC.2018.8550060
