More Like this

1. Flood mitigation in coastal urban catchments using real-time stormwater infrastructure control and reinforcement learning

   https://doi.org/10.2166/hydro.2020.080  

   Bowes, Benjamin D. ; Tavakoli, Arash ; Wang, Cheng ; Heydarian, Arsalan ; Behl, Madhur ; Beling, Peter A. ; Goodall, Jonathan L. ( October 2020 , Journal of Hydroinformatics)

   null (Ed.)

   Abstract Flooding in coastal cities is increasing due to climate change and sea-level rise, stressing the traditional stormwater systems these communities rely on. Automated real-time control (RTC) of these systems can improve performance, and creating control policies for smart stormwater systems is an active area of study. This research explores reinforcement learning (RL) to create control policies to mitigate flood risk. RL is trained using a model of hypothetical urban catchments with a tidal boundary and two retention ponds with controllable valves. RL's performance is compared to the passive system, a model predictive control (MPC) strategy, and a rule-based control strategy (RBC). RL learns to proactively manage pond levels using current and forecast conditions and reduced flooding by 32% over the passive system. Compared to the MPC approach using a physics-based model and genetic algorithm, RL achieved nearly the same flood reduction, just 3% less than MPC, with a significant 88× speedup in runtime. Compared to RBC, RL was able to quickly learn similar control strategies and reduced flooding by an additional 19%. This research demonstrates that RL can effectively control a simple system and offers a computationally efficient method that could scale to RTC of more complex stormwater systems. 

   more » « less
2. Reinforcement learning-based real-time control of coastal urban stormwater systems to mitigate flooding and improve water quality

   https://doi.org/10.1039/D1EW00582K  

   Bowes, Benjamin D. ; Wang, Cheng ; Ercan, Mehmet B. ; Culver, Teresa B. ; Beling, Peter A. ; Goodall, Jonathan L. ( March 2022 , Environmental Science: Water Research & Technology)

   Real-time control of stormwater systems can reduce flooding and improve water quality. Current industry real-time control strategies use simple rules based on water quantity parameters at a local scale. However, system-level control methods that also incorporate observations of water quality could provide improved control and performance. Therefore, the objective of this research is to evaluate the impact of local and system-level control approaches on flooding and sediment-related water quality in a stormwater system within the flood-prone coastal city of Norfolk, Virginia, USA. Deep reinforcement learning (RL), an emerging machine learning technique, is used to learn system-level control policies that attempt to balance flood mitigation and treatment of sediment. RL is compared to the conventional stormwater system and two methods of local-scale rule-based control: (i) industry standard predictive rule-based control with a fixed detention time and (ii) rules based on water quality observations. For the studied system, both methods of rule-based control improved water quality compared to the passive system, but increased total system flooding due to uncoordinated releases of stormwater. An RL agent learned controls that maintained target pond levels while reducing total system flooding by 4% compared to the passive system. When pre-trained from the RL agent that learned to reduce flooding, another RL agent was able to learn to decrease TSS export by an average of 52% compared to the passive system and with an average of 5% less flooding than the rule-based control methods. As the complexity of stormwater RTC implementations grows and climate change continues, system-level control approaches such as the RL used here will be needed to help mitigate flooding and protect water quality. 

   more » « less
3. Deep Reinforcement Learning with Uncertain Data for Real-Time Stormwater System Control and Flood Mitigation

   https://doi.org/10.3390/w12113222  

   Saliba, Sami M. ; Bowes, Benjamin D. ; Adams, Stephen ; Beling, Peter A. ; Goodall, Jonathan L. ( November 2020 , Water)

   null (Ed.)

   Flooding in many areas is becoming more prevalent due to factors such as urbanization and climate change, requiring modernization of stormwater infrastructure. Retrofitting standard passive systems with controllable valves/pumps is promising, but requires real-time control (RTC). One method of automating RTC is reinforcement learning (RL), a general technique for sequential optimization and control in uncertain environments. The notion is that an RL algorithm can use inputs of real-time flood data and rainfall forecasts to learn a policy for controlling the stormwater infrastructure to minimize measures of flooding. In real-world conditions, rainfall forecasts and other state information are subject to noise and uncertainty. To account for these characteristics of the problem data, we implemented Deep Deterministic Policy Gradient (DDPG), an RL algorithm that is distinguished by its capability to handle noise in the input data. DDPG implementations were trained and tested against a passive flood control policy. Three primary cases were studied: (i) perfect data, (ii) imperfect rainfall forecasts, and (iii) imperfect water level and forecast data. Rainfall episodes (100) that caused flooding in the passive system were selected from 10 years of observations in Norfolk, Virginia, USA; 85 randomly selected episodes were used for training and the remaining 15 unseen episodes served as test cases. Compared to the passive system, all RL implementations reduced flooding volume by 70.5% on average, and performed within a range of 5%. This suggests that DDPG is robust to noisy input data, which is essential knowledge to advance the real-world applicability of RL for stormwater RTC. 

   more » « less
4. Leveraging Open Source Software and Parallel Computing for Model Predictive Control Simulation of Urban Drainage Systems Using EPA-SWMM5 and Python

   https://doi.org/10.1007/978-3-319-99867-1_170  

   Sadler J.M., Goodall J.L. ( January 2019 , UDM 2018: New Trends in Urban Drainage Modelling)

   The active control of stormwater systems is a potential solution to increased street flooding in low-lying, low-relief coastal cities due to climate change and accompanying sea level rise. Model predictive control (MPC) has been shown to be a successful control strategy generally and as well as for managing urban drainage specifically. This research describes and demonstrates the implementation of MPC for urban drainage systems using open source software (Python and The United States Environmental Protection Agency (EPA) Storm Water Management Model (SWMM5). The system was demonstrated using a simplified use case in which an actively-controlled outlet of a detention pond is simulated. The control of the pond’s outlet influences the flood risk of a downstream node. For each step in the SWMM5 model, a series of policies for controlling the outlet are evaluated. The best policy is then selected using an evolutionary algorithm. The policies are evaluated against an objective function that penalizes primarily flooding and secondarily deviation of the detention pond level from a target level. Freely available Python libraries provide the key functionality for the MPC workflow: step-by-step running of the SWMM5 simulation, evolutionary algorithm implementation, and leveraging parallel computing. For perspective, the MPC results were compared to results from a rule-based approach and a scenario with no active control. The MPC approach produced a control policy that largely eliminated flooding (unlike the scenario with no active control) and maintained the detention pond’s water level closer to a target level (unlike the rule-based approach). 

   more » « less
5. Mining least privilege attribute based access control policies

   https://doi.org/10.1145/3359789.3359805  

   Sanders, Matthew W ; Yue, Chuan ( December 2019 , Annual Computer Security Applications Conference)

   Creating effective access control policies is a significant challenge to many organizations. Over-privilege increases security risk from compromised credentials, insider threats, and accidental misuse. Under-privilege prevents users from performing their duties. Policies must balance between these competing goals of minimizing under-privilege vs. over-privilege. The Attribute Based Access Control (ABAC) model has been gaining popularity in recent years because of its advantages in granularity, flexibility, and usability. ABAC allows administrators to create policies based on attributes of users, operations, resources, and the environment. However, in practice, it is often very difficult to create effective ABAC policies in terms of minimizing under-privilege and over-privilege especially for large and complex systems because their ABAC privilege spaces are typically gigantic. In this paper, we take a rule mining approach to mine systems' audit logs for automatically generating ABAC policies which minimize both under-privilege and over-privilege. We propose a rule mining algorithm for creating ABAC policies with rules, a policy scoring algorithm for evaluating ABAC policies from the least privilege perspective, and performance optimization methods for dealing with the challenges of large ABAC privilege spaces. Using a large dataset of 4.7 million Amazon Web Service (AWS) audit log events, we demonstrate that our automated approach can effectively generate least privilege ABAC policies, and can generate policies with less over-privilege and under-privilege than a Role Based Access Control (RBAC) approach. Overall, we hope our work can help promote a wider and faster deployment of the ABAC model, and can help unleash the advantages of ABAC to better protect large and complex computing systems. 

   more » « less