More Like this

1. Flood mitigation in coastal urban catchments using real-time stormwater infrastructure control and reinforcement learning

   https://doi.org/10.2166/hydro.2020.080  

   Bowes, Benjamin D. ; Tavakoli, Arash ; Wang, Cheng ; Heydarian, Arsalan ; Behl, Madhur ; Beling, Peter A. ; Goodall, Jonathan L. ( October 2020 , Journal of Hydroinformatics)

   null (Ed.)

   Abstract Flooding in coastal cities is increasing due to climate change and sea-level rise, stressing the traditional stormwater systems these communities rely on. Automated real-time control (RTC) of these systems can improve performance, and creating control policies for smart stormwater systems is an active area of study. This research explores reinforcement learning (RL) to create control policies to mitigate flood risk. RL is trained using a model of hypothetical urban catchments with a tidal boundary and two retention ponds with controllable valves. RL's performance is compared to the passive system, a model predictive control (MPC) strategy, and a rule-based control strategy (RBC). RL learns to proactively manage pond levels using current and forecast conditions and reduced flooding by 32% over the passive system. Compared to the MPC approach using a physics-based model and genetic algorithm, RL achieved nearly the same flood reduction, just 3% less than MPC, with a significant 88× speedup in runtime. Compared to RBC, RL was able to quickly learn similar control strategies and reduced flooding by an additional 19%. This research demonstrates that RL can effectively control a simple system and offers a computationally efficient method that could scale to RTC of more complex stormwater systems. 

   more » « less
2. Reinforcement learning-based real-time control of coastal urban stormwater systems to mitigate flooding and improve water quality

   https://doi.org/10.1039/D1EW00582K  

   Bowes, Benjamin D. ; Wang, Cheng ; Ercan, Mehmet B. ; Culver, Teresa B. ; Beling, Peter A. ; Goodall, Jonathan L. ( March 2022 , Environmental Science: Water Research & Technology)

   Real-time control of stormwater systems can reduce flooding and improve water quality. Current industry real-time control strategies use simple rules based on water quantity parameters at a local scale. However, system-level control methods that also incorporate observations of water quality could provide improved control and performance. Therefore, the objective of this research is to evaluate the impact of local and system-level control approaches on flooding and sediment-related water quality in a stormwater system within the flood-prone coastal city of Norfolk, Virginia, USA. Deep reinforcement learning (RL), an emerging machine learning technique, is used to learn system-level control policies that attempt to balance flood mitigation and treatment of sediment. RL is compared to the conventional stormwater system and two methods of local-scale rule-based control: (i) industry standard predictive rule-based control with a fixed detention time and (ii) rules based on water quality observations. For the studied system, both methods of rule-based control improved water quality compared to the passive system, but increased total system flooding due to uncoordinated releases of stormwater. An RL agent learned controls that maintained target pond levels while reducing total system flooding by 4% compared to the passive system. When pre-trained from the RL agent that learned to reduce flooding, another RL agent was able to learn to decrease TSS export by an average of 52% compared to the passive system and with an average of 5% less flooding than the rule-based control methods. As the complexity of stormwater RTC implementations grows and climate change continues, system-level control approaches such as the RL used here will be needed to help mitigate flooding and protect water quality. 

   more » « less
3. Adaptation of SWAT Watershed Model for Stormwater Management in Urban Catchments: Case Study in Austin, Texas

   https://doi.org/10.3390/w15091770  

   Glick, Roger ; Jeong, Jaehak ; Srinivasan, Raghavan ; Arnold, Jeffrey G. ; Her, Younggu ( May 2023 , Water)

   Computer simulation models are a useful tool in planning, enabling reliable yet affordable what-if scenario analysis. Many simulation models have been proposed and used for urban planning and management. Still, there are a few modeling options available for the purpose of evaluating the effects of various stormwater control measures (SCM), including LID (low-impact development) controls (green roof, rain garden, porous pavement, rainwater harvesting), upland off-line controls (sedimentation, filtration, retention–irrigation) and online controls (detention, wet pond). We explored the utility and potential of the Soil and Water Assessment Tool (SWAT) as a modeling tool for urban stormwater planning and management. This study demonstrates how the hydrologic modeling strategies of SWAT and recent enhancements could help to develop efficient measures for solving urban stormwater issues. The case studies presented in this paper focus on urban watersheds in the City of Austin (COA), TX, where rapid urbanization and population growth have put pressure on the urban stormwater system. Using the enhanced SWAT, COA developed a framework to assess the impacts on erosion, flooding, and aquatic life due to changes in runoff characteristics associated with land use changes. Five catchments in Austin were modeled to test the validity of the SWAT enhancements and the analytical framework. These case studies demonstrate the efficacy of using SWAT and the COA framework to evaluate the impacts of changes in hydrology and the effects of different regulatory schemes. 

   more » « less
4. Data-Driven Model Predictive Control for Real-Time Stormwater Management

   Ning, Jingyun ; Bowes, Benjamin ; Goodall, Jonathan ; Behl, Madhur ( June 2022 , 2022 American Control Conference)

   Low-lying coastal cities across the world are increasingly seeing flooding due to climate change and accompanying sea-level rise. Many such cities rely on old and passive stormwater infrastructure which cannot cope up with the increasing flood risk. One potential solution for addressing coastal flooding is implementing active control strategies in stormwater systems. Active stormwater control relies on rule-based strategies, which is not able to manage the increasing flood risk. Model predictive control (MPC) for stormwater flood management is getting attention over the past decade. However, building physics-based models for MPC in stormwater management is cost and time prohibitive. In this paper, we develop a data-driven approach, which utilizes unstructured state-space models for system identification and predictive control implementation. We demonstrate our results using two real stormwater network configurations, one from the Norfolk, VA region and another model of Ann Arbor region, MI, respectively. Our results indicate that MPC outperforms rule-based strategies by up to 60% of the Norfolk model and up to 90% of the Ann Arbor model in flood management. 

   more » « less
5. Reinforcement Learning for Flooding Mitigation in Complex Stormwater Systems during Large Storms

   https://doi.org/10.1109/EUROCON52738.2021.9535587  

   Wang, Cheng ; Bowes, Benjamin D. ; Beling, Peter A. ; Goodall, Jonathan L. ( July 2021 , IEEE EUROCON 2021 - 19th International Conference on Smart Technologies)

   Compared with capital improvement projects, real-time control of stormwater systems may be a more effective and efficient approach to address the increasing risk of flooding in urban areas. One way to automate the design process of control policies is through reinforcement learning (RL). Recently, RL methods have been applied to small stormwater systems and have demonstrated better performance over passive systems and simple rule-based strategies. However, it remains unclear how effective RL methods are for larger and more complex systems. Current RL-based control policies also suffer from poor convergence and stability, which may be due to large updates made by the underlying RL algorithm. In this study, we use the Proximal Policy Optimization (PPO) algorithm and develop control policies for a medium-sized stormwater system that can significantly mitigate flooding during large storm events. Our approach demonstrates good convergence behavior and stability, and achieves robust out-of-sample performance. 

   more » « less