skip to main content


Title: The Convergence of IoT, Machine Learning, and Big Data for Advancing Flood Analytics Knowledge
Floods are among the most destructive natural hazard that affect millions of people across the world leading to severe loss of life and damage to property, critical infrastructure, and agriculture. Internet of Things (IoTs), machine learning (ML), and Big Data are exceptionally valuable tools for collecting the catastrophic readiness and countless actionable data. The aim of this presentation is to introduce Flood Analytics Information System (FAIS) as a data gathering and analytics system. FAIS application is smartly designed to integrate crowd intelligence, ML, and natural language processing of tweets to provide warning with the aim to improve flood situational awareness and risk assessment. FAIS has been Beta tested during major hurricane events in US where successive storms made extensive damage and disruption. The prototype successfully identifies a dynamic set of at-risk locations/communities using the USGS river gauge height readings and geotagged tweets intersected with watershed boundary. The list of prioritized locations can be updated, as the river monitoring system and condition change over time (typically every 15 minutes). The prototype also performs flood frequency analysis (FFA) using various probability distributions with the associated uncertainty estimation to assist engineers in designing safe structures. This presentation will discuss about the FAIS functionalities and real-time implementation of the prototype across south and southeast USA. This research is funded by the US National Science Foundation (NSF).  more » « less
Award ID(s):
2035685
NSF-PAR ID:
10270714
Author(s) / Creator(s):
;
Date Published:
Journal Name:
European Geosciences Union
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. With the rapid development of the Internet of Things (IoT) and Big Data infrastructure, crowdsourcing techniques have emerged to facilitate data processing and problem solving particularly for flood emergences purposes. A Flood Analytics Information System (FAIS) has been developed as a Python Web application to gather Big Data from multiple servers and analyze flooding impacts during historical and real-time events. The application is smartly designed to integrate crowd intelligence, machine learning (ML), and natural language processing of tweets to provide flood warning with the aim to improve situational awareness for flood risk management. FAIS, a national scale prototype, combines flood peak rates and river level information with geotagged tweets to identify a dynamic set of at-risk locations to flooding. The prototype was successfully tested in real-time during Hurricane Dorian flooding as well as for historical event (Hurricanes Florence) across the Carolinas, USA where the storm made extensive disruption to infrastructure and communities. 
    more » « less
  2. With the rapid development of the Internet of Things (IoT) and Big Data infrastructure, crowdsourcing techniques have emerged to facilitate data processing and problem solving particularly for flood emergences purposes. A Flood Analytics Information System (FAIS) has been developed as a Python Web application to gather Big Data from multiple servers and analyze flooding impacts during historical and real-time events. The application is smartly designed to integrate crowd intelligence, machine learning (ML), and natural language processing of tweets to provide flood warning with the aim to improve situational awareness for flood risk management. FAIS, a national scale prototype, combines flood peak rates and river level information with geotagged tweets to identify a dynamic set of at-risk locations to flooding. The prototype was successfully tested in real-time during Hurricane Dorian flooding as well as for historical event (Hurricanes Florence) across the Carolinas, USA where the storm made extensive disruption to infrastructure and communities. 
    more » « less
  3. Successive hurricane events have brought new challenges to human life, critical infrastructure and the environment in the United States. This study introduced Flood Analytics Information System (FAIS) as a national scale data analytics application for real-time flood data collection and analysis. FAIS has been Beta tested across a mixed urban and rural basin in the Carolinas where Hurricane Florence made extensive damages and disruption. Our research aim was to develop and test an integrated solution based on real time Big data for stakeholder map-based dashboard visualizations that can be applicable to other countries and a range of weather-driven emergency situations. The prototype successfully identifies a dynamic set of at-risk locations using web-based river level and flood information. The list of prioritized locations can be updated every 15 minutes, as the environmental information and condition change. FAIS will be extended to collect data from other countries and other kinds of weather-related hazards such as snowstorms and wildfires. 
    more » « less
  4. Abstract River deltas all over the world are sinking beneath sea-level rise, causing significant threats to natural and social systems. This is due to the combined effects of anthropogenic changes to sediment supply and river flow, subsidence, and sea-level rise, posing an immediate threat to the 500–1,000 million residents, many in megacities that live on deltaic coasts. The Mississippi River Deltaic Plain (MRDP) provides examples for many of the functions and feedbacks, regarding how human river management has impacted source-sink processes in coastal deltaic basins, resulting in human settlements more at risk to coastal storms. The survival of human settlement on the MRDP is arguably coupled to a shifting mass balance between a deltaic landscape occupied by either land built by the Mississippi River or water occupied by the Gulf of Mexico. We developed an approach to compare 50 % L:W isopleths (L:W is ratio of land to water) across the Atchafalaya and Terrebonne Basins to test landscape behavior over the last six decades to measure delta instability in coastal deltaic basins as a function of reduced sediment supply from river flooding. The Atchafalaya Basin, with continued sediment delivery, compared to Terrebonne Basin, with reduced river inputs, allow us to test assumptions of how coastal deltaic basins respond to river management over the last 75 years by analyzing landward migration rate of 50 % L:W isopleths between 1932 and 2010. The average landward migration for Terrebonne Basin was nearly 17,000 m (17 km) compared to only 22 m in Atchafalaya Basin over the last 78 years (p\0.001), resulting in migration rates of 218 m/year (0.22 km/year) and\0.5 m/year, respectively. In addition, freshwater vegetation expanded in Atchafalaya Basin since 1949 compared to migration of intermediate and brackish marshes landward in the Terrebonne Basin. Changes in salt marsh vegetation patterns were very distinct in these two basins with gain of 25 % in the Terrebonne Basin compared to 90 % decrease in the Atchafalaya Basin since 1949. These shifts in vegetation types as L:W ratio decreases with reduced sediment input and increase in salinity also coincide with an increase in wind fetch in Terrebonne Bay. In the upper Terrebonne Bay, where the largest landward migration of the 50 % L:W ratio isopleth occurred, we estimate that the wave power has increased by 50–100 % from 1932 to 2010, as the bathymetric and topographic conditions changed, and increase in maximum storm-surge height also increased owing to the landward migration of the L:W ratio isopleth. We argue that this balance of land relative to water in this delta provides a much clearer understanding of increased flood risk from tropical cyclones rather than just estimates of areal land loss. We describe how coastal deltaic basins of the MRDP can be used as experimental landscapes to provide insights into how varying degrees of sediment delivery to coastal deltaic floodplains change flooding risks of a sinking delta using landward migrations of 50 % L:W isopleths. The nonlinear response of migrating L:W isopleths as wind fetch increases is a critical feedback effect that should influence human river-management decisions in deltaic coast. Changes in land area alone do not capture how corresponding landscape degradation and increased water area can lead to exponential increase in flood risk to human populations in low-lying coastal regions. Reduced land formation in coastal deltaic basins (measured by changes in the land:water ratio) can contribute significantly to increasing flood risks by removing the negative feedback of wetlands on wave and storm-surge that occur during extreme weather events. Increased flood risks will promote population migration as human risks associated with living in a deltaic landscape increase, as land is submerged and coastal inundation threats rise. These system linkages in dynamic deltaic coasts define a balance of river management and human settlement dependent on a certain level of land area within coastal deltaic basins (L). 
    more » « less
  5. The National Space Weather Action Plan, NERC Reliability Standard TPL-007-1,2 associated FERC Orders 779, 830 and subsequent actions, emerging standard TPL-007-3, as well as Executive Order 13744 have prepared the regulatory framework and roadmap for assessing and mitigating the impact on critical infrastructure from space weather. These actions have resulted in an emerging set of benchmarks against which the statistical probability of damage to critical components such as power transmission system high-voltage transformers can be assessed; for the first time the impacts on the intensity of geomagnetically induced currents (GICs) due to the spatial variability of the geomagnetic field and of the Earth’s electrical conductivity structure can be examined systematically. While at present not a strict requirement of the existing reliability standards, there is growing evidence that the strongly three-dimensional nature of the electrical conductivity structure of the North American crust and mantle (heretofore ‘ground conductivity’) has a first-order impact on GIC intensity, with considerable local and regional variability. The strongly location dependent ground electric field intensification and attenuation due to 3-D ground conductivity variations has an equivalent impact on assessment of risk to critical infrastructure due to HEMP (E3 phase) sources of geomagnetic disturbances (GMDs) as it does for natural GMDs. From 2006-2018, Oregon State University (OSU) under NSF EarthScope Program support, installed and acquired ground electric and magnetic field time series (magnetotelluric, or MT) data on a grid of station locations spaced ~70-km apart, at 1161 long-period MT stations covering nearly ⅔ of CONUS. The US Geological Survey completed 47 additional MT stations using functionally identical instrumentation, and the two data sets were merged and made available in the public domain. OSU and its project collaborators have also collected hundreds of wider frequency bandwidth, more densely-spaced MT station data under other project support, and these have been or will be released for public access in the near future. NSF funding was not available to make possible collection of EarthScope MT data 1/3 of CONUS in the southern tier of states, in a band from central California in the west to Alabama in the east and extending along the Gulf Coast and Deep South. OSU, with NASA support just received, plans to complete MT station installation in the remainder of California this year, and with additional support both anticipated and proposed, we hope to complete the MT array in the remainder of CONUS. For this first time this will provide national-scale 3-D electrical conductivity/MT impedance data throughout the US portion of the contiguous North American power grid. Complementary planning and proposal efforts are underway in Canada, including collaborations between OSU, Athabasca University and other Canadian academic and industry groups. In the present work, we apply algorithms we have developed to make use of real-time streams of US Geological Survey, Natural Resources Canada (and other) magnetic observatory data, and the EarthScope and other MT data sets to provide quasi-real time predictions of the geomagnetically induced voltages at high-voltage transmission system transformers/power buses. This goes beyond the statistical benchmarking process currently encapsulated in NERC reliability standards. We seek initially to provide real-time information to power utility control room operators, in the form of a heat map showing which assets are likely experiencing stress due to induced currents. These assessments will be ground-truthed against transmission system sensor data (PMUs, GIC monitors, voltage waveforms and harmonics where available), and by applying machine learning methods we hope to extend this approach to transmission systems that have sparse or non-existent GIC monitoring sensor infrastructure. Ultimately by incorporating predictive models of the geomagnetic field using satellite data as inputs rather than real-time ground magnetic field measurements, a near-term probabilistic assessment of risk to transformers may be possible, ideally providing at least a 15-minute forecast to utility operators. There has been a concerted effort by NOAA to develop a real-time geomagnetically induced ground electric field data product that makes use of our EarthScope MT data, which includes the strong impacts on GICs due to 3-D ground conductivity structure. Both OSU and the USGS have developed methods to determine the GIC-related voltages at substations by integrating the ground electric fields along power transmission line paths. Under National Science Foundation support, the present team of investigators is taking the next step, of applying the GIC-related voltages as inputs to quasi-real time power flow models of the power transmission grid in order to obtain realistic and verifiable predictions of the intensity of induced GICs, the reactive power loss due to GICs, and of GIC effects on the current and voltage waveforms, such as the harmonic distortion. As we work toward integration of predicted induced substation voltages with power flow models, we’ve modified the RTS-GMLC (Reliability Test System Grid Modernization Lab Consortium) test case (https://github.com/GridMod/RTS-GMLC) by moving the geographic location of the case to central Oregon. With the assistance of LANL we have the complete AC and DC network of the RTS-GMLC case, and we are working to integrate the complete case information into Julia (using the PowerModels and PowerModelsGMD packages of LANL), or into PowerWorld. Along a parallel track, we have performed GIC voltage calculations using our geophysical algorithm for a realistic GMD event (Halloween event) for the test case, resulting in GIC transmission line voltages that can be added into our power system model. We’ll discuss our progress in integrating the geophysical estimates of transformer voltages and our DC model using LANL's Julia and PowerModelsGMD package, for power flow simulations on the test case, and to determine the GIC flows and possible impacts on the power waveforms in the system elements. 
    more » « less