skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: A multi-step and resilient predictive Q-learning algorithm for IoT: a case study in water supply networks
In this paper, we consider the problem of deriving recommended resilient and predictive actions for an IoT network in the presence of faulty components and malicious agents. The IoT, combining physical and cyber devices, is formulated as a directed graph with a known topology whose objective is to maintain a constant and resilient flow between a source node and a destination node. The optimal route through this network is evaluated via a predictive and resilient Q-learning algorithm which takes into account historical data about irregular operation, including faults and attacks. To showcase the efficacy of our approach, we utilize anonymized data from Arlington County, Virginia to obtain predictive and resilient scheduling policies for a smart water supply system while avoiding neighborhoods with leaks and other faults.  more » « less
Award ID(s):
1851588
PAR ID:
10078412
Author(s) / Creator(s):
; ;
Date Published:
Journal Name:
Proceedings of the 8th International Conference on the Internet of Things
Page Range / eLocation ID:
1 to 8
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Connectivity is at the heart of the future Internet-of-Things (IoT) infrastructure, which can control and communicate with remote sensors and actuators for the beacons, data collections, and forwarding nodes. Existing sensor network solutions cannot solve the bottleneck problems near the sink node; the tree-based Internet architecture has the single point of failure. To solve current deficiencies in multi-hop mesh network and cross-domain network design, we propose a mesh inside a mesh IoT network architecture. Our designed "edge router" incorporates these two mesh networks together and performs seamlessly transmission of multi-standard packets. The proposed IoT testbed interoperates with existing multi-standards (Wi-Fi, 6LoWPAN) and segments of networks, and provides both high-throughput Internet and resilient sensor coverage throughout the community. 
    more » « less
  2. Efficient and correct operation of an IoT network requires the presence of a failure detector and membership protocol amongst the IoT nodes. This paper presents a new failure de- tector for IoT settings where nodes are connected via a wire- less ad-hoc network. This failure detector, which we name Medley, is fully decentralized, allows IoT nodes to maintain a local membership list of other alive nodes, detects failures quickly (and updates the membership list), and incurs low communication overhead in the underlying ad-hoc network. In order to minimize detection time and communication, we adapt a failure detector originally proposed for datacenters (SWIM), for the IoT environment. In Medley each node picks a medley of ping targets in a randomized and skewed manner, preferring nearer nodes. Via analysis and NS-3 simulation we show the right mix of pinging probabilities that simulta- neously optimize detection time and communication traffic. We have also implemented Medley for Raspberry Pis, and present deployment results. 
    more » « less
  3. This study introduces a mixed-integer linear programming (MILP) model, effectively co-optimizing patrolling, damage assessment, fault isolation, repair, and load re-energization processes. The model is designed to solve a vital operational conundrum: deciding between further network exploration to obtain more comprehensive data or addressing the repair of already identified faults. As information on the fault location and repair timelines becomes available, the model allows for dynamic adaptation of crew dispatch decisions. In addition, this study proposes a conservative power flow constraint set that considers two network loading scenarios within the final network configuration. This approach results in the determination of an upper and a lower bound for node voltage levels and an upper bound for power line flows. To underscore the practicality and scalability of the proposed model, we have demonstrated its application using IEEE 123-node and 8500-node test systems, where it delivered promising results. 
    more » « less
  4. The Internet of Things (IoT) is an emerging technology that aims to connect our environment to the internet in the same way that personal computers connected people. As this technology progresses, the IoT paradigm becomes more prevalent in our everyday lives. The nature of IoT applications necessitates devices that are low-cost, power-sensitive, integrated, unobtrusive, and interoperable with existing cloud platforms and services, for example, Amazon AWS IoT, IBM Watson IoT. As a result, these devices are often small in size, with just enough computing power needed for their specific tasks. These resource-constrained devices are often unable to implement traditional network security measures and represent a vulnerability to network attackers as a result. Few frameworks are positioned to handle the influx of this new technology and the security concerns associated with it. Current solutions fail to provide a comprehensive and multi-layer solution to these inherent IoT security vulnerabilities. This paper presents a layered approach to IoT testbed that aims to bridge multiple connection standards and cloud platforms. To solve challenges surrounding this multi-layer IoT testbed, we propose a mesh inside a mesh IoT network architecture. Our designed "edge router" incorporates two mesh networks together and performs seamlessly transmission of multi-standard packets. The proposed IoT testbed interoperates with existing multi-standards (Wi-Fi, 6LoWPAN) and segments of networks, and provides both Internet and resilient sensor coverage to the cloud platform. To ensure confidentiality and authentication of IoT devices when interoperating with multiple service platforms, we propose optimized cryptographic techniques and software frameworks for IoT devices. We propose to extend and modify the existing open-source IDS platforms such as Snort to support IoT platforms and environments. We validate the efficacy of the proposed system by evaluating its performance and effect on key system resources. The work within this testbed design and implementation provides a solid foundation for further IoT system development. 
    more » « less
  5. null (Ed.)
    The concept of Industry 4.0 introduces the unification of industrial Internet-of-Things (IoT), cyber physical systems, and data-driven business modeling to improve production efficiency of the factories. To ensure high production efficiency, Industry 4.0 requires industrial IoT to be adaptable, scalable, real-time, and reliable. Recent successful industrial wireless standards such as WirelessHART appeared as a feasible approach for such industrial IoT. For reliable and real-time communication in highly unreliable environments, they adopt a high degree of redundancy. While a high degree of redundancy is crucial to real-time control, it causes a huge waste of energy, bandwidth, and time under a centralized approach and are therefore less suitable for scalability and handling network dynamics. To address these challenges, we propose DistributedHART—a distributed real-time scheduling system for WirelessHART networks. The essence of our approach is to adopt local (node-level) scheduling through a time window allocation among the nodes that allows each node to schedule its transmissions using a real-time scheduling policy locally and online. DistributedHART obviates the need of creating and disseminating a central global schedule in our approach, thereby significantly reducing resource usage and enhancing the scalability. To our knowledge, it is the first distributed real-time multi-channel scheduler for WirelessHART. We have implemented DistributedHART and experimented on a 130-node testbed. Our testbed experiments as well as simulations show at least 85% less energy consumption in DistributedHART compared to existing centralized approach while ensuring similar schedulability. 
    more » « less