skip to main content


Title: SmartBench: A Benchmark For Data Management In Smart Spaces
This paper proposes SmartBench, a benchmark focusing on queries resulting from (near) real-time applications and longer-term analysis of IoT data. SmartBench, derived from a deployed smart building monitoring system, is comprised of: 1) An extensible schema that captures the fundamentals of an IoT smart space; 2) A set of representative queries focusing on analytical tasks; and 3) A data generation tool that generates large amounts of synthetic sensor and semantic data based on seed data collected from a real system. We present an evaluation of seven representative database sys- tems and highlight some interesting findings that can be considered when deciding what database technologies to use under different types of IoT query workloads.  more » « less
Award ID(s):
1925610
NSF-PAR ID:
10184924
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Proceedings of the VLDB Endowment
Volume:
13
Issue:
11
ISSN:
2150-8097
Page Range / eLocation ID:
1807-1820
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. null (Ed.)
    As IoT services scale up from single homes to smart cities, directories and mapping services are needed to manage potentially millions of devices. However, directory service providers will likely struggle to accommodate the increasing number of IoT devices, made more challenging by their heterogeneous metadata and the large volume of queries. One of the critical challenges, the high heterogeneity of IoT, is being addressed by a working standard of W3C, which formalizes a physical or virtual device as a formatted Thing Description (TD).We propose a local directory service architecture with a series of design requirements. With a focus on query performance, we build a proof-of-concept system to store metadata of IoT devices as TDs in terms of the working standard. A Raspberry Pi is configured to investigate the query performance of relational database and non-relational database as the classic choices for internal directories. Evaluation results demonstrate that compared with relational database, non-relational database can achieve 2.9 times higher resilience on property query and 2.35 times faster processing on spatial query, with mild loss on aggregation query. 
    more » « less
  2. Collecting, storing, and providing access to Internet of Things (IoT) data are fundamental tasks to many smart city projects. However, developing and integrating IoT systems is still a significant barrier to entry. In this work, we share insights on the development of cloud data storage and visualization tools for IoT smart city applications using flood warning as an example application. The developed system incorporates scalable, autonomous, and inexpensive features that allow users to monitor real-time environmental conditions, and to create threshold-based alert notifications. Built in Amazon Web Services (AWS), the system leverages serverless technology for sensor data backup, a relational database for data management, and a graphical user interface (GUI) for data visualizations and alerts. A RESTful API allows for easy integration with web-based development environments, such as Jupyter notebooks, for advanced data analysis. The system can ingest data from LoRaWAN sensors deployed using The Things Network (TTN). A cost analysis can support users’ planning and decision-making when deploying the system for different use cases. A proof-of-concept demonstration of the system was built with river and weather sensors deployed in a flood prone suburban watershed in the city of Charlottesville, Virginia. 
    more » « less
  3. We are storing and querying datasets with the private information of individuals at an unprecedented scale in settings ranging from IoT devices in smart homes to mining enormous collections of click trails for targeted advertising. Here, the privacy of the people described in these datasets is usually addressed as an afterthought, engineered on top of a DBMS optimized for performance. At best, these systems support security or managing access to sensitive data. This status quo has brought us a plethora of data breaches in the news. In response, governments are stepping in to enact privacy regulations such as the EU’s GDPR. We posit that there is an urgent need for trustworthy database system that offer end-to-end privacy guarantees for their records with user interfaces that closely resemble that of a relational database. As we shall see, these guarantees inform everything in the database’s design from how we store data to what query results we make available to untrusted clients. In this position paper we first define trustworthy database systems and put their research challenges in the context of relevant tools and techniques from the security community. We then use this backdrop to walk through the “life of a query” in a trustworthy database system. We start with the query parsing and follow the query’s path as the system plans, optimizes, and executes it. We highlight how we will need to rethink each step to make it efficient, robust, and usable for database clients. 
    more » « less
  4. Internet of Things (IoT) is a connected network of devices that exchange data using different protocols. The application of IoT ranges from intelligent TVs and intelligent Refrigerators to smart Transportation. This research aims to provide students with hands-on training on how to develop an IoT platform that supports device management, connectivity, and data management. People tend to build interconnected devices without having a basic understanding of how the IoT platform backend function. Studying the Arm Pelion will help to understand how IoT devices operate under the hood. This past summer, Morgan State University has hosted undergraduate engineering students and high school STEM teachers to conduct IoT security research in the Cybersecurity Assurance & Policy (CAP) Center. The research project involved integrating various hardware sensor devices and real-time data monitoring using the Arm Pelion IoT development platform. Some of the student/teacher outcomes from the project include: 1) Learning about IoT Technology and security; 2) Programming an embedded system using Arm Mbed development board and IDE; 3 3) Developing a network of connected IoT devices using different protocols such as LWM2M, MQTT, CoAP; 4) Investigating the cybersecurity risks associated with the platform; and 5) Using data analysis and visualization to understand the network data and packet flow. First, the student/teacher must consider the IoT framework to understand how to address the security. The IoT framework describes the essential functions of an IoT network, breaking it down into separate layers. These layers include an application layer, middleware layer, and connectivity layer. The application layer allows the users to access the platform via a smartphone or any other dashboard. The Middleware layer represents the backend system that provides edge devices with data management, messaging, application services, and authentication. Finally, the connectivity layer includes devices that connect the user to the network, including Bluetooth or WiFi. The platform consists of several commercial IoT devices such as a smart camera, baby monitor, smart light, and other devices. We then create algorithms to classify the network data flow; to visualize the packets flow in the network and the structure of the packets data frame over time. 
    more » « less
  5. null (Ed.)
    Many Internet of Things (IoT) applications are time-critical and dynamically changing. However, traditional data processing systems (e.g., stream processing systems, cloud-based IoT data processing systems, wide-area data analytics systems) are not well-suited for these IoT applications. These systems often do not scale well with a large number of concurrently running IoT applications, do not support low-latency processing under limited computing resources, and do not adapt to the level of heterogeneity and dynamicity commonly present at edge environments. This suggests a need for a new edge stream processing system that advances the stream processing paradigm to achieve efficiency and flexibility under the constraints presented by edge computing architectures. We present \textsc{Dart}, a scalable and adaptive edge stream processing engine that enables fast processing of a large number of concurrent running IoT applications’ queries in dynamic edge environments. The novelty of our work is the introduction of a dynamic dataflow abstraction by leveraging distributed hash table (DHT) based peer-to-peer (P2P) overlay networks, which can automatically place, chain, and scale stream operators to reduce query latency, adapt to edge dynamics, and recover from failures. We show analytically and empirically that DART outperforms Storm and EdgeWise on query latency and significantly improves scalability and adaptability when processing a large number of real-world IoT stream applications' queries. DART significantly reduces application deployment setup times, becoming the first streaming engine to support DevOps for IoT applications on edge platforms. 
    more » « less