skip to main content


Title: Human-machine communication for assistive IoT technologies
Despite the phenomenal advances in the computational power and functionality of electronic systems, human-machine interaction has largely been limited to simple control panels, keyboard, mouse and display. Consequently, these systems either rely critically on close human guidance or operate almost independently from the user. An exemplar technology integrated tightly into our lives is the smartphone. However, the term “smart” is a misnomer, since it has fundamentally no intelligence to understand its user. The users still have to type, touch or speak (to some extent) to express their intentions in a form accessible to the phone. Hence, intelligent decision making is still almost entirely a human task. A life-changing experience can be achieved by transforming machines from passive tools to agents capable of understanding human physiology and what their user wants [1]. This can advance human capabilities in unimagined ways by building a symbiotic relationship to solve real world problems cooperatively. One of the high-impact application areas of this approach is assistive internet of things (IoT) technologies for physically challenged individuals. The Annual World Report on Disability reveals that 15% of the world population lives with disability, while 110 to 190 million of these people have difficulty in functioning [1]. Quality of life for this population can improve significantly if we can provide accessibility to smart devices, which provide sensory inputs and assist with everyday tasks. This work demonstrates that smart IoT devices open up the possibility to alleviate the burden on the user by equipping everyday objects, such as a wheelchair, with decision-making capabilities. Moving part of the intelligent decision making to smart IoT objects requires a robust mechanism for human-machine communication (HMC). To address this challenge, we present examples of multimodal HMC mechanisms, where the modalities are electroencephalogram (EEG), speech commands, and motion sensing. We also introduce an IoT co-simulation framework developed using a network simulator (OMNeT++) and a robot simulation platform Virtual Robot Experimentation Platform (V-REP). We show how this framework is used to evaluate the effectiveness of different HMC strategies using automated indoor navigation as a driver application.  more » « less
Award ID(s):
1651624
NSF-PAR ID:
10062477
Author(s) / Creator(s):
Date Published:
Journal Name:
2016 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ISSS)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. The pervasive nature of smart connected devices has intruded on our daily lives and has become an intrinsic part of our world. However, the wide use of the Internet of Things (IoT) in critical application domains has raised concerns for user privacy and security against growing cyber threats. In particular, the implications of cyber exploitation for IoT devices are beyond financial losses and could constitute risks to human life. Most deployed access control solutions for smart IoT systems do not offer policy individualization, the ability to specify or change the policy according to the individual user’s preference. As a result, currently deployed systems are not well suited to specify access control policies in a multi-user environment, where users access the same devices to perform different operations. The system’s security gets tricky when the smart ecosystem involves complicated social relationships, much like in a smart home. Relationship-based access control (ReBAC), widely used in online social networks, offers the ability to consider user relationships in defining access control decisions and supports policy individualization. However, to the best of our knowledge, no such attempt has been made to develop a formal ReBAC model for smart IoT systems. This paper proposes a ReBAC IoT dynamic and fine-grained access control model which considers the social relationships among users along with the attributes to support an attributes-aware relationship-based access control model for smart IoT systems. ReBAC IoT is formally defined, illustrated through different use cases, implemented, and tested. 
    more » « less
  2. The pervasive nature of smart connected devices has intruded on our daily lives and has become an intrinsic part of our world. However, the wide use of the Internet of Things (IoT) in critical application domains has raised concerns for user privacy and security against growing cyber threats. In particular, the implications of cyber exploitation for IoT devices are beyond financial losses and could constitute risks to human life. Most deployed access control solutions for smart IoT systems do not offer policy individualization, the ability to specify or change the policy according to the individual user’s preference. As a result, currently deployed systems are not well suited to specify access control policies in a multi-user environment, where users access the same devices to perform different operations. The system’s security gets tricky when the smart ecosystem involves complicated social relationships, much like in a smart home. Relationship-based access control (ReBAC), widely used in online social networks, offers the ability to consider user relationships in defining access control decisions and supports policy individualization. However, to the best of our knowledge, no such attempt has been made to develop a formal ReBAC model for smart IoT systems. This paper proposes a ReBAC IoT dynamic and fine-grained access control model which considers the social relationships among users along with the attributes to support an attributes-aware relationship-based access control model for smart IoT systems. ReBAC IoT is formally defined, illustrated through different use cases, implemented, and tested. 
    more » « less
  3. Internet of Things (IoT) is a connected network of devices that exchange data using different protocols. The application of IoT ranges from intelligent TVs and intelligent Refrigerators to smart Transportation. This research aims to provide students with hands-on training on how to develop an IoT platform that supports device management, connectivity, and data management. People tend to build interconnected devices without having a basic understanding of how the IoT platform backend function. Studying the Arm Pelion will help to understand how IoT devices operate under the hood. This past summer, Morgan State University has hosted undergraduate engineering students and high school STEM teachers to conduct IoT security research in the Cybersecurity Assurance & Policy (CAP) Center. The research project involved integrating various hardware sensor devices and real-time data monitoring using the Arm Pelion IoT development platform. Some of the student/teacher outcomes from the project include: 1) Learning about IoT Technology and security; 2) Programming an embedded system using Arm Mbed development board and IDE; 3 3) Developing a network of connected IoT devices using different protocols such as LWM2M, MQTT, CoAP; 4) Investigating the cybersecurity risks associated with the platform; and 5) Using data analysis and visualization to understand the network data and packet flow. First, the student/teacher must consider the IoT framework to understand how to address the security. The IoT framework describes the essential functions of an IoT network, breaking it down into separate layers. These layers include an application layer, middleware layer, and connectivity layer. The application layer allows the users to access the platform via a smartphone or any other dashboard. The Middleware layer represents the backend system that provides edge devices with data management, messaging, application services, and authentication. Finally, the connectivity layer includes devices that connect the user to the network, including Bluetooth or WiFi. The platform consists of several commercial IoT devices such as a smart camera, baby monitor, smart light, and other devices. We then create algorithms to classify the network data flow; to visualize the packets flow in the network and the structure of the packets data frame over time. 
    more » « less
  4. As artificial intelligence (AI) assisted search and recommender systems have become ubiquitous in workplaces and everyday lives, understanding and accounting for fairness has gained increasing attention in the design and evaluation of such systems. While there is a growing body of computing research on measuring system fairness and biases associated with data and algorithms, the impact of human biases that go beyond traditional machine learning (ML) pipelines still remain understudied. In this Perspective Paper, we seek to develop a two-sided fairness framework that not only characterizes data and algorithmic biases, but also highlights the cognitive and perceptual biases that may exacerbate system biases and lead to unfair decisions. Within the framework, we also analyze the interactions between human and system biases in search and recommendation episodes. Built upon the two-sided framework, our research synthesizes intervention and intelligent nudging strategies applied in cognitive and algorithmic debiasing, and also proposes novel goals and measures for evaluating the performance of systems in addressing and proactively mitigating the risks associated with biases in data, algorithms, and bounded rationality. This paper uniquely integrates the insights regarding human biases and system biases into a cohesive framework and extends the concept of fairness from human-centered perspective. The extended fairness framework better reflects the challenges and opportunities in users’ interactions with search and recommender systems of varying modalities. Adopting the two-sided approach in information system design has the potential to enhancing both the effectiveness in online debiasing and the usefulness to boundedly rational users engaging in information-intensive decision-making. 
    more » « less
  5. Internet of Things has become a predominant phenomenon in every sphere of smart life. Connected Cars and Vehicular Internet of Things, which involves communication and data exchange between vehicles, traffic infrastructure or other entities are pivotal to realize the vision of smart city and intelligent transportation. Vehicular Cloud offers a promising architecture wherein storage and processing capabilities of smart objects are utilized to provide on-the-fly fog platform. Researchers have demonstrated vulnerabilities in this emerging vehicular IoT ecosystem, where data has been stolen from critical sensors and smart vehicles controlled remotely. Security and privacy is important in Internet of Vehicles (IoV) where access to electronic control units, applications and data in connected cars should only be authorized to legitimate users, sensors or vehicles. In this paper, we propose an authorization framework to secure this dynamic system where interactions among entities is not pre-defined. We provide an extended access control oriented (E-ACO) architecture relevant to IoV and discuss the need of vehicular clouds in this time and location sensitive environment. We outline approaches to different access control models which can be enforced at various layers of E-ACO architecture and in the authorization framework. Finally, we discuss use cases to illustrate access control requirements in our vision of cloud assisted connected cars and vehicular IoT, and discuss possible research directions. 
    more » « less