More Like this

1. Hazard analysis for human-on-the-loop interactions in sUAS systems

   https://doi.org/10.1145/3468264.3468534  

   Vierhauser, Michael ; Islam, Md Nafee ; Agrawal, Ankit ; Cleland-Huang, Jane ; Mason, James ( August 2021 , Joint European Software Engineering Conference and Symposium on the Foundations of Software Engineering)

   null (Ed.)

   With the rise of new AI technologies, autonomous systems are moving towards a paradigm in which increasing levels of responsibility are shifted from the human to the system, creating a transition from human-in-the-loop systems to human-on-the-loop (HoTL) systems. This has a significant impact on the safety analysis of such systems, as new types of errors occurring at the boundaries of human-machine interactions need to be taken into consideration. Traditional safety analysis typically focuses on system-level hazards with little focus on user-related or user-induced hazards that can cause critical system failures. To address this issue, we construct domain-level safety analysis assets for sUAS (small unmanned aerial systems) applications and describe the process we followed to explicitly, and systematically identify Human Interaction Points (HiPs), Hazard Factors and Mitigations from system hazards. We evaluate our approach by first investigating the extent to which recent sUAS incidents are covered by our hazard trees, and second by performing a study with six domain experts using our hazard trees to identify and document hazards for sUAS usage scenarios. Our study showed that our hazard trees provided effective coverage for a wide variety of sUAS application scenarios and were useful for stimulating safety thinking and helping users to identify and potentially mitigate human-interaction hazards. 

   more » « less
2. On the Generalizability and Predictability of Recommender Systems

   McElfresh, Duncan ; Khandagale, Sujay ; Valverde, Jonathan ; Dickerson, John P. ; White, Colin ( January 2022 , Advances in Neural Information Processing Systems 35 (NeurIPS 2022))

   While other areas of machine learning have seen more and more automation, designing a high-performing recommender system still requires a high level of human effort. Furthermore, recent work has shown that modern recommender system algorithms do not always improve over well-tuned baselines. A natural follow-up question is, "how do we choose the right algorithm for a new dataset and performance metric?" In this work, we start by giving the first large-scale study of recommender system approaches by comparing 24 algorithms and 100 sets of hyperparameters across 85 datasets and 315 metrics. We find that the best algorithms and hyperparameters are highly dependent on the dataset and performance metric. However, there is also a strong correlation between the performance of each algorithm and various meta-features of the datasets. Motivated by these findings, we create RecZilla, a meta-learning approach to recommender systems that uses a model to predict the best algorithm and hyperparameters for new, unseen datasets. By using far more meta-training data than prior work, RecZilla is able to substantially reduce the level of human involvement when faced with a new recommender system application. We not only release our code and pretrained RecZilla models, but also all of our raw experimental results, so that practitioners can train a RecZilla model for their desired performance metric: https://github.com/naszilla/reczilla. 

   more » « less
3. Design of a Real-Time Human-Robot Collaboration System Using Dynamic Gestures

   https://doi.org/10.1115/IMECE2020-23650  

   Chen, Haodong ; Leu, Ming C. ; Tao, Wenjin ; Yin, Zhaozheng ( November 2020 , Proceedings of the ASME 2020 International Mechanical Engineering Congress and Exposition (IMECE 2020))

   With the development of industrial automation and artificial intelligence, robotic systems are developing into an essential part of factory production, and the human-robot collaboration (HRC) becomes a new trend in the industrial field. In our previous work, ten dynamic gestures have been designed for communication between a human worker and a robot in manufacturing scenarios, and a dynamic gesture recognition model based on Convolutional Neural Networks (CNN) has been developed. Based on the model, this study aims to design and develop a new real-time HRC system based on multi-threading method and the CNN. This system enables the real-time interaction between a human worker and a robotic arm based on dynamic gestures. Firstly, a multi-threading architecture is constructed for high-speed operation and fast response while schedule more than one task at the same time. Next, A real-time dynamic gesture recognition algorithm is developed, where a human worker’s behavior and motion are continuously monitored and captured, and motion history images (MHIs) are generated in real-time. The generation of the MHIs and their identification using the classification model are synchronously accomplished. If a designated dynamic gesture is detected, it is immediately transmitted to the robotic arm to conduct a real-time response. A Graphic User Interface (GUI) for the integration of the proposed HRC system is developed for the visualization of the real-time motion history and classification results of the gesture identification. A series of actual collaboration experiments are carried out between a human worker and a six-degree-of-freedom (6 DOF) Comau industrial robot, and the experimental results show the feasibility and robustness of the proposed system.

    

   more » « less
4. Automatic Critical Mechanic Discovery Using Playtraces in Video Games

   https://doi.org/10.1145/3402942.3402955  

   Green, Michael Cerny ; Khalifa, Ahmed ; Barros, Gabriella A. ; Machado, Tiago ; Togelius, Julian ( September 2020 , International Conference on the Foundations of Digital Games)

   null (Ed.)

   We present a new method of automatic critical mechanic discovery for video games using a combination of game description parsing and playtrace information. This method is applied to several games within the General Video Game Artificial Intelligence (GVG-AI) framework. In a user study, human-identified mechanics are compared against system-identified critical mechanics to verify alignment between humans and the system. The results of the study demonstrate that the new method is able to match humans with higher consistency than baseline. Our system is further validated by comparing MCTS agents augmented with critical mechanics and vanilla MCTS agents on 4 games from GVG-AI. Our new playtrace method shows a significant performance improvement over the baseline for all 4 tested games. The proposed method also shows either matched or improved performance over the old method, demonstrating that playtrace information is responsible for more complete critical mechanic discovery. 

   more » « less
5. Data Visualization of Crime Data using Immersive Virtual Reality

   Aishwarya Reehl, Sharad Sharma ( January 2022 , Proceedings of the IS&T International Symposium on Electronic Imaging (EI 2022), in the Engineering Reality of Virtual Reality)

   Visualization explores quantitative content of data with human intuition and plays an integral part in the data mining process. When the data is big there are different analysis methods and approaches used to find inherent patterns and relationships. However, sometimes there is a need to incorporate human in loop approach to find new patterns and relationships. Immersive virtual reality (VR) offers a human centric approach to visualize data by discovering new relationships that an existing algorithm cannot provide. This paper demonstrates the data visualization tool to visualize Baltimore crime data in immersive environment and non-immersive environment. This paper focuses on VR visualization tool design, development, and usability assessment. A pilot user study was conducted for the VR visualization tool based on the system usability scale. The study results show that the crime data visualization tool is relatively easy to use, and that the application can be considered as a storyteller with removing the noise from the data and highlighting the usefulness of the information. 

   more » « less