More Like this

1. Sonification of Motion of Robotic Systems with Many Degrees-of-Freedom

   https://doi.org/10.7302/25802  

   Kacem, Amal; Mohammadi, Alireza (January 2025, University of Michigan)

   Effective human-robot interaction is increasingly vital across various domains, including assistive robotics, emotional communication, entertainment, and industrial automation. Visual feedback, a common feature of current interfaces, may not be suitable for all environments. Audio feedback serves as a critical supplementary communication layer in settings where visibility is low or where robotic operations generate extensive data. Sonification, which transforms a robot's trajectory, motion, and environmental signals into sound, enhances users' comprehension of robot behavior. This improvement in understanding fosters more effective, safe, and reliable Human-Robot Interaction (HRI). Demonstrations of auditory data sonification's benefits are evident in real-world applications such as industrial assembly, robot-assisted rehabilitation, and interactive robotic exhibitions, where it promotes cooperation, boosts performance, and heightens engagement. Beyond conventional HRI environments, auditory data sonification shows substantial potential in managing complex robotic systems and intricate structures, such as hyper-redundant robots and robotic teams. These systems often challenge operators with complex joint monitoring, mathematical kinematic modeling, and visual behavior verification. This dissertation explores the sonification of motion in hyper-redundant robots and teams of industrial robots. It delves into the Wave Space Sonification (WSS) framework developed by Hermann, applying it to the motion datasets of protein molecules modeled as hyper-redundant mechanisms with numerous rigid nano-linkages. This research leverages the WSS framework to develop a sonification methodology for protein molecules' dihedral angle folding trajectories. Furthermore, it introduces a novel approach for the systematic sonification of robotic motion across varying configurations. By employing localized wave fields oriented within the robots' configuration space, this methodology generates auditory outputs with specific timbral qualities as robots move through predefined configurations or along certain trajectories. Additionally, the dissertation examines a team of wheeled industrial/service robots whose motion patterns are sonified using sinusoidal vibratory sounds, demonstrating the practical applications and benefits of this innovative approach. 

   more » « less
2. Sonification of Motion of Unmanned Aerial Vehicles

   https://doi.org/10.7302/25819  

   Ali, Israa M; Mohammadi, Alireza (January 2025, University of Michigan)

   The development of Unmanned Aerial Vehicles (UAVs) can be enhanced through the use of sonification, an emerging field within Human-Robot Interaction (HRI). This dissertation introduces UAVSonification, a computational algorithm that maps simulation data to musical notes using the Musical Instrument Digital Interface (MIDI). By integrating UAVSonification with Formation Flight Simulation in Simulink, this study explores the sonification of UAV trajectories under environmental conditions. The function transforms simulation data into auditory signals, allowing users to discern key dynamics through sound. Specifically, the data series for multiple UAVs is mapped to piano notes via MIDI on MATLAB, providing auditory insights into UAV trajectories, environmental conditions, and control errors. A well-controlled flight path and stable heading controller produce harmonious sounds, while disruptions and deviations result in dissonance. UAVSonification offers a unique auditory approach to understanding UAV behavior in relation to control dynamics and environmental conditions. The sonification of UAVs has the potential to aid in the planning and analysis of UAV trajectories and controllers, as well as in creative endeavors. The effectiveness of the proposed method is shown through MATLAB numerical simulations. 

   more » « less
3. Safe and Coordinated Hierarchical Receding Horizon Control for Mobile Manipulators

   https://doi.org/10.23919/ACC45564.2020.9147238  

   Leu, Jessica; Lim, Rachel; Tomizuka, Masayoshi (July 2020, Safe and Coordinated Hierarchical Receding Horizon Control for Mobile Manipulators)

   null (Ed.)

   Mobile manipulators, constructed by mobile platforms and manipulators, have become a promising solution to future factories for introducing flexibility to manufacturing. This paper presents a method, hierarchical receding horizon control algorithm (HRHC), to assure safety and achieve higher time and space efficiency in robots surrounded by time-varying environments. HRHC contains an optimization based motion planning module that takes account of both the mobile platform and the manipulator to utilize the kinematic redundancy, and a low-level safety controller to deal with fast changes in the environment. With this method, we verify the performance through experiments. The result shows that space efficiency is increased and the HRHC can guarantee local safety in dynamic environments. 

   more » « less
4. Teaching the Sonification of Snake Robot Movement Using Project-Based Learning Activities

   https://doi.org/10.7302/25820  

   Connelly, Lynn E; Mohammadi, Alireza (January 2025, University of Michigan)

   Sonification is a method to represent data and convey information using sound. Just like the Geiger counter, humans can use sound to better understand complex sets of data that are either unable to be seen or visualized or that are too complex to understand with visual displays. Sonification research and learning have been predominantly conducted at the higher education level. However, as STEM-related programs and activities continue to be increasingly important in secondary school education, it is possible to expose high school students to university-level research through project-based learning (PBL) activities in the classroom. Using a physical snake robot prototype that was built and programmed with low-cost materials, high school students are introduced to the field of sonification and its applications to snake robots. This dissertation demonstrates the feasibility of using project-based learning to teach university level research in secondary school education. Using the sonification of snake robot movement, students learned advanced topics in robotics with the goal of realizing that university level research is accessible and understandable through PBL. This paper will begin by discussing the concept of human-robot interaction, introduce sonification, and give a brief overview of project-based learning. A detailed discussion of how the snake robot prototype was constructed and programmed, an in-depth explanation of the sonification algorithm that was used, and how sonification was taught in a high school classroom using PBL is presented along with student feedback and suggestions for future work. 

   more » « less
5. Supporting Accessible Data Visualization Through Audio Data Narratives

   https://doi.org/10.1145/3491102.3517678  

   Siu, Alexa; S-H Kim, Gene; O'Modhrain, Sile; Follmer, Sean (April 2022, CHI '22: Proceedings of the 2022 CHI Conference on Human Factors in Computing Systems)

   Online data visualizations play an important role in informing public opinion but are often inaccessible to screen reader users. To address the need for accessible data representations on the web that provide direct, multimodal, and up-to-date access to the data, we investigate audio data narratives –which combine textual descriptions and sonification (the mapping of data to non-speech sounds). We conduct two co-design workshops with screen reader users to define design principles that guide the structure, content, and duration of a data narrative. Based on these principles and relevant auditory processing characteristics, we propose a dynamic programming approach to automatically generate an audio data narrative from a given dataset. We evaluate our approach with 16 screen reader users. Findings show with audio narratives, users gain significantly more insights from the data. Users describe data narratives help them better extract and comprehend the information in both the sonification and description. 

   more » « less