An official website of the United States government
Abstract topics such as recursion are challenging for many computer science students to understand. In this experience report, we explore function sonification—the addition of sound to a function to communicate information about the function’s behavior in real-time as it runs—as a pedagogical approach for improving students’ understanding of recursion. We present several example iterative and recursive function sonifications, plus spectrograms that illustrate their different sonic behaviors. We also present experimental evidence that using these sonifications significantly improved the understanding of recursion for students who used them, compared to students who used silent (i.e., traditional) versions of the same functions. Based on these experiences, we believe sonification has under-appreciated potential for teaching abstract computing topics.
more »
« less
Listening to life: Sonification for enhancing discovery in biological research
Abstract Sonification, or the practice of generating sound from data, is a promising alternative or complement to data visualization for exploring research questions in the life sciences. Expressing or communicating data in the form of sound rather than graphs, tables, or renderings can provide a secondary information source for multitasking or remote monitoring purposes or make data accessible when visualizations cannot be used. While popular in astronomy, neuroscience, and geophysics as a technique for data exploration and communication, its potential in the biological and biotechnological sciences has not been fully explored. In this review, we introduce sonification as a concept, some examples of how sonification has been used to address areas of interest in biology, and the history of the technique. We then highlight a selection of biology‐related publications that involve sonifications of DNA datasets and protein datasets, sonifications for data collection and interpretation, and sonifications aimed to improve science communication and accessibility. Through this review, we aim to show how sonification has been used both as a discovery tool and a communication tool and to inspire more life‐science researchers to incorporate sonification into their own studies.
more »
« less
- Award ID(s):
- 1950369
- PAR ID:
- 10577086
- Publisher / Repository:
- Wiley Blackwell (John Wiley & Sons)
- Date Published:
- Journal Name:
- Biotechnology and Bioengineering
- Volume:
- 121
- Issue:
- 10
- ISSN:
- 0006-3592
- Format(s):
- Medium: X Size: p. 3009-3019
- Size(s):
- p. 3009-3019
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Much work already exists on algorithm visualization—the graphical representation of an algorithm’s behavior—and its benefits for student learning. Visualization, however, offers limited benefit for students with visual impairments. This paper explores algorithm sonification—the representation of an algorithm’s behavior using sound. To simplify the creation of sonifications for modern algorithms, this paper presents a new Thread Safe Audio Library (TSAL). To illustrate how to create sonifications, the authors have added TSAL calls to four common sorting algorithm implementations, so that as the program accesses a value being sorted, the program plays a tone whose pitch is scaled to that value’s magnitude. In the resulting sonifications, one can (in real time) hear the behavioral differences of the different sorting algorithms as they run, and directly experience how fast (or slow) the algorithms sort the same sequence, compared to one another. This paper presents experimental evidence that the sonifications improve students’ long-term recall of the four sorting algorithms’ relative speeds. The paper also discusses other potential uses of sonification.more » « less
-
The “Accessible Oceans” pilot project aims to inclusively design auditory displays that support perception and understanding of ocean data in informal learning environments (ILEs). The project’s multi-disciplinary team includes expertise from all related fields — ocean scientists, dataset experts, a sound designer with specialization in data sonification, and a learning sciences researcher. In addition, the PI is blind and provides a crucial perspective in our research. We describe the sound design of informative sonifications and respective auditory displays based on iterative design with user input at each stage, including from blind and low-vision (BLV) students, their teachers, and subject-matter experts. We discuss the importance of framing data sonifications through an auditory presentation of contextual information. We also report on our latest auditory display evaluation using Auditory Interface UX Scale (BUZZ) surveys at three ILE test sites. These responses further affirm our auditory display design developments. We include access to the auditory displays media and lessons learned over the course of this multi-year NSF-funded Advancing Informal Stem Learning (AISL) grant https://accessibleoceans.whoi.edu/more » « less
-
Sonification is a method to represent data and convey information using sound. Just like the Geiger counter, humans can use sound to better understand complex sets of data that are either unable to be seen or visualized or that are too complex to understand with visual displays. Sonification research and learning have been predominantly conducted at the higher education level. However, as STEM-related programs and activities continue to be increasingly important in secondary school education, it is possible to expose high school students to university-level research through project-based learning (PBL) activities in the classroom. Using a physical snake robot prototype that was built and programmed with low-cost materials, high school students are introduced to the field of sonification and its applications to snake robots. This dissertation demonstrates the feasibility of using project-based learning to teach university level research in secondary school education. Using the sonification of snake robot movement, students learned advanced topics in robotics with the goal of realizing that university level research is accessible and understandable through PBL. This paper will begin by discussing the concept of human-robot interaction, introduce sonification, and give a brief overview of project-based learning. A detailed discussion of how the snake robot prototype was constructed and programmed, an in-depth explanation of the sonification algorithm that was used, and how sonification was taught in a high school classroom using PBL is presented along with student feedback and suggestions for future work.more » « less
-
Effective human-robot interaction is increasingly vital across various domains, including assistive robotics, emotional communication, entertainment, and industrial automation. Visual feedback, a common feature of current interfaces, may not be suitable for all environments. Audio feedback serves as a critical supplementary communication layer in settings where visibility is low or where robotic operations generate extensive data. Sonification, which transforms a robot's trajectory, motion, and environmental signals into sound, enhances users' comprehension of robot behavior. This improvement in understanding fosters more effective, safe, and reliable Human-Robot Interaction (HRI). Demonstrations of auditory data sonification's benefits are evident in real-world applications such as industrial assembly, robot-assisted rehabilitation, and interactive robotic exhibitions, where it promotes cooperation, boosts performance, and heightens engagement. Beyond conventional HRI environments, auditory data sonification shows substantial potential in managing complex robotic systems and intricate structures, such as hyper-redundant robots and robotic teams. These systems often challenge operators with complex joint monitoring, mathematical kinematic modeling, and visual behavior verification. This dissertation explores the sonification of motion in hyper-redundant robots and teams of industrial robots. It delves into the Wave Space Sonification (WSS) framework developed by Hermann, applying it to the motion datasets of protein molecules modeled as hyper-redundant mechanisms with numerous rigid nano-linkages. This research leverages the WSS framework to develop a sonification methodology for protein molecules' dihedral angle folding trajectories. Furthermore, it introduces a novel approach for the systematic sonification of robotic motion across varying configurations. By employing localized wave fields oriented within the robots' configuration space, this methodology generates auditory outputs with specific timbral qualities as robots move through predefined configurations or along certain trajectories. Additionally, the dissertation examines a team of wheeled industrial/service robots whose motion patterns are sonified using sinusoidal vibratory sounds, demonstrating the practical applications and benefits of this innovative approach.more » « less
