More Like this

1. Auditory Display in Interactive Science Simulations: Description and Sonification Support Interaction and Enhance Opportunities for Learning

   https://doi.org/10.1145/3313831.3376886  

   Tomlinson, Brianna J. ; Walker, Bruce N. ; Moore, Emily B. ( April 2020 , Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems)

   null (Ed.)

   Science simulations are widely used in classrooms to support inquiry-based learning of complex science concepts. These tools typically rely on interactive visual displays to convey relationships. Auditory displays, including verbal description and sonification (non-speech audio), combined with alternative input capabilities, may provide an enhanced experience for learners, particularly learners with visual impairment. We completed semi-structured interviews and usability testing with eight adult learners with visual impairment for two audio-enhanced simulations. We analyzed trends and edge cases in participants' interaction patterns, interpretations, and preferences. Findings include common interaction patterns across simulation use, increased efficiency with second use, and the complementary role that description and sonification play in supporting learning opportunities. We discuss how these control and display layers work to encourage exploration and engagement with science simulations. We conclude with general and specific design takeaways to support the implementation of auditory displays for accessible simulations. 

   more » « less
2. Sonification of Chemotactic Waves of Bacteria

   https://doi.org/10.21785/icad2023.8263  

   Braun, Rhea ; Tfirn, Maxwell ; Ford, Roseanne ( June 2023 , International Community for Auditory Display)

   Chemotaxis is the ability of certain microscopic organisms to sense and swim towards beneficial or away from detrimental chemicals in their surroundings. Identifying this behavior is important for understanding the relationships between species and their environments in the natural world. Predicting migration of an entire population from known characteristics of individual microorganisms is a key contribution, but can be a laborious process and requires watching and waiting for visual evidence of the process on a large population scale. Sonification offers a novel solution to this problem by allowing the observer to tap into our auditory sensory system to process information. In this project, we developed and assessed a proof-of-concept sonification tool as a high throughput, real-time screening tool for chemotaxis in populations of swimming bacteria. The tool operates by reporting the y-axis position of bacteria that appear in the microscope image as microsecond duration pitched notes, giving the user a sense of the average location of the population. In this paper, we present how it has been used as a chemotaxis assay and as a tool to locate traveling waves of bacteria as they pass through the field of view in order to capture data at specific timepoints, which is used to analyze individual swimming patterns of microorganisms within the wave.

    

   more » « less
3. Functional connectivity across the human subcortical auditory system using an autoregressive matrix-Gaussian copula graphical model approach with partial correlations

   https://doi.org/10.1162/imag_a_00258  

   Chandra, Noirrit Kiran ; Sitek, Kevin R ; Chandrasekaran, Bharath ; Sarkar, Abhra ( January 2024 , Imaging Neuroscience)

   The auditory system comprises multiple subcortical brain structures that process and refine incoming acoustic signals along the primary auditory pathway. Due to technical limitations of imaging small structures deep inside the brain, most of our knowledge of the subcortical auditory system is based on research in animal models using invasive methodologies. Advances in ultrahigh-field functional magnetic resonance imaging (fMRI) acquisition have enabled novel noninvasive investigations of the human auditory subcortex, including fundamental features of auditory representation such as tonotopy and periodotopy. However, functional connectivity across subcortical networks is still underexplored in humans, with ongoing development of related methods. Traditionally, functional connectivity is estimated from fMRI data with full correlation matrices. However, partial correlations reveal the relationship between two regions after removing the effects of all other regions, reflecting more direct connectivity. Partial correlation analysis is particularly promising in the ascending auditory system, where sensory information is passed in an obligatory manner, from nucleus to nucleus up the primary auditory pathway, providing redundant but also increasingly abstract representations of auditory stimuli. While most existing methods for learning conditional dependency structures based on partial correlations assume independently and identically Gaussian distributed data, fMRI data exhibit significant deviations from Gaussianity as well as high-temporal autocorrelation. In this paper, we developed an autoregressive matrix-Gaussian copula graphical model (ARMGCGM) approach to estimate the partial correlations and thereby infer the functional connectivity patterns within the auditory system while appropriately accounting for autocorrelations between successive fMRI scans. Our results show strong positive partial correlations between successive structures in the primary auditory pathway on each side (left and right), including between auditory midbrain and thalamus, and between primary and associative auditory cortex. These results are highly stable when splitting the data in halves according to the acquisition schemes and computing partial correlations separately for each half of the data, as well as across cross-validation folds. In contrast, full correlation-based analysis identified a rich network of interconnectivity that was not specific to adjacent nodes along the pathway. Overall, our results demonstrate that unique functional connectivity patterns along the auditory pathway are recoverable using novel connectivity approaches and that our connectivity methods are reliable across multiple acquisitions.

    

   more » « less
4. Visualizing Life in the Deep: A Creative Pipeline for Data-Driven Animations to Facilitate Marine Mammal Research, Outreach, and Conservation

   https://doi.org/10.1109/VISAP52981.2021.00007  

   Kendall-Bar, Jessica ; Kendall-Bar, Nicolas ; Forbes, Angus G. ; McDonald, Gitte ; Ponganis, Paul J. ; Williams, Cassondra ; Horning, Markus ; Hindle, Allyson ; Klinck, Holger ; Beltran, Roxanne S. ; et al ( October 2021 , 2021 IEEE VIS Arts Program (VISAP))

   In this paper, we introduce a creative pipeline to incorporate physiological and behavioral data from contemporary marine mammal research into data-driven animations, leveraging functionality from industry tools and custom scripts to promote scientific insights, public awareness, and conservation outcomes. Our framework can flexibly transform data describing animals’ orientation, position, heart rate, and swimming stroke rate to control the position, rotation, and behavior of 3D models, to render animations, and to drive data sonification. Additionally, we explore the challenges of unifying disparate datasets gathered by an interdisciplinary team of researchers, and outline our design process for creating meaningful data visualization tools and animations. As part of our pipeline, we clean and process raw acceleration and electrophysiological signals to expedite complex multi-stream data analysis and the identification of critical foraging and escape behaviors. We provide details about four animation projects illustrating marine mammal datasets. These animations, commissioned by scientists to achieve outreach and conservation outcomes, have successfully increased the reach and engagement of the scientific projects they describe. These impactful visualizations help scientists identify behavioral responses to disturbance, increase public awareness of human-caused disturbance, and help build momentum for targeted conservation efforts backed by scientific evidence. 

   more » « less
5. Warning users about cyber threats through sounds

   https://doi.org/10.1007/s42452-021-04703-4  

   Datta, Prerit ; Namin, Akbar Siami ; Jones, Keith S. ; Hewett, Rattikorn ( June 2021 , SN Applied Sciences)

   This paper reports a formative evaluation of auditory representations of cyber security threat indicators and cues, referred to as sonifications, to warn users about cyber threats. Most Internet browsers provide visual cues and textual warnings to help users identify when they are at risk. Although these alarming mechanisms are very effective in informing users, there are certain situations and circumstances where these alarming techniques are unsuccessful in drawing the user’s attention: (1) security warnings and features (e.g., blocking out malicious Websites) might overwhelm a typical Internet user and thus the users may overlook or ignore visual and textual warnings and, as a result, they might be targeted, (2) these visual cues are inaccessible to certain users such as those with visual impairments. This work is motivated by our previous work of the use of sonification of security warnings to users who are visually impaired. To investigate the usefulness of sonification in general security settings, this work uses real Websites instead of simulated Web applications with sighted participants. The study targets sonification for three different types of security threats: (1) phishing, (2) malware downloading, and (3) form filling. The results show that on average 58% of the participants were able to correctly remember what the sonification conveyed. Additionally, about 73% of the participants were able to correctly identify the threat that the sonification represented while performing tasks using real Websites. Furthermore, the paper introduces “CyberWarner”, a sonification sandbox that can be installed on the Google Chrome browser to enable auditory representations of certain security threats and cues that are designed based on several URL heuristics.

   It is feasible to develop sonified cyber security threat indicators that users intuitively understand with minimal experience and training.

   Users are more cautious about malicious activities in general. However, when navigating real Websites, they are less informed. This might be due to the appearance of the navigating Websites or the overwhelming issues when performing tasks.

   Participants’ qualitative responses indicate that even when they did not remember what the sonification conveyed, the sonification was able to capture the user’s attention and take safe actions in response.

    

   more » « less