skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.

Attention:

The NSF Public Access Repository (PAR) system and access will be unavailable from 10:00 PM ET on Friday, February 6 until 10:00 AM ET on Saturday, February 7 due to maintenance. We apologize for the inconvenience.


Title: Introducing locus: a nime for immersive exocentric aural environments
Locus is a NIME designed specifically for an interactive, immersive high density loudspeaker array environment. The system is based on a pointing mechanism to interact with a sound scene comprising 128 speakers. Users can point anywhere to interact with the system, and the spatial interaction utilizes motion capture, so it does not require a screen. Instead it is completely controlled via hand gestures using a glove that is populated with motion-tracking markers. The main purpose of this system is to offer intuitive physical interaction with the perimeter based spatial sound sources. Further, its goal is to minimize user-worn technology and thereby enhance freedom of motion by utilizing environmental sensing devices, such as motion capture cameras or infrared sensors. The ensuing creativity enabling technology is applicable to a broad array of possible scenarios, from researching limits of human spatial hearing perception to facilitating learning and artistic performances, including dance. Below we describe our NIME design and implementation, its preliminary assessment, and offer a Unity-based toolkit to facilitate its broader deployment and adoption.  more » « less
Award ID(s):
1748667
PAR ID:
10108413
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
New Interfaces for Musical Expression
Page Range / eLocation ID:
250-255
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Fluid-structure interaction (FSI) studies the interaction between fluid and solid objects. It helps understand how fluid motion affects solid objects and vice versa. FSI research is important in engineering applications such as aerodynamics, hydrodynamics, and structural analysis. It has been used to design efficient systems such as ships, aircraft, and buildings. FSI in biological systems has gained interest in recent years for understanding how organisms interact with their fluidic environment. Our special issue features papers on various biological and bio-inspired FSI problems. Papers in this special issue cover topics ranging from flow physics to optimization and diagonistics. These papers offer new insights into natural systems and inspire the development of new technologies based on natural principles. 
    more » « less
  2. Skarnitzl, R. & (Ed.)
    While motion capture is rapidly becoming the gold standard for research on the intricacies of co-speech gesture and its relationship to speech, traditional marker-based motion capture technology is not always feasible, meaning researchers must code video data manually. We compare two methods for coding co-speech gestures of the hands and arms in video data of spontaneous speech: manual coding and semi-automated coding using OpenPose, a markerless motion capture software. We provide a comparison of the temporal alignment of gesture apexes based on video recordings of interviews with speakers of Medumba (Grassfields Bantu). Our results show a close correlation between the computationally calculated apexes and our hand-annotated apexes, suggesting that both methods are equally valid for coding video data. The use of markerless motion capture technology for gesture coding will enable more rapid coding of manual gestures, while still allowing 
    more » « less
  3. Abstract Microphone arrays are an essential tool in the field of bioacoustics as they provide a non-intrusive way to study animal vocalizations and monitor their movement and behavior. Microphone arrays can be used for passive localization and tracking of sound sources while analyzing beamforming or spatial filtering of the emitted sound. Studying free roaming animals usually requires setting up equipment over large areas and attaching a tracking device to the animal which may alter their behavior. However, monitoring vocalizing animals through arrays of microphones, spatially distributed over their habitat has the advantage that unrestricted/unmanipulated animals can be observed. Important insights have been achieved through the use of microphone arrays, such as the convergent acoustic field of view in echolocating bats or context-dependent functions of avian duets. Here we show the development and application of large flexible microphone arrays that can be used to localize and track any vocalizing animal and study their bio-acoustic behavior. In a first experiment with hunting pallid bats the acoustic data acquired from a dense array with 64 microphones revealed details of the bats’ echolocation beam in previously unseen resolution. We also demonstrate the flexibility of the proposed microphone array system in a second experiment, where we used a different array architecture allowing to simultaneously localize several species of vocalizing songbirds in a radius of 75 m. Our technology makes it possible to do longer measurement campaigns over larger areas studying changing habitats and providing new insights for habitat conservation. The flexible nature of the technology also makes it possible to create dense microphone arrays that can enhance our understanding in various fields of bioacoustics and can help to tackle the analytics of complex behaviors of vocalizing animals. 
    more » « less
  4. We develop a quasiparticle approach to capture the dynamics of open quantum systems coupled to bosonic thermal baths of arbitrary complexity based on the Hierarchical Equations of Motion (HEOM). This is done by generalizing the HEOM dynamics and mapping it into that of the system in interaction with a few bosonic fictitious quasiparticles that we call bexcitons. Bexcitons arise from a decomposition of the bath correlation function into discrete features. Specifically, bexciton creation and annihilation couple the auxiliary density matrices in the HEOM. The approach provides a systematic strategy to construct exact quantum master equations that include the system–bath coupling to all orders even for non-Markovian environments. Specifically, by introducing different metrics and representations for the bexcitons it is possible to straightforwardly generate different variants of the HEOM, demonstrating that all these variants share a common underlying quasiparticle picture. Bexcitonic properties, while unphysical, offer a coarse-grained view of the correlated system–bath dynamics and its numerical convergence. For instance, we use it to analyze the instability of the HEOM when the bath is composed of underdamped oscillators and show that it leads to the creation of highly excited bexcitons. The bexcitonic picture can also be used to develop more efficient approaches to propagate the HEOM. As an example, we use the particle-like nature of the bexcitons to introduce mode-combination of bexcitons in both number and coordinate representation that uses the multi-configuration time-dependent Hartree to efficiently propagate the HEOM dynamics. 
    more » « less
  5. There is a trade-off between the sparseness of an absorber array and its sound absorption imposed by wave physics. Here, near-perfect absorption (99% absorption) is demonstrated when the spatial period of monopole-dipole resonators is close to one working wavelength (95% of the wavelength). The condition for perfect absorption is to render degenerate monopole-dipole resonators critically coupled. Frequency domain simulations, eigenfrequency simulations, and the coupled mode theory are utilized to demonstrate the acoustic performances and the underlying physics. The sparse-resonator-based sound absorber could greatly benefit noise control with air flow and this study could also have implications for electromagnetic wave absorbers. 
    more » « less