Search for: All records

The spatial audio technique wave field synthesis has the ability to perceptually locate sound sources in front of the loudspeaker array. This can create a challenge for pairing congruent visuals when using a traditional projection screen, such as in many immersive environments. If the sound is placed within the environment beyond the boundary of the screen, how can visuals be displayed such that they appear congruent with their corresponding sound? For this, the technique known as forced perspective, where objects are made to appear of different scales or positions than they are in reality, is utilized. This allows for content to be created to appear as if it is spilling into the environment, and thus remain congruent with its respective sound. Assessments to determine perceptions and the effectiveness of audiovisual sources paired in this way are being planned and will be underway. The technique described here can be utilized for various digital experiences, from artistic performances to 3D advertisements. 

This paper proposes an experiential method for learning acoustics and consequences of room design through the rapid creation of audio-visual congruent walkable auralizations. An efficient method produces auralizations of acoustical landmarks using a two-dimensional ray-tracing algorithm and publicly available floor plans for a 128-channel wave-field synthesis system. Late reverberation parameters are calculated using additional volumetric data. Congruent visuals are produced using a web-based interface accessible via personal devices, which automatically formats for and transmits to the immersive display. Massive user-contributed online databases are harnessed through application programming interfaces, such as those offered by the Google Maps Platform, to provide near-instant access to innumerable locations. The approach allows the rapid sonic recreation of historical concert venues with adequate sound sources. Listeners can walk through these recreations over an extended user area (12 m × 10 m). 

A. Chang*, S.R.V. Chabot, J. Mathews*, S. Briggs, T. Strzalkowski, M. Si, J. Braasch (2022) A spatially-aware companion system for language learning and foreign-language dialogue, In: Proceedings of the 24th International Congress on Acoustics (ICA), October 24–28, Gyeongju, Korea, Paper No. ABS-0631 (A-21, Virtual Acoustics), p. 67–76, https://www.ica2022korea.org/data/Proceedings_A21.pdf Full program -- see: https://www.ica2022korea.org/sub_proceedings.php 

3. J. Braasch, S.R.V. Chabot, M.J. Huang*, E.E.K. Scott* (2022) Rapid 3D Auralization of Historically Significant Buildings for Immersive Classroom Activities, In: Proceedings of the 24th International Congress on Acoustics (ICA), October 24–28, Gyeongju, Korea, Choi, Jee Woong, Cho Wan-Ho (ed.), Paper No. ABS-0624, p. 126–135, https://www.ica2022korea.org/data/Proceedings_A21.pdf https://www.ica2022korea.org/sub_proceedings.php 

J. Tyler, M. Si, J. Braasch (2022) Predicting room acoustical parameters from running signals using a precedence effect model and deep neural networks, In: Proceedings of the 24th International Congress on Acoustics (ICA), October 24–28, Gyeongju, Korea, Paper No. ABS-0627, p. 283–290, https://www.ica2022korea.org/data/Proceedings_A11.pdf https://www.ica2022korea.org/sub_proceedings.php 

Acoustic direction of arrival estimation methods allows positional information about sound sources to be transmitted over a network using minimal bandwidth. For these purposes,methods that prioritize low computational overhead and consistent accuracy under non-ideal conditions are preferred. The estimation method introduced in this paper uses a set of steered beams to estimate directional energy at sparsely distributed orientations around a spherical microphone array. By iteratively adjusting beam orientations based on the orientation of maximum energy, an accurate orientation estimate of a sound source may be produced with minimal computational cost. Incorporating conditions based on temporal smoothing and diffuse energy estimation further refines this process. Testing under simulated conditions indicates favorable accuracy under reverberation and source discrimination when compared with several other contemporary localization methods. Outcomes include an average localization error of less than 10◦ under 2 s of reverberation time (T60) and the potential to separate up to four sound sources under the same conditions. Results from testing in a laboratory environment demonstrate potential for integration into real-time frameworks. 

Human-scale immersive environments offer rich, often interactive, experiences and their potential has been demonstrated across areas of research, teaching, and art. The variety of these spaces and their bespoke configurations leads to a requirement for content highly-tailored to individual environments and/or interfaces requiring complicated installations. These introduce hurdles which burden users with tedious and difficult learning curves, leaving less time for project development and rapid prototyping. This project demonstrates an interactive application to control and rapid-prototype within the Collaborative-Research Augmented Immersive Virtual Environment Laboratory, or CRAIVE-Lab. Application Programming Interfaces (APIs) render complex functions of the immersive environment, such as audio spatialization, accessible via the Internet. A front-end interface configured to communicate with these APIs gives users simple and intuitive control over these functions from their personal devices (e.g. laptops, smartphones). While bespoke systems will often require bespoke solutions, this interface allows users to create content on day one, from their own devices, without set up, content-tailoring, or training. Three examples utilizing some or all of these functions are discussed. 

This paper proposes an efficient method to create auralizations of acoustical landmarks using a 2D ray-tracing algorithm and publicly available floor plans for a 128-channel wave field synthesis (WFS) system with 2.5D approximation. Late reverberation parameters are calculated using additional volumetric data. The approach allows the rapid sonic recreation of historical concert venues with adequate sound sources. The listeners can walk through these recreations over an extended user area (1210 sqm), and the software suite can be used to calculate room acoustical parameters for various positions directly using a binaural rendering method or via the WFS simulation. 

This paper introduces a method to estimate the direction of arrival of an acoustic signal based on finding maximum power in iteratively reduced regions of a spherical surface. A plane wave decomposition beamformer is used to produce power estimates at sparsely distributed points on the sphere. Iterating beam orientation based on the orientation of maximum energy produces accurate localization results. The method is tested using varying reverberation times, source-receiver distances, and angular separation of multiple sources and compared against a pseudo-intensity vector estimator. Results demonstrate that this method is suitable for integration into real-time telematic frameworks, especially in reverberant conditions. 

This work, situated at Rensselaer's Collaborative-Research Augmented Immersive Virtual Environment Laboratory (CRAIVELab), uses panoramic image datasets for spatial audio display. A system is developed for the room-centered immersive virtual reality facility to analyze panoramic images on a segment-by-segment basis, using pre-trained neural network models for semantic segmentation and object detection, thereby generating audio objects with respective spatial locations. These audio objects are then mapped with a series of synthetic and recorded audio datasets and populated within a spatial audio environment as virtual sound sources. The resulting audiovisual outcomes are then displayed using the facility's human-scale panoramic display, as well as the 128-channel loudspeaker array for wave field synthesis (WFS). Performance evaluation indicates effectiveness for real-time enhancements, with potentials for large-scale expansion and rapid deployment in dynamic immersive virtual environments.