skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: A spatially-aware companion system for language learning and foreign-language dialogue
A. Chang*, S.R.V. Chabot, J. Mathews*, S. Briggs, T. Strzalkowski, M. Si, J. Braasch (2022) A spatially-aware companion system for language learning and foreign-language dialogue, In: Proceedings of the 24th International Congress on Acoustics (ICA), October 24–28, Gyeongju, Korea, Paper No. ABS-0631 (A-21, Virtual Acoustics), p. 67–76, https://www.ica2022korea.org/data/Proceedings_A21.pdf Full program -- see: https://www.ica2022korea.org/sub_proceedings.php  more » « less
Award ID(s):
1909229
PAR ID:
10418573
Author(s) / Creator(s):
; ; ; ; ; ;
Editor(s):
Choi, Jee Woong; Cho, Wan-Ho
Date Published:
Journal Name:
Proceedings of the 24th International Congress on Acoustics (ICA)
Page Range / eLocation ID:
ABS-0631
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Choi, Jee Woong; Cho Wan-Ho (Ed.)
    J. Tyler, M. Si, J. Braasch (2022) Predicting room acoustical parameters from running signals using a precedence effect model and deep neural networks, In: Proceedings of the 24th International Congress on Acoustics (ICA), October 24–28, Gyeongju, Korea, Paper No. ABS-0627, p. 283–290, https://www.ica2022korea.org/data/Proceedings_A11.pdf https://www.ica2022korea.org/sub_proceedings.php 
    more » « less
  2. Choi, Jee Woong; Cho Wan-Ho (Ed.)
    3. J. Braasch, S.R.V. Chabot, M.J. Huang*, E.E.K. Scott* (2022) Rapid 3D Auralization of Historically Significant Buildings for Immersive Classroom Activities, In: Proceedings of the 24th International Congress on Acoustics (ICA), October 24–28, Gyeongju, Korea, Choi, Jee Woong, Cho Wan-Ho (ed.), Paper No. ABS-0624, p. 126–135, https://www.ica2022korea.org/data/Proceedings_A21.pdf https://www.ica2022korea.org/sub_proceedings.php 
    more » « less
  3. Human listeners are better at telling apart speakers of their native language than speakers of other languages, a phenomenon known as the language familiarity effect. The recent observation of such an effect in infants as young as 4.5 months of age (Fecher & Johnson, in press) has led to new difficulties for theories of the effect. On the one hand, retaining classical accounts—which rely on sophisticated knowledge of the native language (Goggin, Thompson, Strube, & Simental, 1991)–requires an explanation of how infants could acquire this knowledge so early. On the other hand, letting go of these accounts requires an explanation of how the effect could arise in the absence of such knowledge. In this paper, we build on algorithms from unsupervised machine learning and zero-resource speech technology to propose, for the first time, a feasible acquisition mechanism for the language familiarity effect in infants. Our results show how, without relying on sophisticated linguistic knowledge, infants could develop a language familiarity effect through statistical modeling at multiple time-scales of the acoustics of the speech signal to which they are exposed. 
    more » « less
  4. Humans subconsciously engage in geospatial reasoning when reading articles. We recognize place names and their spatial relations in text and mentally associate them with their physical locations on Earth. Although pretrained language models can mimic this cognitive process using linguistic context, they do not utilize valuable geospatial information in large, widely available geographical databases, e.g., OpenStreetMap. This paper introduces GeoLM, a geospatially grounded language model that enhances the understanding of geo-entities in natural language. GeoLM leverages geo-entity mentions as anchors to connect linguistic information in text corpora with geospatial information extracted from geographical databases. GeoLM connects the two types of context through contrastive learning and masked language modeling. It also incorporates a spatial coordinate embedding mechanism to encode distance and direction relations to capture geospatial context. In the experiment, we demonstrate that GeoLM exhibits promising capabilities in supporting toponym recognition, toponym linking, relation extraction, and geo-entity typing, which bridge the gap between natural language processing and geospatial sciences. The code is publicly available at https://github.com/knowledge-computing/geolm. 
    more » « less
  5. Oh, Alice; Naumann, Tristan; Globerson, Amir; Saenko, Kate; Hardt, Moritz; Levine, Sergey (Ed.)
    Diffusion models have achieved great success in modeling continuous data modalities such as images, audio, and video, but have seen limited use in discrete domains such as language. Recent attempts to adapt diffusion to language have presented diffusion as an alternative to existing pretrained language models. We view diffusion and existing language models as complementary. We demonstrate that encoder-decoder language models can be utilized to efficiently learn high-quality language autoencoders. We then demonstrate that continuous diffusion models can be learned in the latent space of the language autoencoder, enabling us to sample continuous latent representations that can be decoded into natural language with the pretrained decoder. We validate the effectiveness of our approach for unconditional, class-conditional, and sequence-to-sequence language generation. We demonstrate across multiple diverse data sets that our latent language diffusion models are significantly more effective than previous diffusion language models. Our code is available at https://github.com/justinlovelace/latent-diffusion-for-language . 
    more » « less