An official website of the United States government
Human-scale immersive environments offer rich, often interactive, experiences and their potential has been demonstrated across areas of research, teaching, and art. The variety of these spaces and their bespoke configurations leads to a requirement for content highly-tailored to individual environments and/or interfaces requiring complicated installations. These introduce hurdles which burden users with tedious and difficult learning curves, leaving less time for project development and rapid prototyping. This project demonstrates an interactive application to control and rapid-prototype within the Collaborative-Research Augmented Immersive Virtual Environment Laboratory, or CRAIVE-Lab. Application Programming Interfaces (APIs) render complex functions of the immersive environment, such as audio spatialization, accessible via the Internet. A front-end interface configured to communicate with these APIs gives users simple and intuitive control over these functions from their personal devices (e.g. laptops, smartphones). While bespoke systems will often require bespoke solutions, this interface allows users to create content on day one, from their own devices, without set up, content-tailoring, or training. Three examples utilizing some or all of these functions are discussed.
more »
« less
"Speakers, More Speakers!!! – Developing Interactive, Distributed, Smartphone-Based, Immersive Experiences for Music and Art"
We describe a multi-speaker, smartphone-based environment for developing interactive, distributed music and art applications, installations, and experiences. This system facilitates audience engagement through participation via personal smartphones, potentially connecting with traditional computing devices via the Internet without additional software or special configurations. The proposed approach has been inspired and motivated in part by the COVID-19 pandemic and builds on earlier works and technology. It demonstrates a design approach that is more efficient and provides a new avenue for music composers and artists to design highly distributed, participatory, immersive music and art experiences, utilizing various input sensors and actuators available in today’s smartphones. These include individual smartphone accelerometers, video cameras, and – of course – speakers. The use of smartphones also provides for relatively precise geolocation through GPS or simple social engineering approaches, such as using dedicated QR codes for different locations (e.g., seats in an auditorium). This allows for composing experiences to be rendered in the same room / auditorium, highly distributed across the Internet, or a combination of both. The paper presents the technological background and describes three case studies of such experiences, in an attempt to demonstrate the approach and inspire new avenues for artistic creativity and expression towards highly immersive, participatory installations / performances of music and art works for the 21st century.
more »
« less
- Award ID(s):
- 1935143
- PAR ID:
- 10374099
- Date Published:
- Journal Name:
- Proceedings of the International Symposium on Electronic Art
- ISSN:
- 2451-8611
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Mobile augmented reality (AR) has the potential to enable immersive, natural interactions between humans and cyber-physical systems. In particular markerless AR, by not relying on fiducial markers or predefined images, provides great convenience and flexibility for users. However, unwanted virtual object movement frequently occurs in markerless smartphone AR due to inaccurate scene understanding, and resulting errors in device pose tracking. We examine the factors which may affect virtual object stability, design experiments to measure it, and conduct systematic quantitative characterizations across six different user actions and five different smartphone configurations. Our study demonstrates noticeable instances of spatial instability in virtual objects in all but the simplest settings (with position errors of greater than 10cm even on the best-performing smartphones), and underscores the need for further enhancements to pose tracking algorithms for smartphone-based markerless AR.more » « less
-
Mobile augmented reality (AR) has the potential to enable immersive, natural interactions between humans and cyber-physical systems. In particular markerless AR, by not relying on fiducial markers or predefined images, provides great convenience and flexibility for users. However, unwanted virtual object movement frequently occurs in markerless smartphone AR due to inaccurate scene understanding, and resulting errors in device pose tracking. We examine the factors which may affect virtual object stability, design experiments to measure it, and conduct systematic quantitative characterizations across six different user actions and five different smartphone configurations. Our study demonstrates noticeable instances of spatial instability in virtual objects in all but the simplest settings (with position errors of greater than 10cm even on the best-performing smartphones), and underscores the need for further enhancements to pose tracking algorithms for smartphone-based markerless AR.more » « less
-
Iannis Xenakis was a pioneer in algorithmic composition of music and art. He combined architecture, mathematics, music, and performance art to create avant-garde compositions and performances that are still being analyzed, performed widely, and discussed today. Xenakis’s musical contributions are deeply algorithmic in nature, inspired by his appreciation and understanding of mathematics and controlled randomness, i.e., stochastic processes. This paper argues that Xenakis's algorithmic approach is style-agnostic, as it may be used with different sonification choices to produce pieces with more traditional aesthetics, possibly bringing broader acceptance, appreciation, and application of his techniques and ideas to more traditional compositional spaces. Also, today’s music technology has evolved tremendously, through the integration of artificial intelligence, advanced computing algorithms, and human computer interaction – techniques and technology that were unavailable to Xenakis, but which would have been inline with his pioneering spirit. We explore some of Xenakis’s early works in algorithmic and stochastic music, and reimagine the types of music Xenakis could possibly be making today, having access to the modern technology of smartphones and computing devices. Examples include a retelling of Xenakis's “Concret PH” using smartphones for sound spatialization and audience interaction / participation; “Éolienne PH”, an example piece which utilizes the same statistical distributions as “Concret PH” to produce a completely different sound aesthetic; “on the Fractal Nature of Being…” which combines Xenakis's stochastic / aleatoric techniques with traditional music theory and modern mathematics / fractal geometry; and, finally “Nereides / Νηρηΐδες”, a piano miniature piece, which is made using statistical distributions from a cloudy sky. We close with some general ideas on the future of the Algorithmic Arts.more » « less
-
Iannis Xenakis was a pioneer in algorithmic composition of music and art. He combined architecture, mathematics, music, and performance art to create avant-garde compositions and performances that are still being analyzed, performed widely, and discussed today. Xenakis’s musical contributions are deeply algorithmic in nature, inspired by his appreciation and understanding of mathematics and controlled randomness, i.e., stochastic processes. This paper argues that Xenakis's algorithmic approach is style-agnostic, as it may be used with different sonification choices to produce pieces with more traditional aesthetics, possibly bringing broader acceptance, appreciation, and application of his techniques and ideas to more traditional compositional spaces. Also, today’s music technology has evolved tremendously, through the integration of artificial intelligence, advanced computing algorithms, and human- computer interaction – techniques and technology that were unavailable to Xenakis, but which would have been inline with his pioneering spirit. We explore some of Xenakis’s early works in algorithmic and stochastic music, and reimagine the types of music Xenakis could possibly be making today, having access to the modern technology of smartphones and computing devices. Examples include a retelling of Xenakis's “Concret PH” using smartphones for sound spatialization and audience interaction / participation; “Éolienne PH”, an example piece which utilizes the same statistical distributions as “Concret PH” to produce a completely different sound aesthetic; “on the Fractal Nature of Being...” which combines Xenakis's stochastic / aleatoric techniques with traditional music theory and modern mathematics / fractal geometry; and, finally “Nereides / Νηρηΐδες”, a piano miniature piece, which is made using statistical distributions from a cloudy sky. We close with some general ideas on the future of the Algorithmic Arts.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- The DOI is not currently available.
