More Like this

1. Improvising Interaction: Toward Applied Improvisation Driven Social Robotics Theory and Education

   Williams, Tom (March 2025, ACM)

   Theater-based design methods are seeing increased use in social robotics, as embodied roleplay is an ideal method for designing embodied interactions. Yet theater-based design methods are often cast as simply one possible tool; there has been little consideration of the importance of specific improvisational skills for theater-based design; and there has been little consideration of how to train students in theater-based design methods. We argue that improvisation is not just one possible tool of social robot design, but is instead central to social robotics. Leveraging recent theoretical work on Applied Improvisation, we show how improvisational skills represent (1) a set of key capabilities needed for any socially interactive robot, (2) a set of learning objectives for training engineers in social robot design, and (3) a set of methodologies for training those engineers to engage in theater-based design methods. Accordingly, we argue for a reconceptualization of Social Robotics as an Applied Improvisation project; we present, as a speculative pedagogical artifact, a sample syllabus for an envisioned Applied Improvisation driven Social Robotics course that might give students the technical and improvisational skills necessary to be effective robot designers; and we present a case study in which Applied Improvisation methods were simultaneously used (a) by instructors, to rapidly scaffold engineering students’ improvisational skills and (b) by those students, to engage in more effective human-robot interaction design. 

   more » « less
2. Musical Collaboration in Rhythmic Improvisation

   https://doi.org/10.3390/e22020233  

   Nakayama, Shinnosuke; Soman, Vrishin R.; Porfiri, Maurizio (February 2020, Entropy)

   Despite our intimate relationship with music in every-day life, we know little about how people create music. A particularly elusive area of study entails the spontaneous collaborative musical creation in the absence of rehearsals or scripts. Toward this aim, we designed an experiment in which pairs of players collaboratively created music in rhythmic improvisation. Rhythmic patterns and collaborative processes were investigated through symbolic-recurrence quantification and information theory, applied to the time series of the sound created by the players. Working with real data on collaborative rhythmic improvisation, we identified features of improvised music and elucidated underlying processes of collaboration. Players preferred certain patterns over others, and their musical experience drove musical collaboration when rhythmic improvisation started. These results unfold prevailing rhythmic features in collaborative music creation while informing the complex dynamics of the underlying processes. 

   more » « less
3. RL-Duet: Online Music Accompaniment Generation Using Deep Reinforcement Learning

   https://doi.org/10.1609/aaai.v34i01.5413  

   Jiang, Nan; Jin, Sheng; Duan, Zhiyao; Zhang, Changshui (June 2020, Proceedings of the AAAI Conference on Artificial Intelligence)

   This paper presents a deep reinforcement learning algorithm for online accompaniment generation, with potential for real-time interactive human-machine duet improvisation. Different from offline music generation and harmonization, online music accompaniment requires the algorithm to respond to human input and generate the machine counterpart in a sequential order. We cast this as a reinforcement learning problem, where the generation agent learns a policy to generate a musical note (action) based on previously generated context (state). The key of this algorithm is the well-functioning reward model. Instead of defining it using music composition rules, we learn this model from monophonic and polyphonic training data. This model considers the compatibility of the machine-generated note with both the machine-generated context and the human-generated context. Experiments show that this algorithm is able to respond to the human part and generate a melodic, harmonic and diverse machine part. Subjective evaluations on preferences show that the proposed algorithm generates music pieces of higher quality than the baseline method. 

   more » « less
4. Facilitating Learner-Centered Interactions Through Applied Improvisation

   Skinner, R.; Harlow, D.; Gunther, D.; Wesolowski, K. J.; Marckwordt, J.; Muller, A. (July 2022, Connected science learning)

   Learner-centered interactions, whether in formal or informal settings, are by their nature unscripted and require both the educator and learner to improvise. In fact, improvisation skills have been recognized as beneficial and applied in a variety of professional development training programs (including science communication, organizational development in university administration, teambuilding and leadership in business, and communication skills in medical education); yet, their inclusion in educator training has been limited. MOXI and UCSB partnered with a professional actor and theater instructor (third author of this paper) to implement applied improvisation training to support informal educators' skills development. After four years of incorporating applied improvisation training in our facilitation training program, we have found that the basic skills of listening, observing, and responding that are critical in learner-centered education are taught effectively through the well-developed, practical, and fun exercises of improvisational theater. In this article, we describe our applied improvisation training and how it builds skills pertinent to implementing learner-centered facilitation, how graduates of our training program connected applied improvisation training to their facilitation, and how other institutions can incorporate it into preparing educators for working in either informal or formal settings. 

   more » « less
5. VampNet: Music Generation via Masked Acoustic Token Modeling

   Flores_Garcia, H; Seetharaman, P; Kumar, R; Pardo, B (December 2023, 24th International Society for Music Information Retrieval Conference)

   We introduce VampNet, a masked acoustic token modeling approach to music synthesis, compression, inpainting, and variation. We use a variable masking schedule during training which allows us to sample coherent music from the model by applying a variety of masking approaches (called prompts) during inference. VampNet is non-autoregressive, leveraging a bidirectional transformer architecture that attends to all tokens in a forward pass. With just 36 sampling passes, VampNet can generate coherent high-fidelity musical waveforms. We show that by prompting VampNet in various ways, we can apply it to tasks like music compression, inpainting, outpainting, continuation, and looping with variation (vamping). Appropriately prompted, VampNet is capable of maintaining style, genre, instrumentation, and other high-level aspects of the music. This flexible prompting capability makes VampNet a powerful music co-creation tool. Code andaudio samples are available online. 

   more » « less