More Like this

1. TunePad Playbooks: Designing Computational Notebooks for Creative Music Coding

   https://doi.org/10.1145/3491102.3502021  

   Horn, Mike; Banerjee, Amartya; Brucker, Matthew (April 2022, ACM Conference on Human Factors in Computing Systems (CHI'22))

   This paper describes the design of an online learning platform that empowers musical creation and performance with Python code. For this platform we have developed an innovative computational note- book paradigm that we call TunePad playbooks. While playbooks borrow ideas from popular computational notebooks like Jupyter, we have designed them from the ground up to support creative mu- sical expression including live performances. After discussing our design principles and features, we share findings from a series of artifact-centered interviews conducted with experienced TunePad users. Our results show how systems like ours might flexibly sup- port a variety of creative workflows, while suggesting opportunities for future work in this area. 

   more » « less
2. Machine learning for human design: Sketch interface for structural morphology ideation using neural networks

   Ong, B.; Danhaive, R.; Mueller, C. (January 2021, Proceedings of the International Association for Shell and Spatial Structures (IASS) Symposium 2020/2021)

   null (Ed.)

   Formal computational approaches in the realm of engineering and architecture, such as parametric modelling and optimization, are increasingly powerful, allowing for systematic and rigorous design processes. However, these methods often bring a steep learning curve, require previous expertise, or are unintuitive and unnatural to human design. On the other hand, analog design methods such as hand sketching are commonly used by architects and engineers alike, and constitute quick, easy, and almost primal modes of generating and transferring design concepts, which in turn facilitates the sharing of ideas and feedback. In the advent of increasing computational power and developments in data analysis, deep learning, and other emerging technologies, there is a potential to bridge the gap between these seemingly divergent processes to develop new hybrid approaches to design. Such methods can provide designers with new opportunities to harness the systematic and data-driven power of computation and performance analysis while maintaining a more creative and intuitive design interface. This paper presents a new method for interpreting human designs in sketch format and predicting their structural performance using recent advances in deep learning. The paper also demonstrates how this new technique can be used in design workflows including performance-based guidance and interpolations between concepts. 

   more » « less
3. Interactive Architectural Design with Diverse Solution Exploration

   https://doi.org/10.1109/TVCG.2019.2938961  

   Berseth, Glen; Haworth, Brandon; Usman, Muhammad; Schaumann, Davide; Khayatkhoei, Mahyar; Kapadia, Mubbasir Turab; Faloutsos, Petros (September 2019, IEEE Transactions on Visualization and Computer Graphics)

   In architectural design, architects explore a vast amount of design options to maximize various performance criteria, while adhering to specific constraints. In an effort to assist architects in such a complex endeavour, we propose IDOME, an interactive system for computer-aided design optimization. Our approach balances automation and control by efficiently exploring, analyzing, and filtering space layouts to inform architects' decision-making better. At each design iteration, IDOME provides a set of alternative building layouts which satisfy user-defined constraints and optimality criteria concerning a user-defined space parametrization. When the user selects a design generated by IDOME, the system performs a similar optimization process with the same (or different) parameters and objectives. A user may iterate this exploration process as many times as needed. In this work, we focus on optimizing built environments using architectural metrics by improving the degree of visibility, accessibility, and information gaining for navigating a proposed space. This approach, however, can be extended to support other kinds of analysis as well. We demonstrate the capabilities of IDOME through a series of examples, performance analysis, user studies, and a usability test. The results indicate that IDOME successfully optimizes the proposed designs concerning the chosen metrics and offers a satisfactory experience for users with minimal training. 

   more » « less
4. An Inconspicuous, Integrated Electronic Travel Aid for Visual Impairment

   https://doi.org/10.1115/1.4050186  

   Boldini, Alain; Garcia, Andy Louis; Sorrentino, Marc; Beheshti, Mahya; Ogedegbe, Okpe; Fang, Yi; Porfiri, Maurizio; Rizzo, John-Ross (October 2021, ASME Letters in Dynamic Systems and Control)

   null (Ed.)

   Abstract With a globally aging population, visual impairment is an increasingly pressing problem for our society. Visual disability drastically reduces quality of life and constitutes a large cost to the health care system. Mobility of the visually impaired is one of the most critical aspects affected by this disability, and yet, it relies on low-tech solutions, such as the white cane. Many avoid solutions entirely. In part, reluctance to use these solutions may be explained by their obtrusiveness, a strong deterrent for the adoption of many new devices. Here, we leverage new advancements in artificial intelligence, sensor systems, and soft electroactive materials toward an electronic travel aid with an obstacle detection and avoidance system for the visually impaired. The travel aid incorporates a stereoscopic camera platform, enabling computer vision, and a wearable haptic device that can stimulate discrete locations on the user’s abdomen to signal the presence of surrounding obstacles. The proposed technology could be integrated into commercial backpacks and support belts, thereby guaranteeing a discreet and unobtrusive solution. 

   more » « less
5. Before, Between, and After: Enriching Robot Communication Surrounding Collaborative Creative Activities

   https://doi.org/10.3389/frobt.2021.662355  

   Savery, Richard; Zahray, Lisa; Weinberg, Gil (April 2021, Frontiers in Robotics and AI)

   null (Ed.)

   Research in creative robotics continues to expand across all creative domains, including art, music and language. Creative robots are primarily designed to be task specific, with limited research into the implications of their design outside their core task. In the case of a musical robot, this includes when a human sees and interacts with the robot before and after the performance, as well as in between pieces. These non-musical interaction tasks such as the presence of a robot during musical equipment set up, play a key role in the human perception of the robot however have received only limited attention. In this paper, we describe a new audio system using emotional musical prosody, designed to match the creative process of a musical robot for use before, between and after musical performances. Our generation system relies on the creation of a custom dataset for musical prosody. This system is designed foremost to operate in real time and allow rapid generation and dialogue exchange between human and robot. For this reason, the system combines symbolic deep learning through a Conditional Convolution Variational Auto-encoder, with an emotion-tagged audio sampler. We then compare this to a SOTA text-to-speech system in our robotic platform, Shimon the marimba player.We conducted a between-groups study with 100 participants watching a musician interact for 30 s with Shimon. We were able to increase user ratings for the key creativity metrics; novelty and coherence, while maintaining ratings for expressivity across each implementation. Our results also indicated that by communicating in a form that relates to the robot’s core functionality, we can raise likeability and perceived intelligence, while not altering animacy or anthropomorphism. These findings indicate the variation that can occur in the perception of a robot based on interactions surrounding a performance, such as initial meetings and spaces between pieces, in addition to the core creative algorithms. 

   more » « less