We present Beholder, a computer vision (CV) toolkit for building tangible controllers for interactive computer systems. Beholder facilitates designers to build physical inputs that are instrumented with CV markers. By observing the properties of these markers, a CV system can detect physical interactions that occur. Beholder provides a software editor that enables designers to map CV marker behavior to keyboard events; thus connecting the CV-driven tangible controllers to any software that responds to keyboard input. We propose three design scenarios for Beholder—controllers to support everyday work, alternative controllers for games, and transforming physical therapy equipment into controllers to monitor patient progress.
more »
« less
Marking Material Interactions with Computer Vision
The electronics-centered approach to physical computing presents challenges when designers build tangible interactive systems due to its inherent emphasis on circuitry and electronic components. To explore an alternative physical computing approach we have developed a computer vision (CV) based system that uses a webcam, computer, and printed fiducial markers to create functional tangible interfaces. Through a series of design studios, we probed how designers build tangible interfaces with this CV-driven approach. In this paper, we apply the annotated portfolio method to reflect on the fifteen outcomes from these studios. We observed that CV markers offer versatile materiality for tangible interactions, afford the use of democratic materials for interface construction, and engage designers in embodied debugging with their own vision as a proxy for CV. By sharing our insights, we inform other designers and educators who seek alternative ways to facilitate physical computing and tangible interaction design.
more »
« less
- Award ID(s):
- 2040489
- PAR ID:
- 10484711
- Publisher / Repository:
- ACM
- Date Published:
- ISBN:
- 9781450394215
- Page Range / eLocation ID:
- 1 to 17
- Format(s):
- Medium: X
- Location:
- Hamburg Germany
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Within the CSCL community and in computing and computational making more broadly, issues of equity continue to be under-researched and undertheorized. Here, we examine how FUSE Studios – a set of in-school, choice-based, STEAM learning environments, based around a set of digital and tangible making challenges – supports equitable access to and participation in making and computing. Drawing on web-log data and video-ethnographic data, we argue that four characteristics of FUSE support equity: the design of the challenges; the diverse ways of knowing and doing supported by the activity system; the specific interactions encouraged by the activity system; and the program’s placement inside school. We focus, here, on gender equity, but also discuss implications and planned research on other aspects of equity.more » « less
-
OpenVX is a recently ratified standard that was expressly proposed to facilitate the design of computer-vision (CV) applications used in real-time embedded systems. Despite its real-time focus, OpenVX presents several challenges when validating real-time constraints. Many of these challenges are rooted in the fact that OpenVX only implicitly defines any notion of a schedulable entity. Under OpenVX, CV applications are specified in the form of processing graphs that are inherently considered to execute monolithically end-to-end. This monolithic execution hinders parallelism and can lead to significant processing-capacity loss. Prior work partially addressed this problem by treating graph nodes as schedulable entities, but under OpenVX, these nodes represent rather coarse-grained CV functions, so the available parallelism that can be obtained in this way is quite limited. In this paper, a much more fine-grained approach for scheduling OpenVX graphs is proposed. This approach was designed to enable additional parallelism and to eliminate schedulability-related processing-capacity loss that arises when programs execute on both CPUs and graphics processing units (GPUs). Response-time analysis for this new approach is presented and its efficacy is evaluated via a case study involving an actual CV application.more » « less
-
Game map interfaces provide an alternative perspective on the worlds players inhabit.compared to navigation applications popular in day-to-day life, game maps have different affordances to match players' situated goals. To contextualize and understand these differences and how they developed, we present a historical chronicle of game map interfaces. Starting from how games came to involve maps, we trace how maps are first separate from the game, becoming more and more integrated into play until converging in smartphone-style interfaces. We synthesize several game history texts with critical engagement with 123 key games to develop this map-focused chronicle, from which we highlight trends and opportunities for future map designs. Our work contributes a record of trends in game map interfaces that can serve as a source of reference and inspiration to game designers, digital physical-world map designers, and game scholars.more » « less
-
We describe a physical interactive system for human-robot collaborative design (HRCD) consisting of a tangible user interface (TUI) and a robotic arm that simultaneously manipulates the TUI with the human designer. In an observational study of 12 participants exploring a complex design problem together with the robot, we find that human designers have to negotiate both the physical and the creative space with the machine. They also often ascribe social meaning to the robot's pragmatic behaviors. Based on these findings, we propose four considerations for future HRCD systems: managing the shared workspace, communicating preferences about design goals, respecting different design styles, and taking into account the social meaning of design acts.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1145/3544548.3580643
