An official website of the United States government
The electronics-centered approach to physical computing presents challenges when designers build tangible interactive systems due to its inherent emphasis on circuitry and electronic components. To explore an alternative physical computing approach we have developed a computer vision (CV) based system that uses a webcam, computer, and printed fiducial markers to create functional tangible interfaces. Through a series of design studios, we probed how designers build tangible interfaces with this CV-driven approach. In this paper, we apply the annotated portfolio method to reflect on the fifteen outcomes from these studios. We observed that CV markers offer versatile materiality for tangible interactions, afford the use of democratic materials for interface construction, and engage designers in embodied debugging with their own vision as a proxy for CV. By sharing our insights, we inform other designers and educators who seek alternative ways to facilitate physical computing and tangible interaction design.
more »
« less
This content will become publicly available on March 4, 2026
Physical Computing with Paper Playground: Exploring a Multimodal Platform
Physical computing enables learners to create interactive projects using tangible materials and electronic components. These projects commonly utilize microcontroller boards like the micro:bit. In contrast, computer vision (CV) is a powerful technique for detecting input through interaction with everyday materials like paper, and it can be utilized for physical computing projects. However, CV-based toolkits are typically limited to input detection and rely on screen-based or projected outputs. This paper presents a hybrid approach that integrates a CV-based platform called Paper Playground with the micro:bit electronics platform. By combining CV-detected, paper-based inputs with the rich input-output possibilities of microcontroller-based systems, we showcase a multimodal physical computing toolkit. Through three project examples, we explore how this hybrid approach can enhance the creative possibilities in physical computing, and develop a preliminary design space combining CV-based and electronics-based physical computing.
more »
« less
- Award ID(s):
- 2119303
- PAR ID:
- 10636975
- Publisher / Repository:
- ACM
- Date Published:
- ISBN:
- 9798400711978
- Page Range / eLocation ID:
- 1 to 7
- Subject(s) / Keyword(s):
- Computer Vision, Physical Computing, Paper, Tangible Interaction
- Format(s):
- Medium: X
- Location:
- Bordeaux/Talence France
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
De novo peptide design exhibits great potential in materials engineering, particularly for the use of plastic-binding peptides to help remediate microplastic pollution. There are no known peptide binders for many plastics—a gap that can be filled with de novo design. Current computational methods for peptide design exhibit limitations in sampling and scaling that could be addressed with quantum computing. Hybrid quantum-classical methods can leverage complementary strengths of near-term quantum algorithms and classical techniques for complex tasks like peptide design. This work introduces a hybrid quantum-classical generative framework for designing plastic-binding peptides combining variational quantum circuits with a variational autoencoder network. We demonstrate the framework’s effectiveness in generating peptide candidates, evaluate its efficiency for property-oriented design, and validate the candidates with molecular dynamics simulations. This quantum computing–based approach could accelerate the development of biomolecular tools for environmental and biomedical applications while advancing the study of biomolecular systems through quantum technologies.more » « less
-
With the rapid expansion of the Internet of Things, a vast number of microcontroller-based IoT devices are now susceptible to attacks through the Internet. Vulnerabilities within the firmware are one of the most important attack surfaces. Fuzzing has emerged as one of the most effective techniques for identifying such vulnerabilities. However, when applied to IoT firmware, several challenges arise, including: (1) the inability of firmware to execute properly in the absence of peripherals, (2) the lack of support for exploring input spaces of multiple peripherals, (3) difficulties in instrumenting and gathering feedback, and (4) the absence of a fault detection mechanism. To address these challenges, we have developed and implemented an innovative peripheral-independent hybrid fuzzing tool called . This tool enables testing of microcontroller-based firmware without reliance on specific peripheral hardware. First, a unified virtual peripheral was integrated to model the behaviors of various peripherals, thus enabling the physical devices-agnostic firmware execution. Then, a hybrid event generation approach was used to generate inputs for different peripheral accesses. Furthermore, two-level coverage feedback was collected to optimize the testcase generation. Finally, a plugin-based fault detection mechanism was implemented to identify typical memory corruption vulnerabilities. A Large-scale experimental evaluation has been performed to show ’s effectiveness and efficiency.more » « less
-
null (Ed.)Abstract When solving, modeling or reasoning about complex problems, it is usually convenient to use the knowledge of a parallel physical system for representing it. This is the case of lumped-circuit abstraction, which can be used for representing mechanical and acoustic systems, thermal and heat-diffusion problems and in general partial differential equations. Integrated photonic platforms hold the prospective to perform signal processing and analog computing inherently, by mapping into hardware specific operations which relies on the wave-nature of their signals, without trusting on logic gates and digital states like electronics. Here, we argue that in absence of a straightforward parallelism a homomorphism can be induced. We introduce a photonic platform capable of mimicking Kirchhoff’s law in photonics and used as node of a finite difference mesh for solving partial differential equation using monochromatic light in the telecommunication wavelength. Our approach experimentally demonstrates an arbitrary set of boundary conditions, generating a one-shot discrete solution of a Laplace partial differential equation, with an accuracy above 95% with respect to commercial solvers. Our photonic engine can provide a route to achieve chip-scale, fast (10 s of ps), and integrable reprogrammable accelerators for the next generation hybrid high-performance computing. Summary A photonic integrated platform which can mimic Kirchhoff’s law in photonics is used for approximately solve partial differential equations noniteratively using light, with high throughput and low-energy levels.more » « less
-
This article describes a sensor-based physical computing system, called the Data Sensor Hub (DaSH), which enables students to process, analyze, and display data streams collected using a variety of sensors. The system is built around the portable and affordable BBC micro:bit microcontroller (expanded with the gator:bit), which students program using a visual, cloud-based programming environment intended for novices. Students connect a variety of sensors (measuring temperature, humidity, carbon dioxide, sound, acceleration, magnetism, etc.) and write programs to analyze and visualize the collected sensor data streams. The article also describes two instructional units intended for middle grade science classes that use this sensor-based system. These inquiry-oriented units engage students in designing the system to collect data from the world around them to investigate scientific phenomena of interest. The units are designed to help students develop the ability to meaningfully integrate computing as they engage in place-based learning activities while using tools that more closely approximate the practices of contemporary scientists as well as other STEM workers. Finally, the article articulates how the DaSH and units have elicited different kinds of teacher practices using student drawn modeling activities, facilitating debugging practices, and developing place-based science practices.more » « less
