More Like this

1. Interaction Harvesting: A Design Probe of User-Powered Widgets

   https://doi.org/10.1145/3610880  

   Mamish, John; Guo, Amy; Cohen, Thomas; Richey, Julian; Zhang, Yang; Hester, Josiah (September 2023, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)

   Whenever a user interacts with a device, mechanical work is performed to actuate the user interface elements; the resulting energy is typically wasted, dissipated as sound and heat. Previous work has shown that many devices can be powered entirely from this otherwise wasted user interface energy. For these devices, wires and batteries, along with the related hassles of replacement and charging, become unnecessary and onerous. So far, these works have been restricted to proof-of-concept demonstrations; a specific bespoke harvesting and sensing circuit is constructed for the application at hand. The challenge of harvesting energy while simultaneously sensing fine-grained input signals from diverse modalities makes prototyping new devices difficult. To fill this gap, we present a hardware toolkit which provides a common electrical interface for harvesting energy from user interface elements. This facilitates exploring the composability, utility, and breadth of enabled applications of interaction-powered smart devices. We design a set of energy as input harvesting circuits, a standard connective interface with 3D printed enclosures, and software libraries to enable the exploration of devices where the user action generates the energy needed to perform the device's primary function. This exploration culminated in a demonstration campaign where we prototype several exemplar popular toys and gadgets, including battery-free Bop-It--- a popular 90s rhythm game, an electronic Etch-a-sketch, a Simon-Says-style memory game, and a service rating device. We run exploratory user studies to understand how generativity, creativity, and composability are hampered or facilitated by these devices. These demonstrations, user study takeaways, and the toolkit itself provide a foundation for building interactive and user-focused gadgets whose usability is not affected by battery charge and whose service lifetime is not limited by battery wear. 

   more » « less
2. End-to-end Characterization of Game Streaming Applications on Mobile Platforms

   https://doi.org/10.1145/3508030  

   Bhuyan, Sandeepa; Zhao, Shulin; Ying, Ziyu; Kandemir, Mahmut T.; Das, Chita R. (February 2022, Proceedings of the ACM on Measurement and Analysis of Computing Systems)

   With the advent of 5G, supporting high-quality game streaming applications on edge devices has become a reality. This is evidenced by a recent surge in cloud gaming applications on mobile devices. In contrast to video streaming applications, interactive games require much more compute power for supporting improved rendering (such as 4K streaming) with the stipulated frames-per second (FPS) constraints. This in turn consumes more battery power in a power-constrained mobile device. Thus, the state-of-the-art gaming applications suffer from lower video quality (QoS) and/or energy efficiency. While there has been a plethora of recent works on optimizing game streaming applications, to our knowledge, there is no study that systematically investigates the design pairs on the end-to-end game streaming pipeline across the cloud, network, and edge devices to understand the individual contributions of the different stages of the pipeline for improving the overall QoS and energy efficiency. In this context, this paper presents a comprehensive performance and power analysis of the entire game streaming pipeline consisting of the server/cloud side, network, and edge. Through extensive measurements with a high-end workstation mimicking the cloud end, an open-source platform (Moonlight-GameStreaming) emulating the edge device/mobile platform, and two network settings (WiFi and 5G) we conduct a detailed measurement-based study with seven representative games with different characteristics. We characterize the performance in terms of frame latency, QoS, bitrate, and energy consumption for different stages of the gaming pipeline. Our study shows that the rendering stage and the encoding stage at the cloud end are the bottlenecks to support 4K streaming. While 5G is certainly more suitable for supporting enhanced video quality with 4K streaming, it is more expensive in terms of power consumption compared to WiFi. Further, fluctuations in 5G network quality can lead to huge frame drops thus affecting QoS, which needs to be addressed by a coordinated design between the edge device and the server. Finally, the network interface and the decoder units in a mobile platform need more energy-efficient design to support high quality games at a lower cost. These observations should help in designing more cost-effective future cloud gaming platforms. 

   more » « less
3. tinyMAN: Lightweight Energy Manager using Reinforcement Learning for Energy Harvesting Wearable IoT Devices

   https://doi.org/10.48550/arXiv.2202.09297  

   Basaklar, Toygun; Tuncel, Yigit; Ogras, Umit Y. (March 2022, TinyML Symposium)

   Advances in low-power electronics and machine learning techniques lead to many novel wearable IoT devices. These devices have limited battery capacity and computational power. Thus, energy harvesting from ambient sources is a promising solution to power these low-energy wearable devices. They need to manage the harvested energy optimally to achieve energy-neutral operation, which eliminates recharging requirements. Optimal energy management is a challenging task due to the dynamic nature of the harvested energy and the battery energy constraints of the target device. To address this challenge, we present a reinforcement learning-based energy management framework, tinyMAN, for resource-constrained wearable IoT devices. The framework maximizes the utilization of the target device under dynamic energy harvesting patterns and battery constraints. Moreover, tinyMAN does not rely on forecasts of the harvested energy which makes it a prediction-free approach. We deployed tinyMAN on a wearable device prototype using TensorFlow Lite for Micro thanks to its small memory footprint of less than 100 KB. Our evaluations show that tinyMAN achieves less than 2.36 ms and 27.75 μJ while maintaining up to 45% higher utility compared to prior approaches. 

   more » « less
4. User-directed Assembly Code Transformations Enabling Efficient Batteryless Arduino Applications

   https://doi.org/10.1145/3659590  

   Kraemer, Christopher; Gelder, William; Hester, Josiah (May 2024, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)

   The time for battery-free computing is now. Lithium mining depletes and pollutes local water supplies and dead batteries in landfills leak toxic metals into the ground. Battery-free devices represent a probable future for sustainable ubiquitous computing and we will need many more new devices and programmers to bring that future into reality. Yet, energy harvesting and battery-free devices that frequently fail are challenging to program. The maker movement has organically developed a considerable variety of platforms to prototype and program ubiquitous sensing and computing devices, but only a few have been modified to be usable with energy harvesting and to hide those pesky power failures that are the norm from variable energy availability (platforms like Microsoft's Makecode and AdaFruit's CircuitPython). Many platforms, especially Arduino (the first and most famous maker platform), do not support energy harvesting devices and intermittent computing. To bridge this gap and lay a strong foundation for potential new platforms for maker programming, we build a tool called BOOTHAMMER: a lightweight assembly re-writer for ARM Thumb. BOOTHAMMER analyzes and rewrites the low-level assembly to insert careful checkpoint and restore operations to enable programs to persist through power failures. The approach is easily insertable in existing toolchains and is general-purpose enough to be resilient to future platforms and devices/chipsets. We close the loop with the user by designing a small set of program annotations that any maker coder can use to provide extra information to this low-level tool that will significantly increase checkpoint efficiency and resolution. These optional extensions represent a way to include the user in decision-making about energy harvesting while ensuring the tool supports existing platforms. We conduct an extensive evaluation using various program benchmarks with Arduino as our chosen evaluation platform. We also demonstrate the usability of this approach by evaluating BOOTHAMMER with a user study and show that makers feel very confident in their ability to write intermittent computing programs using this tool. With this new tool, we enable maker hardware and software for sustainable, energy-harvesting-based computing for all. 

   more » « less
5. Battery-free MakeCode: Accessible Programming for Intermittent Computing

   https://doi.org/10.1145/3517236  

   Kraemer, Christopher; Guo, Amy; Ahmed, Saad; Hester, Josiah (March 2022, Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies)

   Hands-on computing has emerged as an exciting and accessible way to learn about computing and engineering in the physical world for students and makers of all ages. Current end-to-end approaches like Microsoft MakeCode require tethered or battery-powered devices like a micro:bit, limiting usefulness and applicability, as well as abdicating responsibility for teaching sustainable practices. Unfortunately, energy harvesting computing devices are usually only programmable by experts and require significant supporting toolchains and knowledge across multiple engineering and computing disciplines to work effectively. This paper bridges the gap between sustainable computing efforts, the maker movement, and novice-focused programming environments with MakeCode-Iceberg, a set of compiler extensions to Microsoft's open-source MakeCode project. The extensions automatically and invisibly transform user code in any language supported (Blocks, JavaScript, Python)into a version that can safely and correctly execute across intermittent power failures caused by unreliable energy harvesting. Determining where, when, and what to save in a checkpoint on limited energy, time, and hardware budget is challenging. We leverage the unique intermediate representation of the MakeCode source-to-source compiler to design and deploy various checkpointing techniques. Our approach allows us to provide, for the first time, a fully web-based and toolchain-free environment to program intermittent computing devices, making battery-free operation accessible to all. We demonstrate new use cases with multiple energy harvesters, peripherals, and application domains: including a Smart Terrarium, Step Counter, and Combination Lock. MakeCode-Iceberg provides sustainable hands-on computing opportunities to a broad audience of makers and learners, democratizing access to energy harvesting and battery-free embedded systems. 

   more » « less