Applications of neural networks have gained significant importance in embedded mobile devices and Internet of Things (IoT) nodes. In particular, convolutional neural networks have emerged as one of the most powerful techniques in computer vision, speech recognition, and AI applications that can improve the mobile user experience. However, satisfying all power and performance requirements of such low power devices is a significant challenge. Recent work has shown that binarizing a neural network can significantly improve the memory requirements of mobile devices at the cost of minor loss in accuracy. This paper proposes MB-CNN, a memristive accelerator for binary convolutional neural networks that perform XNOR convolution in-situ novel 2R memristive data blocks to improve power, performance, and memory requirements of embedded mobile devices. The proposed accelerator achieves at least 13.26 × , 5.91 × , and 3.18 × improvements in the system energy efficiency (computed by energy × delay) over the state-of-the-art software, GPU, and PIM architectures, respectively. The solution architecture which integrates CPU, GPU and MB-CNN outperforms every other configuration in terms of system energy and execution time.
more »
« less
This content will become publicly available on July 1, 2024
Evolution of Winning Solutions in the 2021 Low-Power Computer Vision Challenge
Mobile and embedded devices are becoming ubiquitous. Applications such as rescue with
autonomous robots and event analysis on traffic cameras rely on devices with limited power
supply and computational sources. Thus, the demand for efficient computer vision algorithms
increases. Since 2015, we have organized the IEEE Low-Power Computer Vision Challenge to
advance the state of the art in low-power computer vision. We describe the competition
organizing details including the challenge design, the reference solution, the dataset, the referee
system, and the evolution of the solutions from two winning teams. We examine the winning
teams’ development patterns and design decisions, focusing on their techniques to balance
power consumption and accuracy. We conclude that a successful competition needs a
well-designed reference solution and automated referee system, and a solution with modularized
components is more likely to win. We hope this paper provides guidelines for future organizers
and contestants of computer vision competitions.
more »
« less
- Award ID(s):
- 2107230
- NSF-PAR ID:
- 10427466
- Date Published:
- Journal Name:
- IEEE intelligent systems
- ISSN:
- 1941-1294
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Computer vision on low-power edge devices enables applications including search-and-rescue and security. State-of-the-art computer vision algorithms, such as Deep Neural Networks (DNNs), are too large for inference on low-power edge devices. To improve efficiency, some existing approaches parallelize DNN inference across multiple edge devices. How-ever, these techniques introduce significant communication and synchronization overheads or are unable to balance workloads across devices. This paper demonstrates that the hierarchical DNN architecture is well suited for parallel processing on multiple edge devices. We design a novel method that creates a parallel inference pipeline for computer vision problems that use hierarchical DNNs. The method balances loads across the collaborating devices and reduces communication costs to facilitate the processing of multiple video frames simultaneously with higher throughput. Our experiments consider a representative computer vision problem where image recognition is performed on each video frame, running on multiple Raspberry Pi 4Bs. With four collaborating low-power edge devices, our approach achieves 3.21× higher throughput, 68% less energy consumption per device per frame, and a 58% decrease in memory when compared with existing sinaledevice hierarchical DNNs.more » « less
-
more » « less
Premise Plant biodiversity is threatened, yet many species remain undescribed. It is estimated that >50% of undescribed species have already been collected and are awaiting discovery in herbaria. Robust automatic species identification algorithms using machine learning could accelerate species discovery.
Methods To encourage the development of an automatic species identification algorithm, we submitted our Herbarium 2019 data set to the Fine‐Grained Visual Categorization sub‐competition (FGVC6) hosted on the Kaggle platform. We chose to focus on the flowering plant family Melastomataceae because we have a large collection of imaged herbarium specimens (46,469 specimens representing 683 species) and taxonomic expertise in the family. As is common for herbarium collections, some species in this data set are represented by few specimens and others by many.
Results In less than three months, the FGVC6 Herbarium 2019 Challenge drew 22 teams who entered 254 models for Melastomataceae species identification. The four best algorithms identified species with >88% accuracy.
Discussion The FGVC competitions provide a unique opportunity for computer vision and machine learning experts to address difficult species‐recognition problems. The Herbarium 2019 Challenge brought together a novel combination of collections resources, taxonomic expertise, and collaboration between botanists and computer scientists.
-
Virtual Reality (VR), together with the network infrastructure, can provide an interactive and immersive experience for multiple users simultaneously and thus enables collaborative VR applications (e.g., VR-based classroom). However, the satisfactory user experience requires not only high-resolution panoramic image rendering but also extremely low latency and seamless user experience. Besides, the competition for limited network resources (e.g., multiple users share the total limited bandwidth) poses a significant challenge to collaborative user experience, in particular under the wireless network with time-varying capacities. While existing works have tackled some of these challenges, a principled design considering all those factors is still missing. In this paper, we formulate a combinatorial optimization problem to maximize the Quality of Experience (QoE), defined as the linear combination of the quality, the average VR content delivery delay, and variance of the quality over a finite time horizon. In particular, we incorporate the influence of imperfect motion prediction when considering the quality of the perceived contents. However, the optimal solution to this problem can not be implemented in real-time since it relies on future decisions. Then, we decompose the optimization problem into a series of combinatorial optimization in each time slot and develop a low-complexity algorithm that can achieve at least 1/2 of the optimal value. Despite this, the trace-based simulation results reveal that our algorithm performs very close to the optimal offline solution. Furthermore, we implement our proposed algorithm in a practical system with commercial mobile devices and demonstrate its superior performance over state-of-the-art algorithms. We open-source our implementations on https://github.com/SNeC-Lab-PSU/ICDCS-CollaborativeVR.more » « less
-
more » « less
Abstract Textile-based compression devices are widely used in fields such as healthcare, astronautics, cosmetics, defense, and more. While traditional compression garments are only able to apply passive pressure on the body, there have been some efforts to integrate smart materials such as shape memory alloys (SMAs) to make compression garments active and controllable. However, despite the advances in this field, accurate control of applied pressure on the body due remains a challenge due to vast population-scale anthropometric variability and intra-subjects variability in tissue softness, even if the actuators themselves are fully characterized. In this study, we begin to address these challenges by developing a novel size-adjustable SMA-based smart tourniquet capable of producing a controllable pressure for circumferential applications. The developed prototype was tested on an inflatable pressure cuff wrapped around a rigid cylinder. The thermal activation of SMA coils was achieved through Joule heating, and a microcontroller and a programmable power supply are used to provide the input signal. To control the compression force, a closed-loop PID controller was implemented, and the performance of the system was evaluated in 5 different testing conditions for variable and cyclic compression levels. The experiments showed that the controlled system could follow the desired control pressure reference with a steady-state of 1 mmHg. The compression tourniquet is able to produce more than 33 mmHg with an average actuation rate of 0.19 mmHg/s. This is the first demonstration of accurate closed-loop control of an SMA-incorporated compression technology to the best of our knowledge. This paper enables new, dynamic systems with controllable activation and low-effort donning and doffing, with applications ranging from healthcare solutions to advanced spacesuit design.
