Deep convolutional neural networks (CNNs) are becoming increasingly popular models to predict neural responses in visual cortex. However, contextual effects, which are prevalent in neural processing and in perception, are not explicitly handled by current CNNs, including those used for neural prediction. In primary visual cortex, neural responses are modulated by stimuli spatially surrounding the classical receptive field in rich ways. These effects have been modeled with divisive normalization approaches, including flexible models, where spatial normalization is recruited only to the degree that responses from center and surround locations are deemed statistically dependent. We propose a flexible normalization model applied to midlevel representations of deep CNNs as a tractable way to study contextual normalization mechanisms in midlevel cortical areas. This approach captures nontrivial spatial dependencies among midlevel features in CNNs, such as those present in textures and other visual stimuli, that arise from tiling high-order features geometrically. We expect that the proposed approach can make predictions about when spatial normalization might be recruited in midlevel cortical areas. We also expect this approach to be useful as part of the CNN tool kit, therefore going beyond more restrictive fixed forms of normalization.
more »
« less
Exploiting Energy-Accuracy Trade-off through Contextual Awareness in Multi-Stage Convolutional Neural Networks
One of the promising solutions for energy-efficient CNNs is to break them down into multiple stages that are executed sequentially (MS-CNN). In this paper, we illustrate that unlike deep CNNs, MS-CNNs develop a form of contextual awareness of input data in initial stages, which could be used to dynamically change the structure and connectivity of such networks to reduce their computational complexity, making them a better fit for low-power and real-time systems. We suggest three run-time optimization policies, which are capable of exploring such contextual knowledge, and illustrate how the proposed policies construct a dynamic architecture suitable for a wide range of applications with varied accuracy requirements, resources, and time-budget, without further need for network re-training. Moreover, we propose variable and dynamic bit-length fixed-point conversion to further reduce the memory footprint of the MS-CNNs.
more »
« less
- NSF-PAR ID:
- 10114202
- Date Published:
- Journal Name:
- 20th International Symposium on Quality Electronic Design (ISQED)
- Page Range / eLocation ID:
- 265 to 270
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
null (Ed.)Deep Convolutional Neural Networks (CNNs) now match human accuracy in many image prediction tasks, resulting in a growing adoption in e-commerce, radiology, and other domains. Naturally, "explaining" CNN predictions is a key concern for many users. Since the internal workings of CNNs are unintuitive for most users, occlusion-based explanations (OBE) are popular for understanding which parts of an image matter most for a prediction. One occludes a region of the image using a patch and moves it around to produce a heatmap of changes to the prediction probability. This approach is computationally expensive due to the large number of re-inference requests produced, which wastes time and raises resource costs. We tackle this issue by casting the OBE task as a new instance of the classical incremental view maintenance problem. We create a novel and comprehensive algebraic framework for incremental CNN inference combining materialized views with multi-query optimization to reduce computational costs. We then present two novel approximate inference optimizations that exploit the semantics of CNNs and the OBE task to further reduce runtimes. We prototype our ideas in a tool we call Krypton. Experiments with real data and CNNs show that Krypton reduces runtimes by up to 5x (resp. 35x) to produce exact (resp. high-quality approximate) results without raising resource requirements.more » « less
-
more » « less
Abstract Multireservoir systems are designed to serve multiple conflicting demands over varying time scales that may be out of phase with the system's hydroclimatic inputs. Adaptive, nonlinear reservoir control policies are often best suited to serve these needs. However, nonlinear operating policies are hard to interpret, so water managers tend to favor simple, static rules that may not effectively manage conflicts between the system's multisectoral demands. In this study, we introduce an analytical framework for opening the black box of optimized nonlinear operating policies, decomposing their time‐varying information sensitivities to show how their adaptive and coordinated release prescriptions better manage hydrologic variability. Interestingly, these information sensitivities vary significantly across policies depending on how they negotiate tradeoffs between conflicting objectives. We illustrate this analysis in the Red River basin of Vietnam, where four major reservoirs serve to protect the capital of Hanoi from flooding while also providing the surrounding region with electric power and meeting multisectoral water demands for the agricultural and urban economies. Utilizing Evolutionary Multi‐Objective Direct Policy Search, we are able to design policies that, using the same information as sequential if/then/else‐based operating guidelines developed by the government, outperform these traditional rules with respect to every objective. Policy diagnostics using time‐varying sensitivity analysis illustrate how the Evolutionary Multi‐Objective Direct Policy Search operations better adapt and coordinate information use to reduce food‐energy‐water conflicts in the basin. These findings accentuate the benefits of transitioning to dynamic operating policies in order to manage evolving hydroclimatic variability and socioeconomic demands in multipurpose reservoir networks.
-
null (Ed.)CNNs (Convolutional Neural Networks) are becoming increasingly important for real-time applications, such as image classification in traffic control, visual surveillance, and smart manufacturing. It is challenging, however, to meet timing constraints of image processing tasks using CNNs due to their complexity. Performing dynamic trade-offs between the inference accuracy and time for image data analysis in CNNs is challenging too, since we observe that more complex CNNs that take longer to run even lead to lower accuracy in many cases by evaluating hundreds of CNN models in terms of time and accuracy using two popular data sets, MNIST and CIFAR-10. To address these challenges, we propose a new approach that (1) generates CNN models and analyzes their average inference time and accuracy for image classification, (2) stores a small subset of the CNNs with monotonic time and accuracy relationships offline, and (3) efficiently selects an effective CNN expected to support the highest possible accuracy among the stored CNNs subject to the remaining time to the deadline at run time. In our extensive evaluation, we verify that the CNNs derived by our approach are more flexible and cost-efficient than two baseline approaches. We verify that our approach can effectively build a compact set of CNNs and efficiently support systematic time vs. accuracy trade-offs, if necessary, to meet the user-specified timing and accuracy requirements. Moreover, the overhead of our approach is little/acceptable in terms of latency and memory consumption.more » « less
-
more » « less
Abstract Advances in both hardware and software are enabling rapid proliferation of in situ plankton imaging methods, requiring more effective machine learning approaches to image classification. Deep Learning methods, such as convolutional neural networks (CNNs), show marked improvement over traditional feature‐based supervised machine learning algorithms, but require careful optimization of hyperparameters and adequate training sets. Here, we document some best practices in applying CNNs to zooplankton and marine snow images and note where our results differ from contemporary Deep Learning findings in other domains. We boost the performance of CNN classifiers by incorporating metadata of different types and illustrate how to assimilate metadata beyond simple concatenation. We utilize both geotemporal (e.g., sample depth, location, time of day) and hydrographic (e.g., temperature, salinity, chlorophyll
a ) metadata and show that either type by itself, or both combined, can substantially reduce error rates. Incorporation of context metadata also boosts performance of the feature‐based classifiers we evaluated: Random Forest, Extremely Randomized Trees, Gradient Boosted Classifier, Support Vector Machines, and Multilayer Perceptron. For our assessments, we use an original data set of 350,000 in situ images (roughly 50% marine snow and 50% non‐snow sorted into 26 categories) from a novel in situZooglider . We document asymptotically increasing performance with more computationally intensive techniques, such as substantially deeper networks and artificially augmented data sets. Our best model achieves 92.3% accuracy with our 27‐class data set. We provide guidance for further refinements that are likely to provide additional gains in classifier accuracy.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/ISQED.2019.8697497
