More Like this

1. A Review of Holography in the Aquatic Sciences: In situ Characterization of Particles, Plankton, and Small Scale Biophysical Interactions

   https://doi.org/10.3389/fmars.2020.572147  

   Nayak, Aditya R. ; Malkiel, Ed ; McFarland, Malcolm N. ; Twardowski, Michael S. ; Sullivan, James M. ( January 2021 , Frontiers in Marine Science)

   null (Ed.)

   The characterization of particle and plankton populations, as well as microscale biophysical interactions, is critical to several important research areas in oceanography and limnology. A growing number of aquatic researchers are turning to holography as a tool of choice to quantify particle fields in diverse environments, including but not limited to, studies on particle orientation, thin layers, phytoplankton blooms, and zooplankton distributions and behavior. Holography provides a non-intrusive, free-stream approach to imaging and characterizing aquatic particles, organisms, and behavior in situ at high resolution through a 3-D sampling volume. Compared to other imaging techniques, e.g., flow cytometry, much larger volumes of water can be processed over the same duration, resolving particle sizes ranging from a few microns to a few centimeters. Modern holographic imaging systems are compact enough to be deployed through various modes, including profiling/towed platforms, buoys, gliders, long-term observatories, or benthic landers. Limitations of the technique include the data-intensive hologram acquisition process, computationally expensive image reconstruction, and coherent noise associated with the holograms that can make post-processing challenging. However, continued processing refinements, rapid advancements in computing power, and development of powerful machine learning algorithms for particle/organism classification are paving the way for holography to be used ubiquitously across different disciplines in the aquatic sciences. This review aims to provide a comprehensive overview of holography in the context of aquatic studies, including historical developments, prior research applications, as well as advantages and limitations of the technique. Ongoing technological developments that can facilitate larger employment of this technique toward in situ measurements in the future, as well as potential applications in emerging research areas in the aquatic sciences are also discussed. 

   more » « less
2. Automated plankton image analysis using convolutional neural networks

   https://doi.org/10.1002/lom3.10285  

   Luo, Jessica Y. ; Irisson, Jean‐Olivier ; Graham, Benjamin ; Guigand, Cedric ; Sarafraz, Amin ; Mader, Christopher ; Cowen, Robert K. ( October 2018 , Limnology and Oceanography: Methods)

   The rise of in situ plankton imaging systems, particularly high‐volume imagers such as the In Situ Ichthyoplankton Imaging System, has increased the need for fast processing and accurate classification tools that can identify a high diversity of organisms and nonliving particles of biological origin. Previous methods for automated classification have yielded moderate results that either can resolve few groups at high accuracy or many groups at relatively low accuracy. However, with the advent of new deep learning tools such as convolutional neural networks (CNNs), the automated identification of plankton images can be vastly improved. Here, we describe an image processing procedure that includes preprocessing, segmentation, classification, and postprocessing for the accurate identification of 108 classes of plankton using spatially sparse CNNs. Following a filtering process to remove images with low classification scores, a fully random evaluation of the classification showed that average precision was 84% and recall was 40% for all groups. Reliably classifying rare biological classes was difficult, so after excluding the 12 rarest taxa, classification accuracy for the remaining biological groups became > 90%. This method provides proof of concept for the effectiveness of an automated classification scheme using deep‐learning methods, which can be applied to a range of plankton or biological imaging systems, with the eventual application in a variety of ecological monitoring and fisheries management contexts.

    

   more » « less
3. Machine learning techniques to characterize functional traits of plankton from image data

   https://doi.org/10.1002/lno.12101  

   Orenstein, Eric C. ; Ayata, Sakina‐Dorothée ; Maps, Frédéric ; Becker, Érica C. ; Benedetti, Fabio ; Biard, Tristan ; de Garidel‐Thoron, Thibault ; Ellen, Jeffrey S. ; Ferrario, Filippo ; Giering, Sarah L. C. ; et al ( June 2022 , Limnology and Oceanography)

   Plankton imaging systems supported by automated classification and analysis have improved ecologists' ability to observe aquatic ecosystems. Today, we are on the cusp of reliably tracking plankton populations with a suite of lab‐based and in situ tools, collecting imaging data at unprecedentedly fine spatial and temporal scales. But these data have potential well beyond examining the abundances of different taxa; the individual images themselves contain a wealth of information on functional traits. Here, we outline traits that could be measured from image data, suggest machine learning and computer vision approaches to extract functional trait information from the images, and discuss promising avenues for novel studies. The approaches we discuss are data agnostic and are broadly applicable to imagery of other aquatic or terrestrial organisms.

    

   more » « less
4. FPGA-Based Acceleration of Time Series Similarity Prediction: From Cloud to Edge

   https://doi.org/10.1145/3555810  

   Kalantar, Amin ; Zimmerman, Zachary ; Brisk, Philip ( July 2022 , ACM Transactions on Reconfigurable Technology and Systems)

   With the proliferation of low-cost sensors and the Internet of Things, the rate of producing data far exceeds the compute and storage capabilities of today’s infrastructure. Much of this data takes the form of time series, and in response, there has been increasing interest in the creation of time series archives in the last decade, along with the development and deployment of novel analysis methods to process the data. The general strategy has been to apply a plurality of similarity search mechanisms to various subsets and subsequences of time series data in order to identify repeated patterns and anomalies; however, the computational demands of these approaches renders them incompatible with today’s power-constrained embedded CPUs. To address this challenge, we present FA-LAMP, an FPGA-accelerated implementation of the Learned Approximate Matrix Profile (LAMP) algorithm, which predicts the correlation between streaming data sampled in real-time and a representative time series dataset used for training. FA-LAMP lends itself as a real-time solution for time series analysis problems such as classification. We present the implementation of FA-LAMP on both edge- and cloud-based prototypes. On the edge devices, FA-LAMP integrates accelerated computation as close as possible to IoT sensors, thereby eliminating the need to transmit and store data in the cloud for posterior analysis. On the cloud-based accelerators, FA-LAMP can execute multiple LAMP models on the same board, allowing simultaneous processing of incoming data from multiple data sources across a network. LAMP employs a Convolutional Neural Network (CNN) for prediction. This work investigates the challenges and limitations of deploying CNNs on FPGAs using the Xilinx Deep Learning Processor Unit (DPU) and the Vitis AI development environment. We expose several technical limitations of the DPU, while providing a mechanism to overcome them by attaching custom IP block accelerators to the architecture. We evaluate FA-LAMP using a low-cost Xilinx Ultra96-V2 FPGA as well as a cloud-based Xilinx Alveo U280 accelerator card and measure their performance against a prototypical LAMP deployment running on a Raspberry Pi 3, an Edge TPU, a GPU, a desktop CPU, and a server-class CPU. In the edge scenario, the Ultra96-V2 FPGA improved performance and energy consumption compared to the Raspberry Pi; in the cloud scenario, the server CPU and GPU outperformed the Alveo U280 accelerator card, while the desktop CPU achieved comparable performance; however, the Alveo card offered an order of magnitude lower energy consumption compared to the other four platforms. Our implementation is publicly available at https://github.com/aminiok1/lamp-alveo. 

   more » « less
5. Keep Clear of the Edges : An Empirical Study of Artificial Intelligence Workload Performance and Resource Footprint on Edge Devices

   https://doi.org/10.1109/IPCCC55026.2022.9894338  

   Suo, Kun ; Nguyen, Tu N. ; Shi, Yong ; He, Jing Selena ; Hung, Chih-Cheng ( November 2022 , 2022 IEEE International Performance, Computing, and Communications Conference (IPCCC))

   Recently, with the advent of the Internet of everything and 5G network, the amount of data generated by various edge scenarios such as autonomous vehicles, smart industry, 4K/8K, virtual reality (VR), augmented reality (AR), etc., has greatly exploded. All these trends significantly brought real-time, hardware dependence, low power consumption, and security requirements to the facilities, and rapidly popularized edge computing. Meanwhile, artificial intelligence (AI) workloads also changed the computing paradigm from cloud services to mobile applications dramatically. Different from wide deployment and sufficient study of AI in the cloud or mobile platforms, AI workload performance and their resource impact on edges have not been well understood yet. There lacks an in-depth analysis and comparison of their advantages, limitations, performance, and resource consumptions in an edge environment. In this paper, we perform a comprehensive study of representative AI workloads on edge platforms. We first conduct a summary of modern edge hardware and popular AI workloads. Then we quantitatively evaluate three categories (i.e., classification, image-to-image, and segmentation) of the most popular and widely used AI applications in realistic edge environments based on Raspberry Pi, Nvidia TX2, etc. We find that interaction between hardware and neural network models incurs non-negligible impact and overhead on AI workloads at edges. Our experiments show that performance variation and difference in resource footprint limit availability of certain types of workloads and their algorithms for edge platforms, and users need to select appropriate workload, model, and algorithm based on requirements and characteristics of edge environments. 

   more » « less