More Like this

1. Predictive Sampling in Image Sensing for Sparse Image Processing

   https://doi.org/10.1109/LSENS.2024.3520408  

   Biglari, Amin; Hu, Qisong; Tang, Wei (January 2025, IEEE Sensors Letters)

   This paper presents a pixel-level predictive sampling method for image sensing and processing to reduce the computing overhead for power-limited image sensing systems. The predictive sampling method scans through rows and columns to identify the location and value of the critical pixels, which are the turning points in the row and column arrays. The prediction is performed using the value of prior pixels and a pre-defined error threshold. When the prediction is successful, the pixel is marked as a non-critical pixel and is skipped for recording and processing. Only the critical pixels are selected for further processing. We proposed reconstruction methods that recover the raw image from the selected critical pixels using interpolation. The experimental results show that the proposed method can reduce the data throughput by 72% with an error of 1.6% for sparse images. The convolutional neural network model applied with this method can achieve a similar detection accuracy in a standard method while only using 27.1% of data size. 

   more » « less
2. SINCO: A Novel Structural Regularizer for Image Compression Using Implicit Neural Representations

   https://doi.org/10.1109/ICASSP49357.2023.10095531  

   Gao, Harry; Gan, Weijie; Sun, Zhixin; Kamilov, Ulugbek S. (June 2023, Proceedings of the IEEE International Conference on Acoustics Speech and Signal Processing)

   Implicit neural representations (INR) have been recently proposed as deep learning (DL) based solutions for image compression. An image can be compressed by training an INR model with fewer weights than the number of image pixels to map the coordinates of the image to corresponding pixel values. While traditional training approaches for INRs are based on enforcing pixel-wise image consistency, we propose to further improve image quality by using a new structural regularizer. We present structural regularization for INR compression (SINCO) as a novel INR method for image compression. SINCO imposes structural consistency of the compressed images to the groundtruth by using a segmentation network to penalize the discrepancy of segmentation masks predicted from compressed images. We validate SINCO on brain MRI images by showing that it can achieve better performance than some recent INR methods. 

   more » « less
3. Quantifying and Reducing Registration Uncertainty of Spatial Vector Labels on Earth Imagery

   https://doi.org/10.1145/3534678.3539410  

   He, Wenchong; Jiang, Zhe; Kriby, Marcus; Xie, Yiqun; Jia, Xiaowei; Yan, Da; Zhou, Yang (August 2022, Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining)

   Given raster imagery features and imperfect vector training labels with registration uncertainty, this paper studies a deep learning framework that can quantify and reduce the registration uncertainty of training labels as well as train neural network parameters simultaneously. The problem is important in broad applications such as streamline classification on Earth imagery or tissue segmentation on medical imagery, whereby annotating precise vector labels is expensive and time-consuming. However, the problem is challenging due to the gap between the vector representation of class labels and the raster representation of image features and the need for training neural networks with uncertain label locations. Existing research on uncertain training labels often focuses on uncertainty in label class semantics or characterizes label registration uncertainty at the pixel level (not contiguous vectors). To fill the gap, this paper proposes a novel learning framework that explicitly quantifies vector labels' registration uncertainty. We propose a registration-uncertainty-aware loss function and design an iterative uncertainty reduction algorithm by re-estimating the posterior of true vector label locations distribution based on a Gaussian process. Evaluations on real-world datasets in National Hydrography Dataset refinement show that the proposed approach significantly outperforms several baselines in the registration uncertainty estimations performance and classification performance. 

   more » « less
4. DropConnect is effective in modeling uncertainty of Bayesian deep networks

   https://doi.org/10.1038/s41598-021-84854-x  

   Mobiny, Aryan; Yuan, Pengyu; Moulik, Supratik K.; Garg, Naveen; Wu, Carol C.; Van Nguyen, Hien (December 2021, Scientific Reports)

   null (Ed.)

   Abstract Deep neural networks (DNNs) have achieved state-of-the-art performance in many important domains, including medical diagnosis, security, and autonomous driving. In domains where safety is highly critical, an erroneous decision can result in serious consequences. While a perfect prediction accuracy is not always achievable, recent work on Bayesian deep networks shows that it is possible to know when DNNs are more likely to make mistakes. Knowing what DNNs do not know is desirable to increase the safety of deep learning technology in sensitive applications; Bayesian neural networks attempt to address this challenge. Traditional approaches are computationally intractable and do not scale well to large, complex neural network architectures. In this paper, we develop a theoretical framework to approximate Bayesian inference for DNNs by imposing a Bernoulli distribution on the model weights. This method called Monte Carlo DropConnect (MC-DropConnect) gives us a tool to represent the model uncertainty with little change in the overall model structure or computational cost. We extensively validate the proposed algorithm on multiple network architectures and datasets for classification and semantic segmentation tasks. We also propose new metrics to quantify uncertainty estimates. This enables an objective comparison between MC-DropConnect and prior approaches. Our empirical results demonstrate that the proposed framework yields significant improvement in both prediction accuracy and uncertainty estimation quality compared to the state of the art. 

   more » « less
5. A 1.2 Billion Pixel Human-Labeled Dataset for Data-Driven Classification of Coastal Environments

   https://doi.org/10.1038/s41597-023-01929-2  

   Buscombe, Daniel; Wernette, Phillipe; Fitzpatrick, Sharon; Favela, Jaycee; Goldstein, Evan B.; Enwright, Nicholas M. (December 2023, Scientific Data)

   Abstract The world’s coastlines are spatially highly variable, coupled-human-natural systems that comprise a nested hierarchy of component landforms, ecosystems, and human interventions, each interacting over a range of space and time scales. Understanding and predicting coastline dynamics necessitates frequent observation from imaging sensors on remote sensing platforms. Machine Learning models that carry out supervised (i.e., human-guided) pixel-based classification, or image segmentation, have transformative applications in spatio-temporal mapping of dynamic environments, including transient coastal landforms, sediments, habitats, waterbodies, and water flows. However, these models require large and well-documented training and testing datasets consisting of labeled imagery. We describe “Coast Train,” a multi-labeler dataset of orthomosaic and satellite images of coastal environments and corresponding labels. These data include imagery that are diverse in space and time, and contain 1.2 billion labeled pixels, representing over 3.6 million hectares. We use a human-in-the-loop tool especially designed for rapid and reproducible Earth surface image segmentation. Our approach permits image labeling by multiple labelers, in turn enabling quantification of pixel-level agreement over individual and collections of images. 

   more » « less