skip to main content

Attention:

The NSF Public Access Repository (NSF-PAR) system and access will be unavailable from 11:00 PM ET on Thursday, October 10 until 2:00 AM ET on Friday, October 11 due to maintenance. We apologize for the inconvenience.


Title: Interpolant-based demosaicing routines for dual-mode visible/near-infrared imaging systems

Dual-mode visible/near-infrared imaging systems, including a bioinspired six-channel design and more conventional four-channel implementations, have transitioned from a niche in surveillance to general use in machine vision. However, the demosaicing routines that transform the raw images from these sensors into processed images that can be consumed by humans or computers rely on assumptions that may not be appropriate when the two portions of the spectrum contribute different information about a scene. A solution can be found in a family of demosaicing routines that utilize interpolating polynomials and splines of different dimensionalities and orders to process images with minimal assumptions.

 
more » « less
Award ID(s):
2030421
NSF-PAR ID:
10370817
Author(s) / Creator(s):
;
Publisher / Repository:
Optical Society of America
Date Published:
Journal Name:
Optics Express
Volume:
30
Issue:
19
ISSN:
1094-4087; OPEXFF
Format(s):
Medium: X Size: Article No. 34201
Size(s):
Article No. 34201
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    One of the key ideas for reducing downlink channel acquisition overhead for FDD massive MIMO systems is to exploit a combination of two assumptions: (i) the dimension of channel models in propagation domain may be much smaller than the next-generation base-station array sizes (e.g., 64 or more antennas), and (ii) uplink and downlink channels may share the same low-dimensional propagation domain. Our channel measurements demonstrate that the two assumptions may not always hold, thereby impacting the predicted performance of methods that rely on the above assumptions. In this paper, we analyze the error in modeling the downlink channel using uplink measurements, caused by the mismatch from the above two assumptions. We investigate how modeling error varies with base-station array size and provide both numerical and experimental results. We observe that modeling error increases with the number of base-station antennas, and channels with larger angular spreads have larger modeling error. Utilizing our modeling error analysis, we then investigate the resulting beamforming performance rate loss. Accordingly, we observe that the rate loss increases with the number of base-station antennas, and channels with larger angular spreads suffer from higher rate loss.

     
    more » « less
  2. null (Ed.)
    Understanding public media channel preferences can inform preparedness plans, response strategies, and long-term recovery. However, questions remain about how media consumption changes across pre-crisis, crisis, and post-crisis phases. Past theories argue that media use may change for several reasons, including during times of societal conflict and challenge. These theories point to the belief that, during a crisis, we expect media channel use to change because media preferences during a crisis will be fundamentally different compared to everyday routines. This paper takes advantage of a survey fielded to Texas residents soon after Hurricane Harvey. Here we ask: (1) What media channels are most prominent in each crisis phase? and (2) Do media channel preferences change across crisis phases? We use simple descriptive statistics and chi-squares tests to describe media channel preferences across the three crisis phases by demographics. Additionally, we use alluvial diagrams to visualize media channel preferences over time. In total, 62% (n=174) of respondents reported no changes in channel preferences. However, chi-square tests identified significant differences in media use changes related to a handful of demographic characteristics. These findings are explored alongside theories that would hypothesize likely media use changes across pre-crisis, crisis, and post-crisis phases. 
    more » « less
  3. There is a growing interest in designing models that can deal with images from different visual domains. If there exists a universal structure in different visual domains that can be captured via a common parameterization, then we can use a single model for all domains rather than one model per domain. A model aware of the relationships between different domains can also be trained to work on new domains with less resources. However, to identify the reusable structure in a model is not easy. In this paper, we propose a multi-domain learning architecture based on depthwise separable convolution. The proposed approach is based on the assumption that images from different domains share cross-channel correlations but have domain-specific spatial correlations. The proposed model is compact and has minimal overhead when being applied to new domains. Additionally, we introduce a gating mechanism to promote soft sharing between different domains. We evaluate our approach on Visual Decathlon Challenge, a benchmark for testing the ability of multi-domain models. The experiments show that our approach can achieve the highest score while only requiring 50% of the parameters compared with the state-of-the-art approaches. 
    more » « less
  4. Most neural networks assume that input images have a fixed number of channels (three for RGB images). However, there are many settings where the number of channels may vary, such as microscopy images where the number of channels changes depending on instruments and experimental goals. Yet, there has not been a systemic attempt to create and evaluate neural networks that are invariant to the number and type of channels. As a result, trained models remain specific to individual studies and are hardly reusable for other microscopy settings. In this paper, we present a benchmark for investigating channel-adaptive models in microscopy imaging, which consists of 1) a dataset of varied-channel single-cell images, and 2) a biologically relevant evaluation framework. In addition, we adapted several existing techniques to create channel-adaptive models and compared their performance on this benchmark to fixed-channel, baseline models. We find that channel-adaptive models can generalize better to out-of-domain tasks and can be computationally efficient. We contribute a curated dataset and an evaluation API to facilitate objective comparisons in future research and applications. 
    more » « less
  5. Major semantic segmentation approaches are designed for RGB color images, which is interpolated from raw Bayer images. The use of RGB images on the one hand provides abundant scene color information. On the other hand, RGB images are easily observable for human users to understand the scene. The RGB color continuity also facilitates researchers to design segmentation algorithms, which becomes unnecessary in end-to-end learning. More importantly, the use of 3 channels adds extra storage and computation burden for neural networks. In contrast, the raw Bayer images can reserve the primitive color information in the largest extent with just a single channel. The compact design of Bayer pattern not only potentially increases a higher segmentation accuracy because of avoiding interpolation, but also significantly decreases the storage requirement and computation time in comparison with standard R, G, B images. In this paper, we propose BayerSeg-Net to segment single channel raw Bayer image directly. Different from RGB color images that already contain neighboring context information during ISP color interpolation, each pixel in raw Bayer images does not contain any context clues. Based on Bayer pattern properties, BayerSeg-Net assigns dynamic attention on Bayer images' spectral frequency and spatial locations to mitigate classification confusion, and proposes a re-sampling strategy to capture both global and local contextual information. We demonstrate the usability of raw Bayer images in segmentation tasks and the efficiency of BayerSeg-Net on multiple datasets. 
    more » « less