An official website of the United States government
Bayer pattern is a widely used Color Filter Array (CFA) for digital image sensors, efficiently capturing different light wavelengths on different pixels without the need for a costly ISP pipeline. The resulting single-channel raw Bayer images offer benefits such as spectral wavelength sensitivity and low time latency. However, object detection based on Bayer images has been underexplored due to challenges in human observation and algorithm design caused by the discontinuous color channels in adjacent pixels. To address this issue, we propose the BayerDetect network, an end-to-end deep object detection framework that aims to achieve fast, accurate, and memory-efficient object detection. Unlike RGB color images, where each pixel encodes spectral context from adjacent pixels during ISP color interpolation, raw Bayer images lack spectral context. To enhance the spectral context, the BayerDetect network introduces a spectral frequency attention block, transforming the raw Bayer image pattern to the frequency domain. In object detection, clear object boundaries are essential for accurate bounding box predictions. To handle the challenges posed by alternating spectral channels and mitigate the influence of discontinuous boundaries, the BayerDetect network incorporates a spatial attention scheme that utilizes deformable convolutional kernels in multiple scales to explore spatial context effectively. The extracted convolutional features are then passed through a sparse set of proposal boxes for detection and classification. We conducted experiments on both public and self-collected raw Bayer images, and the results demonstrate the superb performance of the BayerDetect network in object detection tasks.
more »
« less
Psychophysics of neon color spreading: Chromatic and temporal factors are not limiting
Neon color spreading (NCS) is an illusory color phenomenon that provides a dramatic example of surface completion and filling-in. Numerous studies have varied both spatial and temporal aspects of the neon- generating stimulus to explore variations in the strength of the effect. Here, we take a novel, parametric, low- level psychophysical approach to studying NCS in two experiments. In Experiment 1, we test the ability of both cone-isolating and equiluminant stimuli to generate neon color spreading for both increments and decre- ments in cone modulations. As expected, sensitivity was low to S(hort-wavelength) cone stimuli due to their poor spatial resolution, but sensitivity was similar for the other color directions. We show that when these differences in detection sensitivity are accounted for, the particular cone type, and the polarity (increment or decrement), make little difference in generating neon color spreading, with NCS visible at about twice detection threshold level in all cases. In Experiment 2, we use L-cone flicker modulations (reddish and greenish excursions around grey) to study sensitivity to NCS as a function of temporal frequency from 0.5 to 8 Hz. After accounting for detectability, the temporal contrast sensitivity functions for NCS are approximately constant or even increase over the studied frequency range. Therefore there is no evidence in this study that the processes underlying NCS are slower than the low-level processes of simple flicker detection. These results point to relatively fast mech- anisms, not slow diffusion processes, as the substrate for NCS.
more »
« less
- Award ID(s):
- 1921771
- PAR ID:
- 10547955
- Publisher / Repository:
- Vision Research
- Date Published:
- Journal Name:
- Vision Research
- Volume:
- 223
- Issue:
- C
- ISSN:
- 0042-6989
- Page Range / eLocation ID:
- 108460
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Bayer pattern is a widely used Color Filter Array (CFA) for digital image sensors, efficiently capturing different light wavelengths on different pixels without the need for a costly ISP pipeline. The resulting single-channel raw Bayer images offer benefits such as spectral wavelength sensitivity and low time latency. However, object detection based on Bayer images has been underexplored due to challenges in human observation and algorithm design caused by the discontinuous color channels in adjacent pixels. To address this issue, we propose the BayerDetect network, an end-to-end deep object detection framework that aims to achieve fast, accurate, and memory-efficient object detection. Unlike RGB color images, where each pixel encodes spectral context from adjacent pixels during ISP color interpolation, raw Bayer images lack spectral context. To enhance the spectral context, the BayerDetect network introduces a spectral frequency attention block, transforming the raw Bayer image pattern to the frequency domain. In object detection, clear object boundaries are essential for accurate bounding box predictions. To handle the challenges posed by alternating spectral channels and mitigate the influence of discontinuous boundaries, the BayerDetect network incorporates a spatial attention scheme that utilizes deformable convolutional kernels in multiple scales to explore spatial context effectively. The extracted convolutional features are then passed through a sparse set of proposal boxes for detection and classification. We conducted experiments on both public and self-collected raw Bayer images, and the results demonstrate the superb performance of the BayerDetect network in object detection tasks.more » « less
-
A prominent theoretical framework spanning philosophy, psychology, and neuroscience holds that selective attention penetrates early stages of perceptual processing to alter the subjective visual experience of behaviorally relevant stimuli. For example, searching for a red apple at the grocery store might make the relevant color appear brighter and more saturated compared with seeing the exact same red apple while searching for a yellow banana. In contrast, recent proposals argue that data supporting attention-related changes in appearance reflect decision- and motor-level response biases without concurrent changes in perceptual experience. Here, we tested these accounts by evaluating attentional modulations of EEG responses recorded from male and female human subjects while they compared the perceived contrast of attended and unattended visual stimuli rendered at different levels of physical contrast. We found that attention enhanced the amplitude of the P1 component, an early evoked potential measured over visual cortex. A linking model based on signal detection theory suggests that response gain modulations of the P1 component track attention-induced changes in perceived contrast as measured with behavior. In contrast, attentional cues induced changes in the baseline amplitude of posterior alpha band oscillations (∼9-12 Hz), an effect that best accounts for cue-induced response biases, particularly when no stimuli are presented or when competing stimuli are similar and decisional uncertainty is high. The observation of dissociable neural markers that are linked to changes in subjective appearance and response bias supports a more unified theoretical account and demonstrates an approach to isolate subjective aspects of selective information processing. SIGNIFICANCE STATEMENTDoes attention alter visual appearance, or does it simply induce response bias? In the present study, we examined these competing accounts using EEG and linking models based on signal detection theory. We found that response gain modulations of the visually evoked P1 component best accounted for attention-induced changes in visual appearance. In contrast, cue-induced baseline shifts in alpha band activity better explained response biases. Together, these results suggest that attention concurrently impacts visual appearance and response bias, and that these processes can be experimentally isolated.more » « less
-
Abstract The National Ecological Observatory Network Terrestrial Observation System (NEON TOS) produces open‐access data products that allow data users to investigate the impact of change drivers on key “sentinel” taxa and soils. The spatial and temporal sampling strategy that coordinates implementation of these protocols enables integration across TOS products and with products generated by NEON aquatic, remote sensing, and terrestrial instrument subsystems. Here, we illustrate the plots and sampling units that make up the physical foundation of a NEON TOS site, and we describe the scales (subplot, plot, airshed, and site) at which sampling is spatially colocated across protocols and subsystems. We also describe how moderate resolution imaging spectroradiometer‐enhanced vegetation index (MODIS‐EVI) phenology data are used to temporally coordinate TOS sampling within and across years at the continental scale of the observatory. Individually, TOS protocols produce data products that provide insight into populations, communities, and ecosystem processes. Within the spatial and temporal framework that guides cross‐protocol implementation, the ability to draw inference across data products is enhanced. To illustrate this point, we develop an example using R software that links two TOS data products collected with different temporal frequencies at both plot and site spatial scales. A thorough understanding of how TOS protocols are integrated with each other in space and time, and with other NEON subsystems, is necessary to leverage NEON data products to maximum effect. For example, a researcher must understand the spatial and temporal scales at which soil biogeochemistry data, soil microbe biomass data, and plant litter production and chemistry data may be combined to quantify soil nutrient stocks and fluxes across NEON sites. We present clear links among TOS protocols and across NEON subsystems that will enhance the utility of NEON TOS data products for the data user community.more » « less
-
Abstract Soil microbial communities play critical roles in various ecosystem processes, but studies at a large spatial and temporal scale have been challenging due to the difficulty in finding the relevant samples in available data sets as well as the lack of standardization in sample collection and processing. The National Ecological Observatory Network (NEON) has been collecting soil microbial community data multiple times per year for 47 terrestrial sites in 20 eco‐climatic domains, producing one of the most extensive standardized sampling efforts for soil microbial biodiversity to date. Here, we introduce the neonMicrobe R package—a suite of downloading, preprocessing, data set assembly, and sensitivity analysis tools for NEON’s newly published 16S and ITS amplicon sequencing data products which characterize soil bacterial and fungal communities, respectively. neonMicrobe is designed to make these data more accessible to ecologists without assuming prior experience with bioinformatic pipelines. We describe quality control steps used to remove quality‐flagged samples, report on sensitivity analyses used to determine appropriate quality filtering parameters for the DADA2 workflow, and demonstrate the immediate usability of the output data by conducting standard analyses of soil microbial diversity. The sequence abundance tables produced byneonMicrobecan be linked to NEON’s other data products (e.g., soil physical and chemical properties, plant community composition) and soil subsamples archived in the NEON Biorepository. We provide recommendations for incorporatingneonMicrobeinto reproducible scientific workflows, discuss technical considerations for large‐scale amplicon sequence analysis, and outline future directions for NEON‐enabled microbial ecology. In particular, we believe that NEON marker gene sequence data will allow researchers to answer outstanding questions about the spatial and temporal dynamics of soil microbial communities while explicitly accounting for scale dependence. We expect that the data produced by NEON and theneonMicrobeR package will act as a valuable ecological baseline to inform and contextualize future experimental and modeling endeavors.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.1016/j.visres.2024.108460
