Face recognition (FR) systems are fast becoming ubiquitous. However, differential performance among certain demographics was identified in several widely used FR models. The skin tone of the subject is an important factor in addressing the differential performance. Previous work has used modeling methods to propose skin tone measures of subjects across different illuminations or utilized subjective labels of skin color and demographic information. However, such models heavily rely on consistent background and lighting for calibration, or utilize labeled datasets, which are time-consuming to generate or are unavailable. In this work, we have developed a novel and data-driven skin color measure capable of accurately representing subjects' skin tone from a single image, without requiring a consistent background or illumination. Our measure leverages the dichromatic reflection model in RGB space to decompose skin patches into diffuse and specular bases.
more »
« less
CLTS-GAN: Color-Lighting-Texture-Specular Reflection Augmentation for Colonoscopy
Automatedanalysisofopticalcolonoscopy(OC)videoframes (to assist endoscopists during OC) is challenging due to variations in color, lighting, texture, and specular reflections. Previous methods ei- ther remove some of these variations via preprocessing (making pipelines cumbersome) or add diverse training data with annotations (but expen- sive and time-consuming). We present CLTS-GAN, a new deep learning model that gives fine control over color, lighting, texture, and specular reflection synthesis for OC video frames. We show that adding these colonoscopy-specific augmentations to the training data can improve state-of-the-art polyp detection/segmentation methods as well as drive next generation of OC simulators for training medical students. The code and pre-trained models for CLTS-GAN are available on Computational Endoscopy Platform GitHub (https://github.com/nadeemlab/CEP).
more »
« less
- Award ID(s):
- 1650499
- NSF-PAR ID:
- 10399956
- Date Published:
- Journal Name:
- Springer
- ISSN:
- 0947-5427
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Cloud detection is an inextricable pre-processing step in remote sensing image analysis workflows. Most of the traditional rule-based and machine-learning-based algorithms utilize low-level features of the clouds and classify individual cloud pixels based on their spectral signatures. Cloud detection using such approaches can be challenging due to a multitude of factors including harsh lighting conditions, the presence of thin clouds, the context of surrounding pixels, and complex spatial patterns. In recent studies, deep convolutional neural networks (CNNs) have shown outstanding results in the computer vision domain. These methods are practiced for better capturing the texture, shape as well as context of images. In this study, we propose a deep learning CNN approach to detect cloud pixels from medium-resolution satellite imagery. The proposed CNN accounts for both the low-level features, such as color and texture information as well as high-level features extracted from successive convolutions of the input image. We prepared a cloud-pixel dataset of approximately 7273 randomly sampled 320 by 320 pixels image patches taken from a total of 121 Landsat-8 (30m) and Sentinel-2 (20m) image scenes. These satellite images come with cloud masks. From the available data channels, only blue, green, red, and NIR bands are fed into the model. The CNN model was trained on 5300 image patches and validated on 1973 independent image patches. As the final output from our model, we extract a binary mask of cloud pixels and non-cloud pixels. The results are benchmarked against established cloud detection methods using standard accuracy metrics.more » « less
-
more » « less
Pancreatic ductal adenocarcinoma (PDAC) presents a critical global health challenge, and early detection is crucial for improving the 5-year survival rate. Recent medical imaging and computational algorithm advances offer potential solutions for early diagnosis. Deep learning, particularly in the form of convolutional neural networks (CNNs), has demonstrated success in medical image analysis tasks, including classification and segmentation. However, the limited availability of clinical data for training purposes continues to represent a significant obstacle. Data augmentation, generative adversarial networks (GANs), and cross-validation are potential techniques to address this limitation and improve model performance, but effective solutions are still rare for 3D PDAC, where the contrast is especially poor, owing to the high heterogeneity in both tumor and background tissues. In this study, we developed a new GAN-based model, named 3DGAUnet, for generating realistic 3D CT images of PDAC tumors and pancreatic tissue, which can generate the inter-slice connection data that the existing 2D CT image synthesis models lack. The transition to 3D models allowed the preservation of contextual information from adjacent slices, improving efficiency and accuracy, especially for the poor-contrast challenging case of PDAC. PDAC’s challenging characteristics, such as an iso-attenuating or hypodense appearance and lack of well-defined margins, make tumor shape and texture learning challenging. To overcome these challenges and improve the performance of 3D GAN models, our innovation was to develop a 3D U-Net architecture for the generator, to improve shape and texture learning for PDAC tumors and pancreatic tissue. Thorough examination and validation across many datasets were conducted on the developed 3D GAN model, to ascertain the efficacy and applicability of the model in clinical contexts. Our approach offers a promising path for tackling the urgent requirement for creative and synergistic methods to combat PDAC. The development of this GAN-based model has the potential to alleviate data scarcity issues, elevate the quality of synthesized data, and thereby facilitate the progression of deep learning models, to enhance the accuracy and early detection of PDAC tumors, which could profoundly impact patient outcomes. Furthermore, the model has the potential to be adapted to other types of solid tumors, hence making significant contributions to the field of medical imaging in terms of image processing models.
-
Recent studies have shown that, experiencing the appropriate lighting environment in our day-to-day life is paramount, as different types of light sources impact our mental and physical health in many ways. Researchers have intercon-nected daylong exposure of natural and artificial lights with circadian health, sleep and productivity. That is why having a generalized system to monitor human light exposure and recommending lighting adjustments can be instrumental for maintaining a healthy lifestyle. At present methods for collecting daylong light exposure information and source identification contain certain limitations. Sensing devices are expensive and power consuming and methods of classifications are either inac-curate or possesses certain limitations. In addition, identifying the source of exposure is challenging for a couple of reasons. For example, spectral based classification can be inaccurate, as different sources share common spectral bands or same source can exhibit variation in spectrum. Also irregularities of sensed information in real world makes scenario complex for source identification. In this work, we are presenting a Low Power BLE enabled Color Sensing Board (LPCSB) for sensing background light parameters. Later, utilizing Machine learning and Neural Network based architectures, we try to pinpoint the prime source in the surrounding among four dissimilar types: Incandescent, LED, CFL and Sunlight. Our experimentation includes 27 distinct bulbs and sunlight data in various weather/time of the day/spaces. After tuning classifiers, we have investigated best parameter settings for indoor deployment and also analyzed robustness of each classifier in several imperfect situations. As observed performance degraded significantly after real world deployment, we include synthetic time series examples and filtered data in the training set for boosting accuracy. Result shows that our best model can detect the primary light source type in the surroundings with accuracy up to 99.30% in familiar and up to 90.25% in unfamiliar real world settings with enlarged training set, which is much elevated than earlier endeavors.more » « less
-
more » « less
Abstract This study delves into the polarization properties of various hair colors using several techniques, including polarization ray tracing, full Stokes, and Mueller matrix imaging. Our analysis involved studying hair in both indoor and outdoor settings under varying lighting conditions. Our results demonstrate a strong correlation between hair color and the degree of linear polarization. Specifically, light-colored hair, such as white and blond, exhibits high albedo and low DoLP. In contrast, dark hair, like black and brown hair, has low albedo and high DoLP. Our research also revealed that a single hair strand displays high diattenuation near specular reflections but high depolarization in areas with diffuse reflections. Additionally, we investigated the wavelength dependency of the polarization properties by comparing the Mueller matrix under illumination at 450 nm and 589 nm. Our investigation demonstrates the impact of hair shade and color on polarization properties and the Umov effect.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
