An official website of the United States government
Lu et al. (2025) proved that HDRI camera sensors from different viewpoints can capture consistent and transferable luminance patterns in daylit spaces through Conditional Generative Adversarial Networks (CGANs). Building on that, this paper validates that non-intrusive luminance monitoring can be used to evaluate daylighting preferences, using collected experimental datasets with human subjects at different seating locations in a real open-plan office. To apply paired comparisons for effective learning, subjects compared successive pairs of different visual conditions and indicated their visual preferences through online surveys. Meanwhile, ten small, low-cost, and calibrated cameras captured luminance maps from both the field of view (FOV) of each occupant and non-intrusive viewpoints (on computer monitors, luminaire/ceiling and desk) under various sky conditions and interior luminance distributions. Convolutional Neural Network (CNN) models were developed and trained on luminance similarity index maps (generated from pixel-wise comparisons between successive luminance maps captured from FOV and non-intrusive cameras separately), to classify each subject’s daylight visual preferences. The results showed that the models trained on luminance distributions measured by monitor-mounted and ceiling-mounted cameras produced preference predictions consistent with those derived from FOV cameras, and can reliably learn visual preferences (83-94% accuracy) in all cases except for locations furthest from the windows. Overall, this study is the first to demonstrate that daylight preferences can be learned non-invasively by employing the full potential of HDRI and deep learning techniques, marking a significant milestone toward practical, AI-assisted, human-centered daylighting operation.
more »
« less
This content will become publicly available on January 1, 2027
Non-invasive assessment of the visual environment using conditional generative adversarial networks
Luminance monitoring within the field of view (FOV) is required for assessing visual comfort and overall visual preferences, but it is practically challenging and intrusive. As a result, real-time, human-centered daylighting operation remains a challenge. This paper presents a novel deep-learning based framework method to demonstrate that meaningful features in the occupant’s visual field can be extracted without invasive measurements. It is the first proof of concept to show that it is feasible to monitor luminance distributions as perceived by people, using a non-intrusive camera integrated with deep learning neural networks. A Conditional Generative Adversarial Network (CGAN), pix2pix is used to transfer information from non-intrusive images to FOV images. Two datasets were collected in an open-plan office with compact, low-cost High Dynamic Range Image (HDRI) cameras installed at two alternate locations (a wall or a monitor), to separately train two pix2pix models with the same target FOV images. The results show that the generated FOV images closely resemble the measured FOV images in terms of pixelwise luminance errors, mean luminance, and structural similarity. The main errors are due to bright scenes, visible through windows, confined to a very limited number of pixels. Overall, this work establishes a basis for future studies to assess the effect of visual environment on human perception using non-intrusive measurements. It also provides the theoretical foundation for a connected paper (Lu et al., 2025), which demonstrates that non-intrusive measurements and deep learning techniques can be used to discover daylight preferences and enable AI-assisted daylighting operation.
more »
« less
- Award ID(s):
- 2434194
- PAR ID:
- 10652349
- Publisher / Repository:
- Elsevier
- Date Published:
- Journal Name:
- Building and Environment
- Volume:
- 287
- Issue:
- PA
- ISSN:
- 0360-1323
- Page Range / eLocation ID:
- 113798
- Subject(s) / Keyword(s):
- Daylight preferences Deep learning CGAN Pix2Pix Image sensing Non-intrusive monitoring
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Battery life is an increasingly urgent challenge for today's untethered VR and AR devices. However, the power efficiency of head-mounted displays is naturally at odds with growing computational requirements driven by better resolution, refresh rate, and dynamic ranges, all of which reduce the sustained usage time of untethered AR/VR devices. For instance, the Oculus Quest 2, under a fully-charged battery, can sustain only 2 to 3 hours of operation time. Prior display power reduction techniques mostly target smartphone displays. Directly applying smartphone display power reduction techniques, however, degrades the visual perception in AR/VR with noticeable artifacts. For instance, the "power-saving mode" on smartphones uniformly lowers the pixel luminance across the display and, as a result, presents an overall darkened visual perception to users if directly applied to VR content. Our key insight is that VR display power reduction must be cognizant of the gaze-contingent nature of high field-of-view VR displays. To that end, we present a gaze-contingent system that, without degrading luminance, minimizes the display power consumption while preserving high visual fidelity when users actively view immersive video sequences. This is enabled by constructing 1) a gaze-contingent color discrimination model through psychophysical studies, and 2) a display power model (with respect to pixel color) through real-device measurements. Critically, due to the careful design decisions made in constructing the two models, our algorithm is cast as a constrained optimization problem with a closed-form solution, which can be implemented as a real-time, image-space shader. We evaluate our system using a series of psychophysical studies and large-scale analyses on natural images. Experiment results show that our system reduces the display power by as much as 24% (14% on average) with little to no perceptual fidelity degradation.more » « less
-
Visual discomfort is related to the statistical regularity of visual images. The contribution of luminance contrast to visual discomfort is well understood and can be framed in terms of a theory of efficient coding of natural stimuli, and linked to metabolic demand. While color is important in our interaction with nature, the effect of color on visual discomfort has received less attention. In this study, we build on the established association between visual discomfort and differences in chromaticity across space. We average the local differences in chromaticity in an image and show that this average is a good predictor of visual discomfort from the image. It accounts for part of the variance left unexplained by variations in luminance. We show that the local chromaticity difference in uncomfortable stimuli is high compared to that typical in natural scenes, except in particular infrequent conditions such as the arrangement of colorful fruits against foliage. Overall, our study discloses a new link between visual ecology and discomfort whereby discomfort arises when adaptive perceptual mechanisms are overstimulated by specific classes of stimuli rarely found in nature.more » « less
-
Fitch, T.; Lamm, C.; Leder, H.; Teßmar-Raible, K. (Ed.)Is analogical reasoning a task that must be learned to solve from scratch by applying deep learning models to massive numbers of reasoning problems? Or are analogies solved by computing similarities between structured representations of analogs? We address this question by comparing human performance on visual analogies created using images of familiar three-dimensional objects (cars and their subregions) with the performance of alternative computational models. Human reasoners achieved above-chance accuracy for all problem types, but made more errors in several conditions (e.g., when relevant subregions were occluded). We compared human performance to that of two recent deep learning models (Siamese Network and Relation Network) directly trained to solve these analogy problems, as well as to that of a compositional model that assesses relational similarity between part-based representations. The compositional model based on part representations, but not the deep learning models, generated qualitative performance similar to that of human reasoners.more » « less
-
Realistic wind data are essential in developing, testing, and ensuring the safety of unmanned aerial systems in operation. Alternatives to Dryden and von Kármán turbulence models are required, aimed explicitly at urban air spaces to generate turbulent wind data. We present a novel method to generate realistic wind data for the safe operation of small unmanned aerial vehicles in urban spaces. We propose a non-intrusive reduced order modeling approach to replicate realistic wind data and predict wind fields. The method uses a well-established large-eddy simulation model, the parallelized large eddy simulation model, to generate high-fidelity data. To create a reduced-order model, we utilize proper orthogonal decomposition to extract modes from the three-dimensional space and use specialized recurrent neural networks and long-term short memory for stepping in time. This paper combines the traditional approach of using computational fluid dynamic simulations to generate wind data with deep learning and reduced-order modeling techniques to devise a methodology for a non-intrusive data-based model for wind field prediction. A simplistic model of an isolated urban subspace with a single building setup in neutral atmospheric conditions is considered a test case for the demonstration of the method.more » « less
