More Like this

1. Neural 3D holography: learning accurate wave propagation models for 3D holographic virtual and augmented reality displays

   https://doi.org/10.1145/3478513.3480542  

   Choi, Suyeon; Gopakumar, Manu; Peng, Yifan; Kim, Jonghyun; Wetzstein, Gordon (December 2021, ACM Transactions on Graphics)

   Holographic near-eye displays promise unprecedented capabilities for virtual and augmented reality (VR/AR) systems. The image quality achieved by current holographic displays, however, is limited by the wave propagation models used to simulate the physical optics. We propose a neural network-parameterized plane-to-multiplane wave propagation model that closes the gap between physics and simulation. Our model is automatically trained using camera feedback and it outperforms related techniques in 2D plane-to-plane settings by a large margin. Moreover, it is the first network-parameterized model to naturally extend to 3D settings, enabling high-quality 3D computer-generated holography using a novel phase regularization strategy of the complex-valued wave field. The efficacy of our approach is demonstrated through extensive experimental evaluation with both VR and optical see-through AR display prototypes. 

   more » « less
2. Realizing Colorful Holographic Mimicry by Metasurfaces

   https://doi.org/10.1002/adma.202005864  

   Xiong, Bo; Xu, Yihao; Wang, Jianan; Li, Lin; Deng, Lin; Cheng, Feng; Peng, Ru‐Wen; Wang, Mu; Liu, Yongmin (April 2021, Advanced Materials)

   Abstract Mimicry is a biological camouflage phenomenon whereby an organism can change its shape and color to resemble another object. Herein, the idea of biological mimicry and rich degrees of freedom in metasurface designs are combined to realize holographic mimicry devices. A general mathematical method, called phase matrix transformation, to accomplish the holographic mimicry process is proposed. Based on this method, a dynamic metasurface hologram is designed, which shows an image of a “bird” in the air, and a distinct image of a “fish” when the environment is changed to oil. Furthermore, to make the mimicry behavior more generic, holographic mimicry operating at dual wavelengths is also designed and experimentally demonstrated. Moreover, the fully independent phase modulation realized by phase matrix transformation makes the working efficiency of the device relatively higher than the conventional multiwavelength holographic devices with off‐axis illumination or interleaved subarrays. The work potentially opens a new research paradigm interfacing bionics with nanophotonics, which may produce novel applications for optical information encryption, virtual/augmented reality (VR/AR), and military camouflage systems. 

   more » « less
3. Color-Perception-Guided Display Power Reduction for Virtual Reality

   https://doi.org/10.1145/3550454.3555473  

   Duinkharjav, Budmonde; Chen, Kenneth; Tyagi, Abhishek; He, Jiayi; Zhu, Yuhao; Sun, Qi (December 2022, ACM Transactions on Graphics)

   Battery life is an increasingly urgent challenge for today's untethered VR and AR devices. However, the power efficiency of head-mounted displays is naturally at odds with growing computational requirements driven by better resolution, refresh rate, and dynamic ranges, all of which reduce the sustained usage time of untethered AR/VR devices. For instance, the Oculus Quest 2, under a fully-charged battery, can sustain only 2 to 3 hours of operation time. Prior display power reduction techniques mostly target smartphone displays. Directly applying smartphone display power reduction techniques, however, degrades the visual perception in AR/VR with noticeable artifacts. For instance, the "power-saving mode" on smartphones uniformly lowers the pixel luminance across the display and, as a result, presents an overall darkened visual perception to users if directly applied to VR content. Our key insight is that VR display power reduction must be cognizant of the gaze-contingent nature of high field-of-view VR displays. To that end, we present a gaze-contingent system that, without degrading luminance, minimizes the display power consumption while preserving high visual fidelity when users actively view immersive video sequences. This is enabled by constructing 1) a gaze-contingent color discrimination model through psychophysical studies, and 2) a display power model (with respect to pixel color) through real-device measurements. Critically, due to the careful design decisions made in constructing the two models, our algorithm is cast as a constrained optimization problem with a closed-form solution, which can be implemented as a real-time, image-space shader. We evaluate our system using a series of psychophysical studies and large-scale analyses on natural images. Experiment results show that our system reduces the display power by as much as 24% (14% on average) with little to no perceptual fidelity degradation. 

   more » « less
4. Hogel-Free Holography

   https://doi.org/10.1145/3516428  

   Chakravarthula, Praneeth; Tseng, Ethan; Fuchs, Henry; Heide, Felix (October 2022, ACM Transactions on Graphics)

   Holography is a promising avenue for high-quality displays without requiring bulky, complex optical systems. While recent work has demonstrated accurate hologram generation of 2D scenes, high-quality holographic projections of 3D scenes has been out of reach until now. Existing multiplane 3D holography approaches fail to model wavefronts in the presence of partial occlusion while holographic stereogram methods have to make a fundamental tradeoff between spatial and angular resolution. In addition, existing 3D holographic display methods rely on heuristic encoding of complex amplitude into phase-only pixels which results in holograms with severe artifacts. Fundamental limitations of the input representation, wavefront modeling, and optimization methods prohibit artifact-free 3D holographic projections in today’s displays. To lift these limitations, we introduce hogel-free holography which optimizes for true 3D holograms, supporting both depth- and view-dependent effects for the first time. Our approach overcomes the fundamental spatio-angular resolution tradeoff typical to stereogram approaches. Moreover, it avoids heuristic encoding schemes to achieve high image fidelity over a 3D volume. We validate that the proposed method achieves 10 dB PSNR improvement on simulated holographic reconstructions. We also validate our approach on an experimental prototype with accurate parallax and depth focus effects. 

   more » « less
5. Towards Eyeglass-style Holographic Near-eye Displays with Static Expanded Eyebox

   https://doi.org/10.1109/ISMAR50242.2020.00057  

   Xia, Xinxing; Guan, Yunqing; State, Andrei; Chakravarthula, Praneeth; Cham, Tat-Jen; Fuchs, Henry (November 2020, 2020 IEEE International Symposium on Mixed and Augmented Reality (ISMAR))

   null (Ed.)

   Holography is perhaps the only method demonstrated so far that can achieve a wide field of view (FOV) and a compact eyeglass-style form factor for augmented reality (AR) near-eye displays (NEDs). Unfortunately, the eyebox of such NEDs is impractically small ($$\sim \lt$$ 1 mm). In this paper, we introduce and demonstrate a design for holographic NEDs with a practical, wide eyebox of $$\sim$$ 10 mm and without any moving parts, based on holographic lenslets. In our design, a holographic optical element (HOE) based on a lenslet array was fabricated as the image combiner with expanded eyebox. A phase spatial light modulator (SLM) alters the phase of the incident laser light projected onto the HOE combiner such that the virtual image can be perceived at different focus distances, which can reduce the vergence-accommodation conflict (VAC). We have successfully implemented a bench-top prototype following the proposed design. The experimental results show effective eyebox expansion to a size of $$\sim$$ 10 mm. With further work, we hope that these design concepts can be incorporated into eyeglass-size NEDs. 

   more » « less