More Like this

1. Kernel Foveated Rendering

   https://doi.org/10.1145/3203199  

   Meng, Xiaoxu; Du, Ruofei; Zwicker, Matthias; Varshney, Amitabh (July 2018, Proceedings of the ACM on Computer Graphics and Interactive Techniques)
2. Path-Space Differentiable Rendering

   https://doi.org/10.1145/3386569.3392383  

   Zhang, Cheng; Miller, Bailey; Yan, Kan; Gkioulekas, Ioannis; Zhao, Shuang (July 2020, ACM transactions on graphics)

   Physics-based differentiable rendering, the estimation of derivatives of ra- diometric measures with respect to arbitrary scene parameters, has a diverse array of applications from solving analysis-by-synthesis problems to train- ing machine learning pipelines incorporating forward rendering processes. Unfortunately, general-purpose differentiable rendering remains challenging due to the lack of efficient estimators as well as the need to identify and handle complex discontinuities such as visibility boundaries. In this paper, we show how path integrals can be differentiated with respect to arbitrary differentiable changes of a scene. We provide a detailed theoretical analysis of this process and establish new differentiable rendering formulations based on the resulting differential path integrals. Our path- space differentiable rendering formulation allows the design of new Monte Carlo estimators that offer significantly better efficiency than state-of-the-art methods in handling complex geometric discontinuities and light transport phenomena such as caustics. 

   more » « less
3. Differentiable time-gated rendering

   https://doi.org/10.1145/3478513.3480489  

   Wu, Lifan; Cai, Guangyan; Ramamoorthi, Ravi; Zhao, Shuang (December 2021, ACM Transactions on Graphics)

   The continued advancements of time-of-flight imaging devices have enabled new imaging pipelines with numerous applications. Consequently, several forward rendering techniques capable of accurately and efficiently simulating these devices have been introduced. However, general-purpose differentiable rendering techniques that estimate derivatives of time-of-flight images are still lacking. In this paper, we introduce a new theory of differentiable time-gated rendering that enjoys the generality of differentiating with respect to arbitrary scene parameters. Our theory also allows the design of advanced Monte Carlo estimators capable of handling cameras with near-delta or discontinuous time gates. We validate our theory by comparing derivatives generated with our technique and finite differences. Further, we demonstrate the usefulness of our technique using a few proof-of-concept inverse-rendering examples that simulate several time-of-flight imaging scenarios. 

   more » « less
4. Characterizing the Effects of Haptic Rendering Parameter Variations on Perceived Kinesthetic Rendering Accuracy

   https://doi.org/10.1109/WHC49131.2021.9517158  

   Zhang, Bolun; Hagenow, Michael; Mutlu, Bilge; Gleicher, Michael; Zinn, Michael (July 2021, IEEE)

   To understand how the realism of a kinesthetic haptic rendering is affected by the accurate selection of the rendering model parameters, we conducted a preliminary user study where subjects compared three real-world objects to their equivalent haptic rendering. The subjects rated the rendering realism as the model parameters were varied about their nominal values. The results suggest that the required accuracy of various haptic rendering parameters is not equally important when considering the perceived realism. 

   more » « less
5. MetaSapiens: Real-Time Neural Rendering with Efficiency-Aware Pruning and Accelerated Foveated Rendering

   https://doi.org/10.1145/3669940.3707227  

   Lin, Weikai; Feng, Yu; Zhu, Yuhao (March 2025, ACM)

   Point-Based Neural Rendering (PBNR) is emerging as a promising class of rendering techniques, which are permeating all aspects of society, driven by a growing demand for real-time, photorealistic rendering in AR/VR and digital twins. Achieving real-time PBNR on mobile devices is challenging. This paper proposes MetaSapiens, a PBNR system that for the first time delivers real-time neural rendering on mobile devices while maintaining human visual quality. MetaSapiens combines three techniques. First, we present an efficiencyaware pruning technique to optimize rendering speed. Second, we introduce a Foveated Rendering (FR) method for PBNR, leveraging humans’ low visual acuity in peripheral regions to relax rendering quality and improve rendering speed. Finally, we propose an accelerator design for FR, addressing the load imbalance issue in (FR-based) PBNR. Our evaluation shows that our system achieves an order of magnitude speedup over existing PBNR models without sacrificing subjective visual quality, as confirmed by a user study. The code and demo are available at: https://horizonlab.org/metasapiens/. 

   more » « less