Search for: All records

Placing and orienting a camera to compose aesthetically meaningful shots of a scene is not only a key objective in real-world photography and cinematography but also for virtual content creation. The framing of a camera often significantly contributes to the story telling in movies, games, and mixed reality applications. Generating single camera poses or even contiguous trajectories either requires a significant amount of manual labor or requires solving highdimensional optimization problems, which can be computationally demanding and error-prone. In this paper, we introduce GAIT, a Deep Reinforcement Learning (DRL) agent, that learns to automatically control a camera to generate a sequence of aesthetically meaningful views for synthetic 3D indoor scenes. To generate sequences of frames with high aesthetic value, GAIT relies on a neural aesthetics estimator, which is trained on a crowed-sourced dataset. Additionally, we introduce regularization techniques for diversity and smoothness to generate visually interesting trajectories for a 3D environment, and to constrain agent acceleration in the reward function to generate a smooth sequence of camera frames. We validated our method by comparing it to baseline algorithms, based on a perceptual user study, and through ablation studies. The source code of our method will be released with the final version of our paper. 

Abstract Many factors shape public perceptions of extreme weather risk; understanding these factors is important to encourage preparedness. This article describes a novel workshop designed to encourage individual and community decision-making about predicted storm surge flooding. Over 160 U.S. college students participated in this 4-h experience. Distinctive features included 1) two kinds of visualizations, standard weather forecasting graphics versus 3D computer graphics visualization; 2) narrative about a fictitious storm, role-play, and guided discussion of participants’ concerns; and 3) use of an “ethical matrix,” a collective decision-making tool that elicits diverse perspectives based on the lived experiences of diverse stakeholders. Participants experienced a narrative about a hurricane with potential for devastating storm surge flooding on a fictitious coastal college campus. They answered survey questions before, at key points during, and after the narrative, interspersed with forecasts leading to predicted storm landfall. During facilitated breakout groups, participants role-played characters and filled out an ethical matrix. Discussing the matrix encouraged consideration of circumstances impacting evacuation decisions. Participants’ comments suggest several components may have influenced perceptions of personal risk, risks to others, the importance of monitoring weather, and preparing for emergencies. Surprisingly, no differences between the standard forecast graphics versus the immersive, hyperlocal visualizations were detected. Overall, participants’ comments indicate the workshop increased appreciation of others’ evacuation and preparation challenges. 

Augmented Reality (AR) is widely considered the next evolution in personal devices, enabling seamless integration of the digital world into our reality. Such integration, however, often requires unfettered access to sensor data, causing significant over privilege for applications that run on these platforms. Through analysis of 17 AR systems and 45 popular AR applications, we explore existing mechanisms for access control in AR platforms, identify key trends in how AR applications use sensor data, and pinpoint unique threats users face in AR environments. Using these findings, we design and implement Erebus, an access control framework for AR platforms that enables fine-grained control over data used by AR applications. Erebus achieves the principle of least privileged through the creation of a domain-specific language (DSL) for permission control in AR platforms, allowing applications to specify data needed for their functionality. Using this DSL, Erebus further enables users to customize app permissions to apply under specific user conditions. We implement Erebus on Google’s ARCore SDK and port five existing AR applications to demonstrate the capability of Erebus to secure various classes of apps. Performance results using these applications and various microbenchmarks show that Erebus achieves its security goals while being practical, introducing negligible performance overhead to the AR system. 

Spatial information understanding is fundamental to visual perception in Metaverse. Beyond the stereoscopic visual cues naturally carried in Metaverse, the human vision system may use other auxiliary information provided by any shadow casting or motion parallax available to perceive the 3D virtual world. However, the combined use of shadows and motion parallax to improve 3D perception have not been fully studied. In particular, when visualizing the combination of volumetric data and associated skeleton models in VR, how to provide the auxiliary visual cues to enhance observers’ perception of the structural information is a key yet underexplored topic. This problem is particularly challenging for visualization of data in biomedical research. In this paper, we focus on immersive analytics in neurobiology where the structural information includes the relative position of objects (nuclei / cell body) in the 3D space and the spatial measurement and connectivity of segments (axons and dendrites) in a model. We present a perceptual experiment designed for understanding the consequence of shadow casting and motion parallax in the neuron structures observation and the feedback and analysis of the experiment are reported and discussed.