An official website of the United States government
Observing users interacting with a learning game, often referred to as playtesting, is a critical component of usability testing. Unfortunately, this practice is expensive, requiring users and researchers to be in the same place at the same time as the participants. With Virtual Reality, this difficulty is amplified due to the experience being hidden from researcher view by default, and the extra complexity of setting up casting to an external monitor, especially with groups. In this paper we develop and utilize a replay-based approach to usability testing that relieves these concerns. The approach uses a low-bandwidth stream of telemetry signals that are generated by the original play session. These signals are then reconstructed into a full representation of the original experience at a different time or place and leveraged to identify usability issues. Once the issues have been discovered, automated processes are developed to computationally identify their presence and severity in arbitrarily large public audiences. This work contributes a demonstrated use of replay for Virtual Reality usability research, and a novel use of replay combined with educational data mining to develop an automated process for studying large audiences at low cost.
more »
« less
A Walk Through Spectral Bands: Using Virtual Reality to Better Visualize Hyperspectral Data
One of the basic challenges of understanding hyperspectral data arises from the fact that it is intrinsically 3-dimensional. A diverse range of algorithms have been developed to help visualize hyperspectral data trichromatically in 2-dimensions. In this paper we take a different approach and show how virtual reality provides a way of visualizing a hyperspectral data cube without collapsing the spectral dimension. Using several different real datasets, we show that it is straightforward to find signals of interest and make them more visible by exploiting the immersive, interactive environment of virtual reality. This enables signals to be seen which would be hard to detect if we were simply examining hyperspectral data band by band.
more »
« less
- Award ID(s):
- 1633830
- PAR ID:
- 10099071
- Date Published:
- Journal Name:
- Advances in intelligent systems and computing
- Volume:
- 976
- ISSN:
- 2194-5357
- Page Range / eLocation ID:
- 160-165
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
Stephanidis C., Antona M. (Ed.)The objective of this study is to develop and use a virtual reality game as a tool to assess the effects of realistic stress on the behavioral and physiological responses of participants. The game is based on a popular Steam game called Keep Talking Nobody Explodes, where the players collaborate to defuse a bomb. Varying levels of difficulties in solving a puzzle and time pressures will result in different stress levels that can be measured in terms of errors, response times, and other physiological measurements. The game was developed using 3D programming tools including Blender and a virtual reality development kit (VRTK). To measure response times accurately, we added LSL (Lab Stream Layer) Markers to collect and synchronize physiological signals, behavioral data, and the timing of game events. We recorded Electrocardiogram (ECG) data during gameplay to assess heart rate and heart-rate variability (HRV) that have been shown as reliable indicators of stress. Our empirical results showed that heart rate increased significantly while HRV reduced significantly when the participants under high stress, which are consistent with the prior mainstream stress research. This VR game framework is publicly available in GitHub and allows researchers to measure and synchronize other physiological signals such as electroencephalogram, electromyogram, and pupillometry.more » « less
-
Habituation to signals that warn of a potential danger in high-risk work environments is a critical causal factor of workplace accidents. Such habituation is hard to measure in a real-world setting, and no existing intervention can effectively curb it. Here, we present a protocol to enhance workers’ sensory responses to frequently encountered warnings at workplaces using a virtual-reality-based behavioral intervention. We describe steps for performing a virtual reality experiment and an electroencephalography (EEG) experiment with human participants.more » « less
-
Ca2+ and cAMP are ubiquitous second messengers known to differentially regulate a variety of cellular functions over a wide range of timescales. Studies from a variety of groups support the hypothesis that these signals can be localized to discrete locations within cells, and that this subcellular localization is a critical component of signaling specificity. However, to date, it has been difficult to track second messenger signals at multiple locations within a single cell. This difficulty is largely due to the inability to measure multiplexed florescence signals in real time. To overcome this limitation, we have utilized both emission scan- and excitation scan-based hyperspectral imaging approaches to track second messenger signals as well as labeled cellular structures and/or proteins in the same cell. We have previously reported that hyperspectral imaging techniques improve the signal-to-noise ratios of both fluorescence and FRET measurements, and are thus well suited for the measurement of localized second messenger signals. Using these approaches, we have measured near plasma membrane and near nuclear membrane cAMP signals, as well as distributed signals within the cytosol, in several cell types including airway smooth muscle, pulmonary endothelial, and HEK-293 cells. We have also measured cAMP and Ca2+ signals near autofluorescent structures that appear to be golgi. Our data demonstrate that hyperspectral imaging approaches provide unique insight into the spatial and kinetic distributions of cAMP and Ca2+ signals in single cells.more » « less
-
Hyperspectral sensors acquire spectral responses from objects with a large number of narrow spectral bands. The large volume of data may be costly in terms of storage and computational requirements. In addition, hyperspectral data are often information-wise redundant. Band selection intends to overcome these limitations by selecting a small subset of spectral bands that provide more information or better performance for particular tasks. However, existing band selection techniques do not directly maximize the task-specific performance, but rather utilize hand-crafted metrics as a proxy to the final goal of performance improvement. In this paper, we propose a deep learning (DL) architecture composed of a constrained measurement learning network for band selection, followed by a classification network. The proposed joint DL architecture is trained in a data-driven manner to optimize the classification loss along band selection. In this way, the proposed network directly learns to select bands that enhance the classification performance. Our evaluation results with Indian Pines (IP) and the University of Pavia (UP) datasets show that the proposed constrained measurement learning-based band selection approach provides higher classification accuracy compared to the state-of-the-art supervised band selection methods for the same number of bands selected. The proposed method shows 89.08% and 97.78% overall accuracy scores for IP and UP respectively, being 1.34% and 2.19% higher than the second-best method.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- The DOI is not currently available.
