skip to main content


Title: Evaluating the Scalability of Non-Preferred Hand Mode Switching in Augmented Reality
Mode switching allows applications to support a wide range of operations (e.g. selection, manipulation, and navigation) using a limited input space. While the performance of different mode switching techniques has been extensively examined for pen- and touch-based interfaces, investigating mode switching in augmented reality (AR) is still relatively new. Prior work found that using non-preferred hand is an efficient mode switching technique in AR. However, it is unclear how the technique performs when increasing the number of modes, which is more indicative of real-world applications. Therefore, we examined the scalability of non-preferred hand mode switching in AR with two, four, six, and eight modes. We found that as the number of modes increase, performance plateaus after the four-mode condition. We also found that counting gestures have varying effects on mode switching performance in AR. Our findings suggest that modeling mode switching performance in AR is more complex than simply counting the number of available modes. Our work lays a foundation for understanding the costs associated with scaling interaction techniques in AR.  more » « less
Award ID(s):
1750840
NSF-PAR ID:
10175137
Author(s) / Creator(s):
; ; ; ;
Date Published:
Journal Name:
International Conference on Advanced Visual Interfaces
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Taking part in creating location-based augmented reality (LBAR) experiences that focus on communication, art and design could serve as an entry point for art-oriented girls and young women towards career pathways in computer science and information communication technology. This conceptual paper presents our theory-based approach and subsequent application, as well as lessons learned informed by team discussions and reflections. We built an LBAR program entitled AR Girls on four foundational principles: stealth science (embedding science in familiar appealing experiences), place-based education (situating learning in one’s own community), non-hierarchical design (collaborations where both adults and youth generate content), and learning through design (engaging in design, not just play). To translate these principles into practice, we centered the program around the theme of art by forming partnerships with small community art organizations and positioning LBAR as an art-based communication medium. We found that LBAR lends itself to an interdisciplinary approach that blends technology, art, science and communication. We believe our approach helped girls make connections to their existing interests and build soft skills such as leadership and interpersonal communication as they designed local environmentally-focused LBAR walking tours. Our “use-modify-create” approach provided first-hand experiences with the AR software early on, and thus supported the girls and their art educators in designing and showcasing their walking tours. Unfortunately, the four foundational principles introduced considerable complexity to AR Girls, which impacted recruitment and retention, and at times overwhelmed the art educators who co-led the program. To position AR Girls for long-term success, we simplified the program approach and implementation, including switching to a more user-friendly AR software; reducing logistical challenges of location-based design and play; narrowing the topic addressed by the girls design; and making the involvement of community partners optional. Overall, our initial work was instrumental in understanding how to translate theoretical considerations for learning in out-of-school settings into an LBAR program aimed at achieving multiple complementary outcomes for participating girls. Ultimately, we achieved better scalability by simplifying AR Girls both conceptually and practically. The lessons learned from AR Girls can inform others using LBAR for education and youth development programming. 
    more » « less
  2. In this work, we investigate the influence of different visualizations on a manipulation task in virtual reality (VR). Without the haptic feedback of the real world, grasping in VR might result in intersections with virtual objects. As people are highly sensitive when it comes to perceiving collisions, it might look more appealing to avoid intersections and visualize non-colliding hand motions. However, correcting the position of the hand or fingers results in a visual-proprioceptive discrepancy and must be used with caution. Furthermore, the lack of haptic feedback in the virtual world might result in slower actions as a user might not know exactly when a grasp has occurred. This reduced performance could be remediated with adequate visual feedback. In this study, we analyze the performance, level of ownership, and user preference of eight different visual feedback techniques for virtual grasping. Three techniques show the tracked hand (with or without grasping feedback), even if it intersects with the grasped object. Another three techniques display a hand without intersections with the object, called outer hand, simulating the look of a real world interaction. One visualization is a compromise between the two groups, showing both a primary outer hand and a secondary tracked hand. Finally, in the last visualization the hand disappears during the grasping activity. In an experiment, users perform a pick-and-place task for each feedback technique. We use high fidelity marker-based hand tracking to control the virtual hands in real time. We found that the tracked hand visualizations result in better performance, however, the outer hand visualizations were preferred. We also find indications that ownership is higher with the outer hand visualizations. 
    more » « less
  3. Obeid, I. ; Selesnik, I. ; Picone, J. (Ed.)
    The Neuronix high-performance computing cluster allows us to conduct extensive machine learning experiments on big data [1]. This heterogeneous cluster uses innovative scheduling technology, Slurm [2], that manages a network of CPUs and graphics processing units (GPUs). The GPU farm consists of a variety of processors ranging from low-end consumer grade devices such as the Nvidia GTX 970 to higher-end devices such as the GeForce RTX 2080. These GPUs are essential to our research since they allow extremely compute-intensive deep learning tasks to be executed on massive data resources such as the TUH EEG Corpus [2]. We use TensorFlow [3] as the core machine learning library for our deep learning systems, and routinely employ multiple GPUs to accelerate the training process. Reproducible results are essential to machine learning research. Reproducibility in this context means the ability to replicate an existing experiment – performance metrics such as error rates should be identical and floating-point calculations should match closely. Three examples of ways we typically expect an experiment to be replicable are: (1) The same job run on the same processor should produce the same results each time it is run. (2) A job run on a CPU and GPU should produce identical results. (3) A job should produce comparable results if the data is presented in a different order. System optimization requires an ability to directly compare error rates for algorithms evaluated under comparable operating conditions. However, it is a difficult task to exactly reproduce the results for large, complex deep learning systems that often require more than a trillion calculations per experiment [5]. This is a fairly well-known issue and one we will explore in this poster. Researchers must be able to replicate results on a specific data set to establish the integrity of an implementation. They can then use that implementation as a baseline for comparison purposes. A lack of reproducibility makes it very difficult to debug algorithms and validate changes to the system. Equally important, since many results in deep learning research are dependent on the order in which the system is exposed to the data, the specific processors used, and even the order in which those processors are accessed, it becomes a challenging problem to compare two algorithms since each system must be individually optimized for a specific data set or processor. This is extremely time-consuming for algorithm research in which a single run often taxes a computing environment to its limits. Well-known techniques such as cross-validation [5,6] can be used to mitigate these effects, but this is also computationally expensive. These issues are further compounded by the fact that most deep learning algorithms are susceptible to the way computational noise propagates through the system. GPUs are particularly notorious for this because, in a clustered environment, it becomes more difficult to control which processors are used at various points in time. Another equally frustrating issue is that upgrades to the deep learning package, such as the transition from TensorFlow v1.9 to v1.13, can also result in large fluctuations in error rates when re-running the same experiment. Since TensorFlow is constantly updating functions to support GPU use, maintaining an historical archive of experimental results that can be used to calibrate algorithm research is quite a challenge. This makes it very difficult to optimize the system or select the best configurations. The overall impact of all of these issues described above is significant as error rates can fluctuate by as much as 25% due to these types of computational issues. Cross-validation is one technique used to mitigate this, but that is expensive since you need to do multiple runs over the data, which further taxes a computing infrastructure already running at max capacity. GPUs are preferred when training a large network since these systems train at least two orders of magnitude faster than CPUs [7]. Large-scale experiments are simply not feasible without using GPUs. However, there is a tradeoff to gain this performance. Since all our GPUs use the NVIDIA CUDA® Deep Neural Network library (cuDNN) [8], a GPU-accelerated library of primitives for deep neural networks, it adds an element of randomness into the experiment. When a GPU is used to train a network in TensorFlow, it automatically searches for a cuDNN implementation. NVIDIA’s cuDNN implementation provides algorithms that increase the performance and help the model train quicker, but they are non-deterministic algorithms [9,10]. Since our networks have many complex layers, there is no easy way to avoid this randomness. Instead of comparing each epoch, we compare the average performance of the experiment because it gives us a hint of how our model is performing per experiment, and if the changes we make are efficient. In this poster, we will discuss a variety of issues related to reproducibility and introduce ways we mitigate these effects. For example, TensorFlow uses a random number generator (RNG) which is not seeded by default. TensorFlow determines the initialization point and how certain functions execute using the RNG. The solution for this is seeding all the necessary components before training the model. This forces TensorFlow to use the same initialization point and sets how certain layers work (e.g., dropout layers). However, seeding all the RNGs will not guarantee a controlled experiment. Other variables can affect the outcome of the experiment such as training using GPUs, allowing multi-threading on CPUs, using certain layers, etc. To mitigate our problems with reproducibility, we first make sure that the data is processed in the same order during training. Therefore, we save the data from the last experiment and to make sure the newer experiment follows the same order. If we allow the data to be shuffled, it can affect the performance due to how the model was exposed to the data. We also specify the float data type to be 32-bit since Python defaults to 64-bit. We try to avoid using 64-bit precision because the numbers produced by a GPU can vary significantly depending on the GPU architecture [11-13]. Controlling precision somewhat reduces differences due to computational noise even though technically it increases the amount of computational noise. We are currently developing more advanced techniques for preserving the efficiency of our training process while also maintaining the ability to reproduce models. In our poster presentation we will demonstrate these issues using some novel visualization tools, present several examples of the extent to which these issues influence research results on electroencephalography (EEG) and digital pathology experiments and introduce new ways to manage such computational issues. 
    more » « less
  4. null (Ed.)
    Acoustic devices have played a major role in telecommunications for decades as the leading technology for filtering in RF and microwave frequencies. While filter requirements for insertion loss and bandwidth become more stringent, more functionality is desired for many applications to improve overall system level performance. For instance, a filter with non-reciprocal transmission can minimize losses due to mismatch and protect the source from reflections while also performing its filtering duties. A device such as this one was originally researched by scientists decades ago. These devices were based on the acoustoelectric effect where surface acoustic waves (SAW) traveling in the same direction are as drift carriers in a nearby semiconductor are amplified. While several experiments were successfully demonstrated in [1], [2], [3]. these devices suffered from extremely high operating electric fields and noise figure [4], [5]. In the past few years, new techniques have been developed for implementing non-reciprocal devices such as isolators and circulators without utilizing magnetic materials [6], [7], [8], [9]. The most popular technique has been spatio-temporal modulation (STM) where commutated clock signals synchronized with delay elements result in non-reciprocal transmission through the network. STM has also been adapted by researchers to create non-reciprocal filters. The work in [10] utilizes 4 clocks signals to obtain a non-reciprocal filter with an insertion loss of -6.6 dB an isolation of 25.4 dB. Another filter demonstrated in [11] utilizes 6 synchronized clock signals to obtain a non-reciprocal filter with an insertion loss of -5.6 dB and an Isolation of 20 dB. In this work, a novel non-reciprocal topology is explored with the use of only one modulation signal. The design is based on asymmetrical SAW delay lines with a parametric amplifier. The device can operate in two different modes: phase coherent mode and phase incoherent mode. In phase coherent mode, the device is capable of over +12 dB of gain and 20.2 dB of isolation. A unique feature of this mode is that the phase of the pump signal can be utilized to tune the frequency response of the filter. Under the phase-incoherent mode, the pump frequency remains constant and the device behaves as a normal filter with non-reciprocal transmission exhibiting over +7 dB of gain and 17.33 dB of isolation. While the tuning capability is lost in this mode, phase-coherence is no longer necessary so the device can be utilized in most filtering applications. 
    more » « less
  5. Today's high-performance computing (HPC) applications are producing vast volumes of data, which are challenging to store and transfer efficiently during the execution, such that data compression is becoming a critical technique to mitigate the storage burden and data movement cost. Huffman coding is arguably the most efficient Entropy coding algorithm in information theory, such that it could be found as a fundamental step in many modern compression algorithms such as DEFLATE. On the other hand, today's HPC applications are more and more relying on the accelerators such as GPU on supercomputers, while Huffman encoding suffers from low throughput on GPUs, resulting in a significant bottleneck in the entire data processing. In this paper, we propose and implement an efficient Huffman encoding approach based on modern GPU architectures, which addresses two key challenges: (1) how to parallelize the entire Huffman encoding algorithm, including codebook construction, and (2) how to fully utilize the high memory-bandwidth feature of modern GPU architectures. The detailed contribution is four-fold. (1) We develop an efficient parallel codebook construction on GPUs that scales effectively with the number of input symbols. (2) We propose a novel reduction based encoding scheme that can efficiently merge the codewords on GPUs. (3) We optimize the overall GPU performance by leveraging the state-of-the-art CUDA APIs such as Cooperative Groups. (4) We evaluate our Huffman encoder thoroughly using six real-world application datasets on two advanced GPUs and compare with our implemented multi-threaded Huffman encoder. Experiments show that our solution can improve the encoding throughput by up to 5.0x and 6.8x on NVIDIA RTX 5000 and V100, respectively, over the state-of-the-art GPU Huffman encoder, and by up to 3.3x over the multi-thread encoder on two 28-core Xeon Platinum 8280 CPUs. 
    more » « less