skip to main content


This content will become publicly available on October 13, 2024

Title: A Testbed for Exploring Virtual Reality User Interfaces for Assigning Tasks to Agents at Multiple Sites
In virtual reality (VR) teleoperation and remote task guidance, a remote user may need to assign tasks to local technicians or robots at multiple sites. We are interested in scenarios where the user works with one site at a time, but must maintain awareness of the other sites for future intervention. We present an instrumented VR testbed for exploring how different spatial layouts of site representations impact user performance. In addition, we investigate ways of supporting the remote user in handling errors and interruptions from sites other than the one with which they are currently working, and switching between sites. We conducted a pilot study and explored how these factors affect user performance.  more » « less
Award ID(s):
2037101
NSF-PAR ID:
10482128
Author(s) / Creator(s):
; ; ;
Publisher / Repository:
ACM
Date Published:
Journal Name:
SUI '23: Proceedings of the 2023 ACM Symposium on Spatial User Interaction
Page Range / eLocation ID:
1 to 2
Format(s):
Medium: X
Location:
Sydney NSW Australia
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    The National Ecological Observatory Network Terrestrial Observation System (NEON TOS) produces open‐access data products that allow data users to investigate the impact of change drivers on key “sentinel” taxa and soils. The spatial and temporal sampling strategy that coordinates implementation of these protocols enables integration across TOS products and with products generated by NEON aquatic, remote sensing, and terrestrial instrument subsystems. Here, we illustrate the plots and sampling units that make up the physical foundation of a NEON TOS site, and we describe the scales (subplot, plot, airshed, and site) at which sampling is spatially colocated across protocols and subsystems. We also describe how moderate resolution imaging spectroradiometer‐enhanced vegetation index (MODIS‐EVI) phenology data are used to temporally coordinate TOS sampling within and across years at the continental scale of the observatory. Individually, TOS protocols produce data products that provide insight into populations, communities, and ecosystem processes. Within the spatial and temporal framework that guides cross‐protocol implementation, the ability to draw inference across data products is enhanced. To illustrate this point, we develop an example using R software that links two TOS data products collected with different temporal frequencies at both plot and site spatial scales. A thorough understanding of how TOS protocols are integrated with each other in space and time, and with other NEON subsystems, is necessary to leverage NEON data products to maximum effect. For example, a researcher must understand the spatial and temporal scales at which soil biogeochemistry data, soil microbe biomass data, and plant litter production and chemistry data may be combined to quantify soil nutrient stocks and fluxes across NEON sites. We present clear links among TOS protocols and across NEON subsystems that will enhance the utility of NEON TOS data products for the data user community.

     
    more » « less
  2. Many proteins exhibit a property called ‘allostery’. In allostery, an input signal at a specific site of a protein – such as a molecule binding, or the protein absorbing a photon of light – leads to a change in output at another site far away. For example, the protein might catalyze a chemical reaction faster or bind to another molecule more tightly in the presence of the input signal. This protein ‘remote control’ allows cells to sense and respond to changes in their environment. An ability to rapidly engineer new allosteric mechanisms into proteins is much sought after because this would provide an approach for building biosensors and other useful tools. One common approach to engineering new allosteric regulation is to combine a ‘sensor’ or input region from one protein with an ‘output’ region or domain from another. When researchers engineer allostery using this approach of combining input and output domains from different proteins, the difference in the output when the input is ‘on’ versus ‘off’ is often small, a situation called ‘modest allostery’. McCormick et al. wanted to know how to optimize this domain combination approach to increase the difference in output between the ‘on’ and ‘off’ states. More specifically, McCormick et al. wanted to find out whether swapping out or mutating specific amino acids (each of the individual building blocks that make up a protein) enhances or disrupts allostery. They also wanted to know if there are many possible mutations that change the effectiveness of allostery, or if this property is controlled by just a few amino acids. Finally, McCormick et al. questioned where in a protein most of these allostery-tuning mutations were located. To answer these questions, McCormick et al. engineered a new allosteric protein by inserting a light-sensing domain (input) into a protein involved in metabolism (a metabolic enzyme that produces a biomolecule called a tetrahydrofolate) to yield a light-controlled enzyme. Next, they introduced mutations into both the ‘input’ and ‘output’ domains to see where they had a greater effect on allostery. After filtering out mutations that destroyed the function of the output domain, McCormick et al. found that only about 5% of mutations to the ‘output’ domain altered the allosteric response of their engineered enzyme. In fact, most mutations that disrupted allostery were found near the site where the ‘input’ domain was inserted, while mutations that enhanced allostery were sprinkled throughout the enzyme, often on its protein surface. This was surprising in light of the commonly-held assumption that mutations on protein surfaces have little impact on the activity of the ‘output’ domain. Overall, the effect of individual mutations on allostery was small, but McCormick et al. found that these mutations can sometimes be combined to yield larger effects. McCormick et al.’s results suggest a new approach for optimizing engineered allosteric proteins: by introducing mutations on the protein surface. It also opens up new questions: mechanically, how do surface sites affect allostery? In the future, it will be important to characterize how combinations of mutations can optimize allosteric regulation, and to determine what evolutionary trajectories to high performance allosteric ‘switches’ look like. 
    more » « less
  3. In-person human interaction relies on our spatial perception of each other and our surroundings. Current remote communication tools partially address each of these aspects. Video calls convey real user representations but without spatial interactions. Augmented and Virtual Reality (AR/VR) experiences are immersive and spatial but often use virtual environments and characters instead of real-life representations. Bridging these gaps, we introduce DualStream, a system for synchronous mobile AR remote communication that captures, streams, and displays spatial representations of users and their surroundings. DualStream supports transitions between user and environment representations with different levels of visuospatial fidelity, as well as the creation of persistent shared spaces using environment snapshots. We demonstrate how DualStream can enable spatial communication in real-world contexts, and support the creation of blended spaces for collaboration. A formative evaluation of DualStream revealed that users valued the ability to interact spatially and move between representations, and could see DualStream fitting into their own remote communication practices in the near future. Drawing from these findings, we discuss new opportunities for designing more widely accessible spatial communication tools, centered around the mobile phone. 
    more » « less
  4. Abstract

    The NeonTreeCrowns dataset is a set of individual level crown estimates for 100 million trees at 37 geographic sites across the United States surveyed by the National Ecological Observation Network’s Airborne Observation Platform. Each rectangular bounding box crown prediction includes height, crown area, and spatial location. 

    How can I see the data?

    A web server to look through predictions is available through idtrees.org

    Dataset Organization

    The shapefiles.zip contains 11,000 shapefiles, each corresponding to a 1km^2 RGB tile from NEON (ID: DP3.30010.001). For example "2019_SOAP_4_302000_4100000_image.shp" are the predictions from "2019_SOAP_4_302000_4100000_image.tif" available from the NEON data portal: https://data.neonscience.org/data-products/explore?search=camera. NEON's file convention refers to the year of data collection (2019), the four letter site code (SOAP), the sampling event (4), and the utm coordinate of the top left corner (302000_4100000). For NEON site abbreviations and utm zones see https://www.neonscience.org/field-sites/field-sites-map. 

    The predictions are also available as a single csv for each file. All available tiles for that site and year are combined into one large site. These data are not projected, but contain the utm coordinates for each bounding box (left, bottom, right, top). For both file types the following fields are available:

    Height: The crown height measured in meters. Crown height is defined as the 99th quartile of all canopy height pixels from a LiDAR height model (ID: DP3.30015.001)

    Area: The crown area in m2 of the rectangular bounding box.

    Label: All data in this release are "Tree".

    Score: The confidence score from the DeepForest deep learning algorithm. The score ranges from 0 (low confidence) to 1 (high confidence)

    How were predictions made?

    The DeepForest algorithm is available as a python package: https://deepforest.readthedocs.io/. Predictions were overlaid on the LiDAR-derived canopy height model. Predictions with heights less than 3m were removed.

    How were predictions validated?

    Please see

    Weinstein, B. G., Marconi, S., Bohlman, S. A., Zare, A., & White, E. P. (2020). Cross-site learning in deep learning RGB tree crown detection. Ecological Informatics56, 101061.

    Weinstein, B., Marconi, S., Aubry-Kientz, M., Vincent, G., Senyondo, H., & White, E. (2020). DeepForest: A Python package for RGB deep learning tree crown delineation. bioRxiv.

    Weinstein, Ben G., et al. "Individual tree-crown detection in RGB imagery using semi-supervised deep learning neural networks." Remote Sensing 11.11 (2019): 1309.

    Were any sites removed?

    Several sites were removed due to poor NEON data quality. GRSM and PUUM both had lower quality RGB data that made them unsuitable for prediction. NEON surveys are updated annually and we expect future flights to correct these errors. We removed the GUIL puerto rico site due to its very steep topography and poor sunangle during data collection. The DeepForest algorithm responded poorly to predicting crowns in intensely shaded areas where there was very little sun penetration. We are happy to make these data are available upon request.

    # Contact

    We welcome questions, ideas and general inquiries. The data can be used for many applications and we look forward to hearing from you. Contact ben.weinstein@weecology.org. 

    Gordon and Betty Moore Foundation: GBMF4563 
    more » « less
  5. Construction project management requires frequent inspections to ensure the quality and progress of the construction work. Multiple stakeholders are involved in the inspection process during the project lifecycle. Some project stakeholders, such as architects, owners, structural engineers are involved with multiple construction projects at a time and are responsible to conduct timely inspection and monitoring tasks. This paper studies the potential of Virtual Reality (VR) and robotics for real-time remote inspection. The benefits and challenges of using VR for construction inspection and monitoring were identified and ranked through a systematic literature review. The top 5 benefits were found to be enhanced collaboration, realistic and immersive visualization, remote presence, reduction in inspection time, and support for decision-making. The top 5 challenges identified in this study include low- resolution displays, limited integration with existing technologies (such as BIM), causing disorientation and dizziness for the user, cost of adoption, and job site internet access limitations. Finally, a new approach was investigated for using VR to enable an immersive experience in remote inspection with an inspector assistant robot for real-time remote construction inspection. The experimental investigation verified the identified benefits and challenges. 
    more » « less