skip to main content

Title: Augmented Reality: Telehealth Demonstration Application
Augmented Reality (AR) as a technology will improve the way we work and live in the future. The Microsoft HoloLens device allows for rendering of interactive virtual components into a real world space. The HoloLens is an augmented reality headset and can display these virtual components in front of the user’s eyes, so the data needed to complete a real-world task will always be available. The nature of a HoloLens device lends itself useful for applications in a healthcare setting. Potential benefits come from transitioning to a more hands-free environment such as allowing the logging of data while in sterile environments without needing to sterilize repeatedly from touching paper or tablet. This project developed an augmented reality (AR) application to include a care plan tracker established by a patient’s doctor to allow the patient to do daily tasks without a health care worker’s supervision. The application displays the medications that the patient needs to ingest, daily tasks to complete, and health data to record. The application allows the physician to retrieve useful patient information regularly without scheduled physicals. This project sets a baseline that will provide future developers with documentation, research, and this sample application to assist in the design and construction of more complex applications in the future at the University of New Hampshire.
Authors:
; ; ;
Award ID(s):
1659377
Publication Date:
NSF-PAR ID:
10225096
Journal Name:
Practice and Experience in Advanced Research Computing (PEARC ’20)
Page Range or eLocation-ID:
452 to 455
Sponsoring Org:
National Science Foundation
More Like this
  1. In a seminal article on augmented reality (AR) [7], Ron Azuma defines AR as a variation of virtual reality (VR), which completely immerses a user inside a synthetic environment. Azuma says “In contrast, AR allows the user to see the real world, with virtual objects superimposed upon or composited with the real world” [7] (emphasis added). Typically, a user wears a tracked stereoscopic head-mounted display (HMD) or holds a smartphone, showing the real world through optical or video means, with superimposed graphics that provide the appearance of virtual content that is related to and registered with the real world. Whilemore »AR has been around since the 1960s [72], it is experiencing a renaissance of development and consumer interest. With exciting products from Microsoft (HoloLens), Metavision (Meta 2), and others; Apple’s AR Developer’s Kit (ARKit); and well-funded startups like Magic Leap [54], the future is looking even brighter, expecting that AR technologies will be absorbed into our daily lives and have a strong influence on our society in the foreseeable future.« less
  2. Smart bracelets able to interpret the wearer's emotional state and communicate it to a remote decision-support facility will have broad applications in healthcare, elder care, the military, and other fields. While there are existing commercial embedded devices, such as the Apple Watch, that have health-monitoring sensors, such devices cannot sufficiently support a real-time health-monitoring system with battery-efficient remote data delivery. Ongoing R&D is developing solutions capable of monitoring multiple psycho-physiological signals. Possible hardware configurations include wrist-worn devices and sensors across an augmented reality headset (e.g., HoloLens 2). The device should carry an array of sensors of psycho-physiological signals, including amore »galvanic skin response sensor, motion sensor, skin temperature sensor, and a heart rate sensor. Output from these sensors can be intelligently fused to monitor the affective state and to determine specific trigger events for the wearer. To enable real-time remote monitoring applications, the device needs to be low-power to allow persistent monitoring while prolonging usage before recharging. For many applications, specialized sensor arrays are required, e.g. a galvanic skin response sensor. An application-flexible device would allow adding/removing sensors and would provide a choice of communication modules (e.g., Bluetooth 5.0 low-energy vs ZigBee). Appropriate configurations of the device would support applications in military health monitoring, drug-addiction mitigation, autistic trigger monitoring, and augmented reality exploration. A configuration example is: motion sensors (3-axis accelerometers, gyroscopes, and magnetometers to track steps, falls, and energy usage), a heart-rate sensor (e.g., an optical-based heart rate sensor with a single monitoring zone using the process of photoplethysmography (PPS)), at least a Bluetooth 5.0 (but a different communication device may be needed depending on the use case), and flash memory to temporarily store data when the device is not remotely communicating. The wearables field has greatly advanced in the quality of sensors; the fusion of multi-sensor data is the current frontier.« less
  3. Augmented reality (AR) technologies, such as Microsoft’s HoloLens head-mounted display and AR-enabled car windshields, are rapidly emerging. AR applications provide users with immersive virtual experiences by capturing input from a user’s surroundings and overlaying virtual output on the user’s perception of the real world. These applications enable users to interact with and perceive virtual content in fundamentally new ways. However, the immersive nature of AR applications raises serious security and privacy concerns. Prior work has focused primarily on input privacy risks stemming from applications with unrestricted access to sensor data. However, the risks associated with malicious or buggy AR outputmore »remain largely unexplored. For example, an AR windshield application could intentionally or accidentally obscure oncoming vehicles or safety-critical output of other AR applications. In this work, we address the fundamental challenge of securing AR output in the face of malicious or buggy applications. We design, prototype, and evaluate Arya, an AR platform that controls application output according to policies specified in a constrained yet expressive policy framework. In doing so, we identify and overcome numerous challenges in securing AR output.« less
  4. Mobile Augmented Reality (AR) provides immersive experiences by aligning virtual content (holograms) with a view of the real world. When a user places a hologram it is usually expected that like a real object, it remains in the same place. However, positional errors frequently occur due to inaccurate environment mapping and device localization, to a large extent determined by the properties of natural visual features in the scene. In this demonstration we present SceneIt, the first visual environment rating system for mobile AR based on predictions of hologram positional error magnitude. SceneIt allows users to determine if virtual content placedmore »in their environment will drift noticeably out of position, without requiring them to place that content. It shows that the severity of positional error for a given visual environment is predictable, and that this prediction can be calculated with sufficiently high accuracy and low latency to be useful in mobile AR applications.« less
  5. Augmented reality (AR), which overlays virtual content on top of the user’s perception of the real world, has now begun to enter the consumer market. Besides smartphone platforms, early-stage head-mounted displays such as the Microsoft HoloLens are under active development. Many compelling uses of these technologies are multi-user: e.g., inperson collaborative tools, multiplayer gaming, and telepresence. While prior work on AR security and privacy has studied potential risks from AR applications, new risks will also arise among multiple human users. In this work, we explore the challenges that arise in designing secure and private content sharing for multi-user AR. Wemore »analyze representative application case studies and systematize design goals for security and functionality that a multi-user AR platform should support. We design an AR content sharing control module that achieves these goals and build a prototype implementation (ShareAR) for the HoloLens. This work builds foundations for secure and private multi-user AR interactions.« less